Abstract
One of the main challenges in content-based image retrieval still remains to bridge the gap between low-level features and semantic information. In this paper, we present our first results concerning a medical image retrieval approach using a semantic medical image and report indexing within a fusion framework, based on the Unified Medical Language System (UMLS) metathesaurus. We propose a structured learning framework based on Support Vector Machines to facilitate modular design and extract medical semantics from images. We developed two complementary visual indexing approaches within this framework: a global indexing to access image modality, and a local indexing to access semantic local features. Visual indexes and textual indexes – extracted from medical reports using MetaMap software application – constitute the input of the late fusion module. A weighted vectorial norm fusion algorithm allows the retrieval system to increase its meaningfulness, efficiency and robustness. First results on the CLEF medical database are presented. The important perspectives of this approach in terms of semantic query expansion and data-mining are discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Abe, H., MacMahon, H., Engelmann, R., Li, Q., Shiraishi, J., Katsuragawa, S., Aoyama, M., Ishida, T., Ashizawa, K., Metz, C.E., Doi, K.: Computer-aided diagnosis in chest radiography: Results of large-scale observer tests at the 1996- 2001 rsna scientific assemblies. RadioGraphics 23(1), 255–265 (2003)
Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. Journal of machine learning research 3, 1107–1135 (2003)
Barnard, K., Forsyth, D.: Learning the semantics of words and pictures. In: Proceedings of the International Conference on Computer Vision, vol. 2, pp. 408–415 (2001)
Bui, A.A.T., Taira, R.K., Dionision, J.D.N., Aberle, D.R., El-Saden, S., Kangarloo, H.: Evidence-based radiology. Academic Radiology 9(6), 662–669 (2002)
Chang, N.-S., Fu, K.-S.: Query-by-pictorial-example. IEEE Transactions on Software Engineering 6(6), 519–524 (1980)
Chu, W.W., Alfonso, F.C., Ricky, K.T.: Knowledge-based image retrieval with spatial and temporal constructs. IEEE Transactions on Knowledge and Data Engineering 10, 872–888 (1998)
Fleury, C.: Apports reciproques des informations textuelles et visuelles par analyse de la semantique latente pour la recherche d’information. Master’s thesis, Intelligence, Interaction and Information, CLIPS-MRIM Grenoble, France, IPAL UMI CNRS 2955, Singapore (June 2006)
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P.: Query by image and video content: The qbic system. IEEE Computer 28(9), 23–32 (1995)
Guld, M.O., Kohnen, M., Keysers, D., Schubert, H., Wein, B.B., Bredno, J., Lehmann, T.M.: Quality of dicom header information for image categorization. In: Proceedings of the International Symposium on Medical Imaging, vol. 4685, pp. 280–287. SPIE, San Jose (2002)
Kahn, C.E.: Artificial intelligence in radiology: Decision support systems. RadioGraphics 14, 849–861 (1994)
Korn, P., Sidiropoulos, N., Faloutsos, C., Siegel, E., Protopapas, Z.: Fast and effective retrieval of medical tumor shapes. IEEE Transactions on Knowledge and Data Engineering 10, 889–904 (1998)
LeBozec, C., Jaulent, M.-C., Zapletal, E., Degoulet, P.: Unified modeling language and design of a case-based retrieval system in medical imaging. In: Proceedings of the Annual Symposium of the American Society for Medical Informatics, Nashville, TN, USA (1998)
Lehmann, T.M., Gld, M.O., Thies, C., Fischer, B., Spitzer, K., Keysers, D., Ney, H., Kohnen, M., Schubert, H., Wein, B.B.: Content-based image retrieval in medical application. Methods of Information in Medicine 43(4), 354–361 (2004)
Lim, J., Chevallet, J.P.: Vismed: a visual vocabulary approach for medical image indexing and retrieval. In: Proceedings of the Asia Information Retrieval Symposium, pp. 84–96 (2005)
Lim, J., Jin, J.: A structured learning framework for content-based image indexing and visual query. Multimedia Systems 10, 317–331 (2005)
Muller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content-based image retrieval systems in medical applications - clinical benefits and future directions. International Journal of Medical Informatics 73, 1–23 (2004)
Niblack, W., Barber, R., Equitz, W., Flickner, M.D., Glasman, E.H., Petkovic, D., Yanker, P., Faloutsos, C., Taubin, G.: QBICproject: querying images by content, using color, texture, and shape. In: Niblack, W. (ed.) Storage and Retrieval for Image and Video Databases, vol. 1908, pp. 173–187. SPIE, San Jose (1993)
Pentland, A., Picard, R.W., Sclaroff, S.: Photobook: Tools for content-based manipulation of image databases. International Journal of Computer Vision 18, 233–254 (1996)
Sclaroff, S., la Cascia, M., Sethi, S.: Unifyng textual and visual cues for contentbased image retrieval on the world wide web. Computer Vision and Image Understanding 75(1/2), 86–98 (1998)
Shyu, C.-R., Brodley, C.E., Kak, A.C., Kosaka, A., Aisen, A.M., Broderick, L.S.: ASSERT: A physician-in-the-loop content-based retrieval system for HRCT image databases. Computer Vision and Image Understanding 75, 111–132 (1999); Special issue on content-based access for image and video libraries
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Contentbased image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1349–1380 (2000)
Westerveld, T.: Image retrieval: Content versus context. In: Recherche d’Information Assistee par Ordinateur (2000)
Zhao, R., Grosky, W.: Narrowing the semantic gap - improved text-based web document retrieval using visual features. IEEE Transactions on Multimedia 4(2), 189–200 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Racoceanu, D., Lacoste, C., Teodorescu, R., Vuillemenot, N. (2006). A Semantic Fusion Approach Between Medical Images and Reports Using UMLS. In: Ng, H.T., Leong, MK., Kan, MY., Ji, D. (eds) Information Retrieval Technology. AIRS 2006. Lecture Notes in Computer Science, vol 4182. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11880592_35
Download citation
DOI: https://doi.org/10.1007/11880592_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45780-0
Online ISBN: 978-3-540-46237-8
eBook Packages: Computer ScienceComputer Science (R0)