More Web Proxy on the site http://driver.im/

Article

Semantic facets: an in-depth analysis of a semantic image retrieval system

Authors:

Jonathon S. Hare,

Peter G. B. Enser,

Christine J. SandomAuthors Info & Claims

CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval

Pages 250 - 257

https://doi.org/10.1145/1282280.1282320

Published: 09 July 2007 Publication History

Abstract

This paper introduces a faceted model of image semantics which attempts to express the richness of semantic content interpretable within an image. Using a large image data-set from a museum collection the paper shows how the facet representation can be applied. The second half of the paper describes our semantic retrieval system, and demonstrates its use with the museum image collection. A retrieval evaluation is performed using the system to investigate how the retrieval performance varies with respect to each of the facet categories. A number of factors related to the image dataset that affect the quality of retrieval are also discussed.

References

[1]

L. H. Armitage and P. G. B. Enser. Analysis of user need in image archives. Journal of Information Sciences, 23(4):287--299, 1997.

[2]

K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. J. Mach. Learn. Res., 3:1107--1135, 2003.

Digital Library

[3]

M. W. Berry, S. T. Dumais, and G. W. O'Brien. Using linear algebra for intelligent information retrieval. Technical Report UT-CS-94-270, University of Tennessee, 1994.

Digital Library

[4]

S. C. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41(6):391--407, 1990.

[5]

P. Duygulu, K. Barnard, J. F. G. de Freitas, and D. A. Forsyth. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In ECCV '02: Proceedings of the 7th European Conference on Computer Vision-Part IV, pages 97--112, London, UK, 2002. Springer-Verlag.

Digital Library

[6]

P. G. B. Enser, C. J. Sandom, and P. H. Lewis. Automatic annotation of images from the practitioner perspective. In Leow et al. {19}, pages 497--506.

[7]

P. G. B. Enser, C. J. Sandom, and P. H. Lewis. Surveying the reality of semantic image retrieval. In S. Bres and R. Laurini, editors, VISUAL 2005, volume 3736 of LNCS, pages 177--188, Amsterdam, Netherlands, July 2005. Springer.

Digital Library

[8]

S. L. Feng, R. Manmatha, and V. Lavrenko. Multiple bernoulli relevance models for image and video annotation. In CVPR (2), pages 1002--1009, 2004.

Digital Library

[9]

W. I. Grosky and R. Zhao. Negotiating the semantic gap: From feature maps to semantic landscapes. Lecture Notes in Computer Science, 2234:33, 2001.

Digital Library

[10]

J. S. Hare. Saliency for Image Description and Retrieval. PhD thesis, University of Southampton, 2005.

[11]

J. S. Hare and P. H. Lewis. Salient regions for query by image content. In P. G. B. Enser, Y. Kompatsiaris, N. E. O'Connor, A. F. Smeaton, and A. W. M. Smeulders, editors, Image and Video Retrieval: Third International Conference, CIVR 2004, Dublin, Ireland, July 21--23, 2004. Proceedings, volume 3115 of Lecture Notes in Computer Science, pages 317--325. Springer, 2004.

[12]

J. S. Hare and P. H. Lewis. On image retrieval using salient regions with vector-spaces and latent semantics. In Leow et al. {19}, pages 540--549.

[13]

J. S. Hare and P. H. Lewis. Saliency-based models of image content and their application to auto-annotation by semantic propagation. In Proceedings of the Second European Semantic Web Conference (ESWC2005), Heraklion, Crete, May 2005.

[14]

J. S. Hare, P. H. Lewis, P. G. B. Enser, and C. J. Sandom. A Linear-Algebraic Technique with an Application in Semantic Image Retrieval. In H. Sundaram, M. Naphade, J. R. Smith, and Y. Rui, editors, Image and Video Retrieval, 5th International Conference, CIVR 2006, Tempe, AZ, USA, July 2006, Proceedings, volume 4071 of Lecture Notes in Computer Science, pages 31--40. Springer, 2006.

Digital Library

[15]

J. S. Hare, P. H. Lewis, P. G. B. Enser, and C. J. Sandom. Mind the gap. In E. Y. Chang, A. Hanjalic, and N. Sebe, editors, Multimedia Content Analysis, Management, and Retrieval 2006, volume 6073, pages 607309-1-607309-12, San Jose, California, USA, January 2006. SPIE.

[16]

A. Jaimes and S. F. Chang. A conceptual framework for indexing visual information at multiple levels. In IS&T/SPIE Internet Imaging, volume 3964, San Jose, California, USA, January 2000.

[17]

J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 119--126, New York, NY, USA, 2003. ACM Press.

Digital Library

[18]

T. K. Landauer and M. L. Littman. Fully automatic cross-language document retrieval using latent semantic indexing. In Proceedings of the Sixth Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research, pages 31--38, UW Centre for the New OED and Text Research, Waterloo, Ontario, Canada, October 1990.

[19]

W. K. Leow, M. S. Lew, T.-S. Chua, W.-Y. Ma, L. Chaisorn, and E. M. Bakker, editors. Image and Video Retrieval, 4th International Conference, CIVR 2005, Singapore, July 20--22, 2005, Proceedings, volume 3568 of Lecture Notes in Computer Science. Springer, 2005.

[20]

D. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91--110, January 2004.

Digital Library

[21]

K. Mikolajczyk. Detection of local features invariant to affine transformations. PhD thesis, Institut National Polytechnique de Grenoble, France, 2002.

[22]

K. Mikolajczyk and C. Schmid. A performance evaluation of local descriptors. In International Conference on Computer Vision & Pattern Recognition, volume 2, pages 257--263, June 2003.

[23]

F. Monay and D. Gatica-Perez. On image auto-annotation with latent space models. In MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia, pages 275--278. ACM Press, 2003.

Digital Library

[24]

Y. Mori, H. Takahashi, and R. Oka. Image-to-word transformation based on dividing and vector quantizing images with words. In Proceedings of the First International Workshop on Multimedia Intelligent Storage and Retrieval Management (MISRM'99), 1999.

[25]

E. Rosch, C. Mervis, W. Gray, D. Johnson, and P. Boyes-Braem. Basic objects in natural categories. Cognitive Psychology, 8(3):382--439, 1976.

[26]

N. Sebe, Q. Tian, E. Loupias, M. Lew, and T. Huang. Evaluation of salient point techniques. Image and Vision Computing, 21:1087--1095, 2003.

[27]

S. Shatford. Analyzing the subject of a picture: a theoretical approach. Cataloguing & Classification Quarterly, 5(3):39--61, 1986.

[28]

S. Shatford Layne. Some issues in the indexing of images. Journal of the American Society for Information Science, 45(8):583--588, 1994.

Digital Library

[29]

J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. In International Conference on Computer Vision, pages 1470--1477, October 2003.

Digital Library

[30]

A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000.

Digital Library

[31]

University of Washington. Ground truth image database. http://www.cs.washington.edu/research/imagedatabase/groundtruth/, Accessed 6/11/2003.

Cited By

Naeimi Sadigh ABahraini TSadoghi Yazdi H(2024)Semantic labeling of social big media using distributed online robust classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107928132(107928)Online publication date: Jun-2024
https://doi.org/10.1016/j.engappai.2024.107928
Alzboon MAlqaraleh MAljarrah EAlomari S(2022)Semantic Image Analysis on Social Networks and Data ProcessingHandbook of Research on Foundations and Applications of Intelligent Business Analytics10.4018/978-1-7998-9016-4.ch009(189-214)Online publication date: 2022
https://doi.org/10.4018/978-1-7998-9016-4.ch009
Bayrakdar SYucedag ISimsek MDogru I(2020)Semantic analysis on social networks: A surveyInternational Journal of Communication Systems10.1002/dac.442433:11Online publication date: 16-Apr-2020
https://doi.org/10.1002/dac.4424
Show More Cited By

Index Terms

Semantic facets: an in-depth analysis of a semantic image retrieval system
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Evaluation of retrieval results

Recommendations

Semantic Image Retrieval Using Region Based Inverted File
DICTA '09: Proceedings of the 2009 Digital Image Computing: Techniques and Applications

Image data is as common as textual data in this digital world. There is an urgent demand of image management tools as efficient as those text search engines. Decades of research on image retrieval has found there is a significant gap between the ...
Content-Based Image Retrieval and the Semantic Gap in the Deep Learning Era
Pattern Recognition. ICPR International Workshops and Challenges
Abstract
Content-based image retrieval has seen astonishing progress over the past decade, especially for the task of retrieving images of the same object that is depicted in the query image. This scenario is called instance or object retrieval and ...
Fast semantic image retrieval based on random forest
MM '12: Proceedings of the 20th ACM international conference on Multimedia

This paper introduces random forest as a computational and data structure paradigm for fusing low-level visual features and high-level semantic concepts for image retrieval. We use visual features to split the tree nodes and use the image labels to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval

July 2007

655 pages

ISBN:9781595937339

DOI:10.1145/1282280

General Chairs:
Nicu Sebe
Univ. of Amsterdam, The Netherlands
,
Marcel Worring
Univ. of Amsterdam, The Netherlands

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CIVR07

Sponsor:

SIGMM

CIVR07: International Conference on Image and Video Retrieval 2007

July 9 - 11, 2007

Amsterdam, The Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
517
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Naeimi Sadigh ABahraini TSadoghi Yazdi H(2024)Semantic labeling of social big media using distributed online robust classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107928132(107928)Online publication date: Jun-2024
https://doi.org/10.1016/j.engappai.2024.107928
Alzboon MAlqaraleh MAljarrah EAlomari S(2022)Semantic Image Analysis on Social Networks and Data ProcessingHandbook of Research on Foundations and Applications of Intelligent Business Analytics10.4018/978-1-7998-9016-4.ch009(189-214)Online publication date: 2022
https://doi.org/10.4018/978-1-7998-9016-4.ch009
Bayrakdar SYucedag ISimsek MDogru I(2020)Semantic analysis on social networks: A surveyInternational Journal of Communication Systems10.1002/dac.442433:11Online publication date: 16-Apr-2020
https://doi.org/10.1002/dac.4424
Yoon J(2014)Image Retrieval Practice and ResearchEncyclopedia of Information Science and Technology, Third Edition10.4018/978-1-4666-5888-2.ch587(5937-5946)Online publication date: 31-Jul-2014
https://doi.org/10.4018/978-1-4666-5888-2.ch587
Komamizu TKamie MFukui KAmagasa TKitagawa H(2012)A Scheme of Fragment-Based Faceted Image SearchDatabase and Expert Systems Applications10.1007/978-3-642-32597-7_40(450-457)Online publication date: 2012
https://doi.org/10.1007/978-3-642-32597-7_40
Sandhaus PBoll S(2011)Semantic analysis and retrieval in personal and social photo collectionsMultimedia Tools and Applications10.1007/s11042-010-0673-151:1(5-33)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.1007/s11042-010-0673-1
Yoon JChung E(2011)Understanding image needs in daily life by analyzing questions in a social Q&A siteJournal of the American Society for Information Science and Technology10.1002/asi.2163762:11(2201-2213)Online publication date: 1-Nov-2011
https://dl.acm.org/doi/10.1002/asi.21637
van Zwol RSigurbjornsson BAdapala RGarcia Pueyo LKatiyar AKurapati KMuralidharan MMuthu SMurdock VNg PRamani ASahai ASathish SVasudev HVuyyuru URappa MJones PFreire JChakrabarti S(2010)Faceted exploration of image search resultsProceedings of the 19th international conference on World wide web10.1145/1772690.1772788(961-970)Online publication date: 26-Apr-2010
https://dl.acm.org/doi/10.1145/1772690.1772788
Ivasic-Kos MIpsic IRibaric S(2010)Image annotation using fuzzy knowledge representation scheme2010 International Conference of Soft Computing and Pattern Recognition10.1109/SOCPAR.2010.5686618(407-412)Online publication date: Dec-2010
https://doi.org/10.1109/SOCPAR.2010.5686618
Yoon JO'Connor B(2010)Engineering an image‐browsing environment: re‐purposing existing denotative descriptorsJournal of Documentation10.1108/0022041101106682666:5(750-774)Online publication date: 7-Sep-2010
https://doi.org/10.1108/00220411011066826
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents