[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Image Abstraction in Crossmedia Retrieval for Text Illustration

  • Conference paper
Advances in Information Retrieval (ECIR 2012)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 7224))

Included in the following conference series:

Abstract

Text illustration is a multimedia retrieval task that consists in finding suitable images to illustrate text fragments such as blog entries, news reports or children stories. In this paper we describe a crossmedia retrieval system which, given a textual input, selects a short list of candidate images from a large media collection. This approach makes use of a recently proposed method to map metadata and visual features into a common textual representation that can be handled by traditional information retrieval engines. Content-based analysis is enhanced by visual abstraction, namely the Anisotropic Kuwahara Filter, which impacts feature information captured by the Joint Composite and Speeded Up Robust Features visual descriptors. For evaluation purposes, we used the well-established MIRFlickr photo collection, with 25,000 photos and user tags collected from Flickr as well as manual annotations provided as image retrieval groundtruth. Results show that image abstraction can improve visual retrieval as well as significantly reduce processing and storage requirements, even more when paired with Google’s WebP image format. We conclude that applying a visual rerank after an initial text retrieval step improves the quality of results, and that the adopted text mapping method for visual descriptors provides an effective crossmedia approach for text illustration.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 35.99
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 44.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based Multimedia Information Retrieval: State of the Art and Challenges. ACM Transactions on Multimedia Computing, Communications, and Applications 2(1), 1–19 (2006)

    Article  Google Scholar 

  2. Liu, Y., Zhang, D., Lu, G., Ma, W.Y.: A Survey of Content-based Image Retrieval with High-level Semantics. Pattern Recognition 40(1), 262–282 (2007)

    Article  MATH  Google Scholar 

  3. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image Retrieval: Ideas, Influences, and Trends of the New Age. ACM Computing Surveys (CSUR) 40(2), 1–60 (2008)

    Article  Google Scholar 

  4. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence (2000)

    Google Scholar 

  5. Joshi, D., Wang, J.Z., Li, J.: The Story Picturing Engine: Finding Elite Images to Illustrate a Story Using Mutual Reinforcement. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 119–126. ACM, New York (2004)

    Chapter  Google Scholar 

  6. Joshi, D., Wang, J.Z., Li, J.: The Story Picturing Engine—a System for Automatic Text Illustration. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 2(1), 89 (2006)

    Google Scholar 

  7. Delgado, D., Magalhães, J., Correia, N.: Assisted news reading with automated illustration. In: Proceedings of the International Conference on Multimedia, MM 2010, pp. 1647–1650. ACM, New York (2010)

    Chapter  Google Scholar 

  8. Coelho, F., Ribeiro, C.: Automatic Illustration with Cross-media Retrieval in Large-scale Collections. In: Proceedings of the 9th International Workshop on Content-based Multimedia Indexing (2011)

    Google Scholar 

  9. Coelho, F., Ribeiro, C.: Dpikt: Automatic Illustration System for Media Content. In: Proceedings of the 9th International Workshop on Content-Based Multimedia Indexing (2011)

    Google Scholar 

  10. Chatzichristofis, S.A., Arampatzis, A., Boutalis, Y.S.: Investigating the Behavior of Compact Composite Descriptors in Early Fusion, Late Fusion and Distributed Image Retrieval. Radioengineering 19(4), 725 (2010)

    Google Scholar 

  11. Coelho, F., Ribeiro, C.: Evaluation of Global Descriptors for Multimedia Retrieval in Medical Applications. In: Proceedings of the 4th International Workshop on Management and Interaction with Multimodal Information Content. IEEE Computer Society (2010)

    Google Scholar 

  12. Deselaers, T., Keysers, D., Ney, H.: Features for Image Retrieval: An Experimental Comparison. Information Retrieval 11(2), 77–107 (2008)

    Article  Google Scholar 

  13. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  14. Evans, C.: Notes on the OpenSURF Library. Technical Report CSTR-09-001, University of Bristol (January 2009)

    Google Scholar 

  15. Kyprianidis, J.E., Kang, H., Döllner, J.: Image and Video Abstraction by Anisotropic Kuwahara Filtering. Computer Graphics Forum 28(7), 1955–1963 (2009); Special issue on Pacific Graphics 2009

    Article  Google Scholar 

  16. Kyprianidis, J.E., Kang, H., Döllner, J.: Anisotropic Kuwahara Filtering on the GPU. In: Engel, W. (ed.) GPU Pro - Advanced Rendering Techniques, pp. 247–264 (2010)

    Google Scholar 

  17. Gennaro, C., Amato, G., Bolettieri, P., Savino, P.: An Approach to Content-Based Image Retrieval Based on the Lucene Search Engine Library. In: Lalmas, M., Jose, J., Rauber, A., Sebastiani, F., Frommholz, I. (eds.) ECDL 2010. LNCS, vol. 6273, pp. 55–66. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  18. Amato, G., Bolettieri, P., Falchi, F., Gennaro, C., Rabitti, F.: Combining Local and Global Visual Feature Similarity Using a Text Search Engine. In: Proceedings of the 9th International Workshop on Content-based Multimedia Indexing, pp. 49–54. IEEE (2011)

    Google Scholar 

  19. Amato, G., Savino, P.: Approximate Similarity Search in Metric Spaces Using Inverted Files. In: Proceedings of the 3rd International Conference on Scalable Information Systems, p. 28. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) (2008)

    Google Scholar 

  20. Esuli, A.: PP-Index: Using Permutation Prefixes for Efficient and Scalable Approximate Similarity Search. In: Proceedings of the 7th Workshop on Large-Scale and Distributed Systems for Information Retrieval (2009) ISSN: 1613–0073

    Google Scholar 

  21. Arthur, D., Vassilvitskii, S.: k-means++: The Advantages of Careful Seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)

    Google Scholar 

  22. Huiskes, M.J., Lew, M.S.: The MIR Flickr Retrieval Evaluation. In: MIR 2008: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval. ACM, New York (2008)

    Google Scholar 

  23. Huiskes, M.J., Thomee, B., Lew, M.S.: New Trends and Ideas in Visual Concept Detection: The MIR Flickr Retrieval Evaluation Initiative. In: Proceedings of the International Conference on Multimedia Information Retrieval, pp. 527–536. ACM (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Coelho, F., Ribeiro, C. (2012). Image Abstraction in Crossmedia Retrieval for Text Illustration. In: Baeza-Yates, R., et al. Advances in Information Retrieval. ECIR 2012. Lecture Notes in Computer Science, vol 7224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28997-2_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28997-2_28

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28996-5

  • Online ISBN: 978-3-642-28997-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics