More Web Proxy on the site http://driver.im/

research-article

Using eye-tracking to assess different image retargeting methods

Authors:

Susana Castillo,

Diego GutierrezAuthors Info & Claims

APGV '11: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization

Pages 7 - 14

https://doi.org/10.1145/2077451.2077453

Published: 27 August 2011 Publication History

Abstract

Assessing media retargeting results is not a trivial issue. When resizing one image to a particular percentage of its original size, some content has to be removed, which may affect the image's original meaning and/or composition. We examine the impact of the retargeting process on human fixations, by gathering eye-tracking data for a representative benchmark of retargeted images. We compute their derived saliency maps as input to a set of computational image distance metrics. When analyzing the fixations, we found that even strong artifacts may go unnoticed for areas outside the original regions of interest. We also note that the most important alterations in semantics are due to content removal. Since using an eye tracker is not always a feasible option, we additionally show how an existing model of prediction of human fixations also works sufficiently well in a retargeting context.

References

[1]

Anstis, S. 1998. Picturing peripheral acuity. Perception 27, 817--825.

[2]

Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM TOG 28, 3.

Digital Library

[3]

Biederman, I., Mezzanotte, R. J., and Rabinowitz, J. C. 1982. Scene perception: Detecting and judging objects undergoing relational violations. Cognitive Psychology 14, 2, 143--177.

[4]

Buswell, G. T. 1935. How people look at pictures. a study of the psychology of perception in art. Chicago: University of Chicago Press.

[5]

Chamaret, C., and Le Meur, O. 2008. Attention-based video reframing: Validation using eye-tracking. In Pattern Recognition, 2008. ICPR 2008. 19th International Conference on, 1--4.

[6]

Chamaret, C., Meur, O. L., Guillotel, P., and Chevet, J. 2010. How to measure the relevance of a retargeting approach? In Workshop Media retargeting, ECCV 2010.

Digital Library

[7]

Chen, L. 2005. The topological approach to perceptual organization. Visual Cognition 12(4), 553--637.

[8]

Choi, M. J., Lim, J. J., Torralba, A., and Willsky, A. S. 2010. Exploiting hierarchical context on a large database of object categories. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]

David, H. 1963. The Method of Paired Comparisons. Charles Griffin and Company.

[10]

De Graef, P., Christiaens, D., and dYdewalle, G. 1990. Perceptual effects of scene context on object identification. Psychological Research 52, 317--329.

[11]

Ding, Y., Jing, X., and Yu, J. 2011. Importance filtering for image retargeting. In To appear in Proceedings of IEEE CVPR 2011.

Digital Library

[12]

Ehinger, K., and Oliva, A. 2011. Canonical views of scenes depend on the shape of the space. Submitted manuscript.

[13]

Greene, M., and Oliva, A. 2006. Natural scene categorization from the conjunction of ecological global properties. In Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society, 291--296.

[14]

Greene, M. R., and Oliva, A. 2009. Recognition of natural scenes from global properties: seeing the forest without representing the trees. Cognitive psychology 58, 2 (Mar.), 137--176.

[15]

Gutierrez, D., Lopez-Moreno, J., Fandos, J., Seron, F., Sanchez, M., and Reinhard, E. 2008. Depicting procedural caustics in single images. ACM Transactions on Graphics (Proc. of SIGGRAPH Asia) 27, 5, 120:1--120:9.

Digital Library

[16]

Henderson, J. M., Brockmole, J. R., Castelhano, M. S., van R. Mackand M. Fischer, M., Murray, W., and Hill, R. 2006. Visual saliency does not account for eye-movements during visual search in real-world scenes. Oxford: Elsevier.

[17]

Itti, L., and Baldi, P. 2009. Bayesian surprise attracts human attention. Vision research 49, 10 (June), 1295--1306.

[18]

Judd, T., Ehinger, K., Durand, F., and Torralba, A. 2009. Learning to predict where humans look. In IEEE International Conference on Computer Vision (ICCV).

[19]

Karni, Z., Freedman, D., and Gotsman, C. 2009. Energy-based image deformation. Computer Graphics Forum 28, 5, 1257--1268.

[20]

Katusani, E., and Yamada, A. 2001. The mpeg-7 color layout descriptor: a compact image feature description for high speed image/video segment retrieval. In In International Conference on Image Processing, 674--677.

[21]

Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28 (December), 126:1--126:10.

Digital Library

[22]

Le Meur, O., Castellan, X., Le Callet, P., and Barba, D. 2006. Efficient saliency-based repurposing method. In Image Processing, 2006 IEEE International Conference on, 421--424.

[23]

Liu, C., Yuen, J., Torralba, A., Sivic, J., and Freeman, W. 2008. Sift flow: Dense correspondence across different scenes. In ECCV, 28--42.

Digital Library

[24]

Liu, L., Chen, R., Wolf, L., and Cohen-Or, D. 2010. Optimizing photo composition. Computer Graphic Forum (Proceedings of Eurographics) 29, 2, 469--478.

[25]

Liu, Y.-J., Luo, X., Xuan, Y.-M., Chen, W.-F., and Fu, X.-L. 2011. Image retargeting quality assessment. Computer Graphics Forum (Eurographics 2011 Proceedings) 30, 2.

[26]

Loftus, G. R., and Mackworth, N. H. 1978. Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psychology: Human Perception and Performance 4, 565--572.

[27]

Manjunath, B., Ohm, J., Vasudevan, V., and Yamada, A. 2001. Color and texture descriptors. IEEE Trans. Circuits and Systems for Video Technology 11, 703--715.

Digital Library

[28]

MPEG-7. 2002. Iso/iec 15938: Multimedia content description interface.

[29]

O. Le Meur, P. L. C. 2009. What we see is most likely to be what matters: Visual attention and applications. In IEEE ICIP.

Digital Library

[30]

Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: a holistic representation of the spatial envelope. International Journal of Computer Vision 42, 145--175.

Digital Library

[31]

Oliva, A., and Torralba, A. 2006. Building the gist of a scene: The role of global image features in recognition. Progress in Brain Research: Visual perception 155, 23--26.

[32]

Oliva, A. 2009. Visual Scene Perception. Bruce Goldstein. Sage Edition.

[33]

Pearson, E., and Hartley, H. 1966. Biometrika Tables for Statisticians, vol. 3_rd ed., vol. 1. Cambridge University Press.

[34]

Pele, O., and Werman, M. 2009. Fast and robust earth movers distances. In ICCV'09.

[35]

Pritch, Y., Kav-Venaki, E., and Peleg, S. 2009. Shift-map image editing. In ICCV'09, 151--158.

[36]

Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 27, 16:1--16:9.

Digital Library

[37]

Rubinstein, M., Shamir, A., and Avidan, S. 2009. Multioperator media retargeting. ACM Transactions on Graphics (Proceedings SIGGRAPH 2009) 28, 3, 1--11.

Digital Library

[38]

Rubinstein, M., Gutierrez, D., Sorkine, O., and Shamir, A. 2010. A comparative study of image retargeting. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 29, 5.

Digital Library

[39]

Setyawan, I., and Lagendijk, R. 2004. Human perception of geometric distortions in images. In Proceedings of SPIE, Security, Steganography and Watermarking of Multimedia Contents VI, 256--267.

[40]

Shamir, A., and Sorkine, O. 2009. Visual media retargeting. In ACM SIGGRAPH ASIA 2009 Courses, ACM, SIGGRAPH ASIA '09, 11:1--11:13.

Digital Library

[41]

Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR '08.

[42]

Torralba, A., Oliva, A., Castelhano, M. S., and Henderson, J. M. 2006. Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychological Review 113, 4 (October), 766--786.

[43]

Torralba, A. 2003. Contextual priming for object detection. International Journal of Computer Vision 53, 2, 169--191.

Digital Library

[44]

V. Kadiyala, S. Pinneli, E. C. L., and Chandler, D. M. 2008. Quantifying the perceived interest of objects in images: Effects of size, location, blur, and contrast. In Proc. Human Vision and Electronic Imaging 2008.

[45]

Wang, Y.-S., Lin, H.-C., Sorkine, O., and Lee, T.-Y. 2010. Motion-based video retargeting with optimized crop-and-warp. ACM Trans. Graph. 29 (July), 90:1--90:9.

Digital Library

[46]

Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In Proceedings of the Eleventh IEEE International Conference on Computer Vision (ICCV-07).

[47]

Wu, H., Wang, Y.-S., Feng, K.-C., Wong, T.-T., Lee, T.-Y., and Heng, P.-A. 2010. Resizing by symmetry-summarization. ACM Trans. Graph. 29 (December), 159:1--159:10.

Digital Library

[48]

Yarbus, A. L. 1967. Eye movements and vision. New York: Plenum.

[49]

Yu-Shuen Wang, Chiew-Lan Tai, O. S., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. (Proceedings of ACM SIGGRAPH ASIA) 27, 5.

Digital Library

Cited By

Zhang LChen MTu BLi YXia Y(2024)Retargeting HR Aerial Photos Under Contaminated Labels With Application in Smart NavigationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.328887725:1(349-358)Online publication date: Jan-2024
https://doi.org/10.1109/TITS.2023.3288877
Alsmirat MKharsa RAlzoubi R(2024)Supervised Deep Learning for Ideal Identification of Image Retargeting TechniquesIEEE Access10.1109/ACCESS.2024.3510675(1-1)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3510675
Fan LCai SHe CLi XZheng L(2024)Geometry-Based Feature Selection and Deep Aggregation Model for Architectural Scenery Recomposition Toward EducationIEEE Access10.1109/ACCESS.2024.349220512(163724-163738)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3492205
Show More Cited By

Index Terms

Using eye-tracking to assess different image retargeting methods
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
      2. Computer vision tasks
        Scene understanding

Recommendations

Eye detection and coarse localization of pupil for video-based eye tracking systems
Abstract
A video-based eye tracking system generally captures NIR images, each of which contains one or two eyes of a subject. The subject’s point of gaze is then determined using 3D eye model and pupil centre corneal reflection technique. Eye detection ...
Eye&Head: Synergetic Eye and Head Movement for Gaze Pointing and Selection
UIST '19: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology

Eye gaze involves the coordination of eye and head movement to acquire gaze targets, but existing approaches to gaze pointing are based on eye-tracking in abstraction from head motion. We propose to leverage the synergetic movement of eye and head, and ...
Video-oculography eye tracking towards clinical applications: A review
Abstract
Most neurological diseases are usually accompanied by a broad spectrum of oculomotor alterations. Being able to record and analyze these different types of eye movements would be a valuable tool to understand the functional integrity ...
Highlights
- Invariance to head movements is one of the biggest challenges in eye tracking.
- ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

APGV '11: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization

August 2011

128 pages

ISBN:9781450308892

DOI:10.1145/2077451

Conference Chairs:
Rachel McDonnell
Trinity College Dublin
,
Simon Thorpe
Université Toulouse 3
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Diego Gutierrez
Universidad de Zaragoza
,
Martin Giese
University of Tübingen

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

APGV '11

Sponsor:

SIGGRAPH

APGV '11: ACM Symposium on Applied Perception in Graphics & Visualization 2011

August 27 - 28, 2011

Toulouse, France

Acceptance Rates

APGV '11 Paper Acceptance Rate 19 of 33 submissions, 58%;

Overall Acceptance Rate 19 of 33 submissions, 58%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
436
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)2

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang LChen MTu BLi YXia Y(2024)Retargeting HR Aerial Photos Under Contaminated Labels With Application in Smart NavigationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.328887725:1(349-358)Online publication date: Jan-2024
https://doi.org/10.1109/TITS.2023.3288877
Alsmirat MKharsa RAlzoubi R(2024)Supervised Deep Learning for Ideal Identification of Image Retargeting TechniquesIEEE Access10.1109/ACCESS.2024.3510675(1-1)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3510675
Fan LCai SHe CLi XZheng L(2024)Geometry-Based Feature Selection and Deep Aggregation Model for Architectural Scenery Recomposition Toward EducationIEEE Access10.1109/ACCESS.2024.349220512(163724-163738)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3492205
Ma P(2024)Soccer Training Optimization for Education: A Multi-Layer Architecture Simulating How Observers Understand Soccer SceneriesIEEE Access10.1109/ACCESS.2024.345272512(125510-125522)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3452725
Zhang YLiu LZhu Y(2024)Application of Multimodal Feature Selection-Based Scene Recognition for Medical EducationIEEE Access10.1109/ACCESS.2024.340968612(87934-87943)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3409686
Xu KMousavi VLiu D(2024)Retargeting Low-Resolution Aerial Imagery by Distribution-Preserving Perceptual Feature SelectionIEEE Access10.1109/ACCESS.2024.336439912(25612-25622)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3364399
Groth CScholz TCastillo STauscher JMagnor M(2023)Instant Hand Redirection in Virtual Reality Through Electrical Muscle Stimulation-Triggered Eye BlinksProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615717(1-11)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3611659.3615717
Groth CFricke SCastillo SMagnor M(2023)Wavelet-Based Fast Decoding of 360° VideosIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324708029:5(2508-2516)Online publication date: 22-Feb-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3247080
Wang GZhang LLi YSheng Y(2023)Photo Semantic Understanding and Retargeting by a Noise-Robust Regularized Topic ModelIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2023.324774516(3495-3505)Online publication date: 2023
https://doi.org/10.1109/JSTARS.2023.3247745
Woehler Lvon Estorff MCastillo SMagnor M(2022)Automatic Generation of Customized Areas of Interest and Evaluation of Observers' Gaze in Portrait VideosProceedings of the ACM on Human-Computer Interaction10.1145/35308856:ETRA(1-14)Online publication date: 13-May-2022
https://dl.acm.org/doi/10.1145/3530885
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents