More Web Proxy on the site http://driver.im/

research-article

Real-world vision: Selective perception and task

Author:

Roxanne L. CanosaAuthors Info & Claims

ACM Transactions on Applied Perception (TAP), Volume 6, Issue 2

Article No.: 11, Pages 1 - 34

https://doi.org/10.1145/1498700.1498705

Published: 10 March 2009 Publication History

Abstract

Visual perception is an inherently selective process. To understand when and why a particular region of a scene is selected, it is imperative to observe and describe the eye movements of individuals as they go about performing specific tasks. In this sense, vision is an active process that integrates scene properties with specific, goal-oriented oculomotor behavior. This study is an investigation of how task influences the visual selection of stimuli from a scene. Four eye tracking experiments were designed and conducted to determine how everyday tasks affect oculomotor behavior. A portable eyetracker was created for the specific purpose of bringing the experiments out of the laboratory and into the real world, where natural behavior is most likely to occur. The experiments provide evidence that the human visual system is not a passive collector of salient environemental stimuli, nor is vision general-purpose. Rather, vision is active and specific, tightly coupled to the requirements of a task and a plan of action. The experiments support the hypothesis that the purpose of selective attention is to maximize task efficiency by fixating relevant objects in the scene. A computational model of visual attention is presented that imposes a high-level constraint on the bottom-up salient properties of a scene for the purpose of locating regions that are likely to correspond to foreground objects rather than background or other salient nonobject stimuli. In addition to improving the correlation to human subject fixation densities over a strictly bottom-up model [Itti et al. 1998; Parkhurst et al. 2002], this model predicts a central fixation tendency when that tendency is warranted, and not as an artificially primed location bias.

References

[1]

Andrews, T. J. and Coppola, D. M. 1999. Idiosyncratic characteristics of saccadic eye movements when viewing different visual environments. Vis. Res. 39, 2947--2953.

[2]

Arrington, K. F. 2000. History of eye tracking methods. http://www.arringtonresearch.com.

[3]

Ballard, D. H., Hayhoe, M. M., and Pelz, J. B. 1995. Memory representations in natural tasks. J. Cogn. Neurosci. 7, 1, 66--80.

Digital Library

[4]

Bonaiuto, J. J. and Itti, L. 2005. Combining attention and recognition for rapid scene analysis. In Proceedings of IEEE-CVPR Workshop on Attention and Performance in Computer Vision (WAPCV '05). IEEE, Los Alamitos, CA, 1--6.

Digital Library

[5]

Bruce, N. and Tsotsos, J. 2006. Saliency based on information maximization. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS '05). 155--162.

[6]

Burt, P. J. and Adelson, E. H. 1983. The laplacian pyramimd as a compact image code. IEEE Trans. Comm. 31, 4, 532--540.

[7]

Buswell, G. T. 1935. How People Look at Pictures: A Study of the Psychology of Perception in Art. University of Chicago Press, Chicago.

[8]

Canny, F. F. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Vis. 8, 769--798.

Digital Library

[9]

Canosa, R. L. 2005. Modeling selective perception of complex natural scenes. Int. J. AI. 14, 1(2), 233--260.

[10]

Carmi, R. and Itti, L. 2006. Visual causes versus correlates of attentional selection in dynamic scenes. Vis. Res. 46, 26, 4333--4345.

[11]

Carpenter, D. H. S. 1988. Movements of the Eyes. Pion, London.

[12]

Chapman, D. 1991. Vision, Instruction, and Action. MIT Press, Cambridge, MA.

Digital Library

[13]

Desimone, R. and Duncan, J. 1995. Neural mechanisms of selective visual attention. Ann. Rev. Neurosci. 18, 193--222.

[14]

Egeth, H. E. and Yantis, S. 1997. Visual attention: Control, representation, and time course. Ann. Rev. Psych. 48, 269--297.

[15]

Epelboim, J. and Suppes, P. 2001. A model of eye movements and visual working memory during problem solving in geometry. Vis. Res. 41, 1561--1574.

[16]

Fairchild, M. D. 1998. Color Appearance Models. Addison-Wesley, Reading, MA.

[17]

Fecteau, J. H. Andmunoz, D. P. 2006. Salience, relevance, and firing: A priority map for target selection. TRENDS Cogn. Sci. 10, 8, 382--390.

[18]

Findlay, J. M. and Gilchrist, I. D. 2003. Active Vision: The Psychology of Looking and Seeing. Oxford University Press, Oxford, UK.

[19]

Gabor, D. 1946. Theory of communication. IEEE Proc. 93, 429--441.

[20]

Godijn, R. and Theeuwes, J. 2002. Programming of endogenous and exogenous saccades: Evidence for a competitive integration model. J. Exper. Psych. Hum. Percep. Perform. 28, 5, 1039--1054.

[21]

Goldberg, M. E., Eggers, H. M., and Gouras, P. 1991. The ocular motor system. In Principles of Neural Science 3rd Ed. Appleton & Lange, Norwalk, CT, 660--679.

[22]

Gottlieb, J. P., Kusunoki, M., and Goldberg, M. E. 1998. The representation of visual salience in monkey parietal cortex. Nature 391, 29, 481--484.

[23]

Harnad, S. 1990. The symbol grounding problem. Physica D 42, 335--346.

Digital Library

[24]

Hayhoe, M. M. 2000. Vision using routines: A functional account of vision. Vis. Cogn. 7, 43--64.

[25]

Hayhoe, M. M., Ballard, D. H., and Pelz, J. B. 1994. Visual representations in natural tasks. In Proceedings of the Workshop on Visual Behaviors. IEEE, Los Alamitos, CA.

[26]

Hebb, D. O. 1949. The Organization of Behavior. John Wiley & Sons, New York.

[27]

Hubel, D. H. and Wiesel, T. N. 1968. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195, 215--243.

[28]

Hunt, R. W. G. 1995. The Reproduction of Color 5th ed. Fountain Press, Kingston-Upon-Thames, England.

[29]

Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Patt. Anal. Mach. Intell. 20, 11, 1254--1259.

Digital Library

[30]

Itti, L., Rees, G., and J. K. Tsotsos, E. 2005. Neurobiology of Attention. Elsevier, San Diego, CA.

[31]

Ivanov, Y. and Bobick, A. 2000. Recognition of visual activities and interactions by stochastic parsing. IEEE Trans. Patt. Anal. Mach. Intell. 22, 8, 852--872.

Digital Library

[32]

Kaufman, L. and Richards, W. 1969. Spontaneous fixation tendencies for visual forms. Percept. Psychophys. 5, 2, 85--88.

[33]

Kowler, E. 1995. Eye movements. In Visual Cognition 39 2nd Ed. MIT Press, Cambridge, MA.

[34]

Land, M. L., Mennie, N., and Rusted, J. 1999. The roles of vision and eye movements in the control of activities of daily living. Perception 28, 1311--1328.

[35]

LeMeur, O., Callet, P., Barba, D., and Thoreau, D. 2005. A coherent computational approach to model bottom-up visual attention. Trans. Patt. Anal. Mach. Intell. 28, 5, 802--817.

Digital Library

[36]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2, 91--110.

Digital Library

[37]

Magee, D., Needham, C. J., Santos, P., Cohn, A. G., and Hogg, D. C. 2004. Autonomous learning for a cognitive agent using continuous models and inductive logic programming from audio-visual input. In Proceedings of the Workshop on Anchoring Symbols to Sensor Data 4 (AAI-04). AAAI, Menlo Park, CA, 17--24.

[38]

Manno, J. L. and Sakrison, D. J. 1974. The effects of a visual fidelity criterion on the encoding of images. IEEE Trans. Inform. Theory 20, 4, 525--535.

[39]

Milanese, R., Wechsler, H., Gil, S., Bost, J., and Pun, T. 1994. Integration of bottom-up and top-down cues for visual attention using non-linear relaxation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Los Alamitos, CA, 781--785.

[40]

Moran, J. and Desimone, R. 1985. Selective attention gates visual processing in the extrastriate cortex. Science 229, 782--784.

[41]

Morgan, M. J., Hole, G. J., and Glennerstner, A. 1990. Biases and sensitivities in geometrical illusions. Vis. Res. 30, 11, 1793--1810.

[42]

Navalpakkam, V. and Itti, L. 2002. A goal oriented attention guidance model. In Proceedings of the 2nd Workshop on Biologically Motivated Computer Vision. Lecture Notes in Computer Science, Springer Verlag, Tubingen, Germany.

Digital Library

[43]

Navalpakkam, V. and Itti, L. 2006. Top-down attention selection is fine grained. J. Vis. 6, 1180--1193.

[44]

Navalpakkam, V. and Itti, L. 2007. Search goal tunes visual features optimally. Neuron 53, 4, 605--617.

[45]

Northdurft, H. C. 2005. Salience of feature contrast. Neurobiol Attention, 233--239.

[46]

Noton, D. and Stark, L. 1971. Scan paths and saccadic eye movements while viewing and recognizing patterns. Vis. Res. 11, 929--942.

[47]

Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. MIT Press, Cambridge, MA.

[48]

Parkhurst, D., Law, K., and Niebur, E. 2002. Modeling the role of salience in the allocation of overt visual attention. Vis. Res. 42, 1, 107--123.

[49]

Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., and Greenberg, D. P. 1998. A multi-scale model of adaptation and spatial vision for realistic image display. In Proceedings of the ACM SIGGRAPH. ACM, New York, 287--298.

Digital Library

[50]

Pelz, J. B. and Canosa, R. L. 2001. Oculomotor behavior and perceptual strategies in complex tasks. Vis. Res. 41, 3587--3596.

[51]

Pelz, J. B., Canosa, R. L., and Babcock, J. 2000. Extended tasks elicit complex eye movement patterns. In Proceedings of the ACM SIGCHI Eye-tracking Research and Applications Symposium. ACM, New York.

Digital Library

[52]

Peters, R. J. and Itti, L. 2007. Beyond bottom-up: Incorporating task-dependent influences into a computational model of spatial attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Los Alamitos, CA, 1--8.

[53]

Peters, R. J. and Itti, L. 2008. Applying computational tools to predict gaze direction in interactive visual environments. ACM Trans. Appl. Percept. 5, 2.

Digital Library

[54]

Peters, R. J., Iyer, A., Itti, L., and Koch, C. 2005. Components of bottom-up gaze allocation in natural images. Vis. Res. 45, 18, 2397--2416.

[55]

Posner, M. I. 1980. Orienting of attention. Q. J. Exper. Psych 32, 3--25.

[56]

Privitera, C. M. and Stark, L.W. 2000. Algorithms for defining visual regions of interest: Comparison with eye fixations. IEEE Trans. Patt. Anal. Mach. Intell. 22, 970--982.

Digital Library

[57]

Rayner, K. 1983. Eye Movements in Reading: Percpetual and Language Processes. Academic Press, New York.

Digital Library

[58]

Rensink, R. A. 2000a. The dynamic representation of scenes. Vis. Cogn. 7, 17--42.

[59]

Rensink, R. A. 2000b. Seeing, sensing, and scrutinizing. Vis. Res. 40, 1469--1487.

[60]

Rensink, R. A., O'Regan, J. K., and Clark, J. J. 1995. Image flicker is as good as saccades in making large scene changes invisible. Perception 24, 26--27.

[61]

Rubin, E. 1921. Visuell Wahrgenommene Figuren. Glydenalske Boghandel, Copenhagen.

[62]

Santos, P., Magee, D., Cohn, A., and Hogg, D. 2004. Combining multiple answers for learning mathematical structures from visual observations. In Proceedings of the 16th European Conference on Artificial Intelligence 16. IOS Press, The Netherlands, 544--554.

[63]

Tatler, B. W., Baddeley, R. J., and Gilchrist, I. D. 2005. Visual correlates of fixation selection: Effects of scale and time. Vis. Res. 45, 643--659.

[64]

Theeuwes, J. 2004. Top-down search strategies cannot override attentional capture. Psychonomic Bull. Rev. 11, 65--70.

[65]

Torralba, A. 2003. Modeling global scene factors in attention. J. Opt. Soc. Am. 20, 7, 1407--1418.

[66]

Torralba, A., Oliva, A., Castelhano, M. S., and Henderson, J. M. 2006. Contextual guidance of eye movements and attention in real-world scenes: The role of global features on object search. Psych. Rev. 113, 4, 766--786.

[67]

Treisman, A. and Gelade, G. 1980. A feature integration theory of attention. Cogn. Psych. 12, 97--136.

[68]

Ullman, S. 1984. Visual routines. Cognition 18, 97--106.

[69]

van Zoest,W., Donk, M., and Theeuwes, J. 2004. The role of stimulus-driven and goal-driven control in saccadic visual selection. J. Exper. Psych. Hum. Percept. Perform. 30, 4, 746--759.

[70]

Walther, D. and Koch, C. 2006. Modeling attention to salient proto-objects. Neural Netw. 19, 1395--1407.

Digital Library

[71]

Walther, D., Rutishauser, U., Koch, C., and Perona, P. 2005. Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Comput. Vis. Image Understand. 100, 41--63.

Digital Library

[72]

Yarbus, A. 1967. Eye Movements and Vision L. A. Riggs, translator. Plenum Press, New York.

Cited By

Qing TStrauch CVan Maanen LVan der Stigchel S(2024)Shifting reliance between the internal and external world: A meta-analysis on visual-working memory usePsychonomic Bulletin & Review10.3758/s13423-024-02623-zOnline publication date: 4-Dec-2024
https://doi.org/10.3758/s13423-024-02623-z
Krongauz DAyali AKaminka G(2024)Vision-based collective motion: A locust-inspired reductionist modelPLOS Computational Biology10.1371/journal.pcbi.101179620:1(e1011796)Online publication date: 29-Jan-2024
https://doi.org/10.1371/journal.pcbi.1011796
Neuland JFox AGrüning M(2024)Picture Content in Annual Reports Matters: Framing as a Strategy for Impressing Potential InvestorsAustralian Accounting Review10.1111/auar.12435Online publication date: 20-Oct-2024
https://doi.org/10.1111/auar.12435
Show More Cited By

Index Terms

Real-world vision: Selective perception and task
1. Applied computing
  1. Law, social and behavioral sciences
    1. Psychology
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

An overview of space-variant and active vision mechanisms for resource-constrained human inspired robotic vision
Abstract
In order to explore and understand the surrounding environment in an efficient manner, humans have developed a set of space-variant vision mechanisms that allow them to actively attend different locations in the surrounding environment and ...
CeDAR: A real-world vision system: Mechanism, control and visual processing

We report on the development of a multi-purpose active visual sensor system for real-world application. The <u>C</u>abl<u>e</u>-<u>D</u>rive <u>A</u>ctive-Vision <u>R</u>obot, <i>CeDAR</i>, has been designed for use on a diverse range of platforms to ...
APES: Attentively Perceiving Robot

Robot vision systems--inspired by human-like vision--are required to employ mechanisms similar to those that have proven to be crucial in human visual performance. One of these mechanisms is attentive perception. Findings from vision science research ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Applied Perception

ACM Transactions on Applied Perception Volume 6, Issue 2

February 2009

111 pages

ISSN:1544-3558

EISSN:1544-3965

DOI:10.1145/1498700

Issue’s Table of Contents

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2009

Accepted: 01 July 2008

Revised: 01 January 2008

Received: 01 July 2007

Published in TAP Volume 6, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
1,651
Total Downloads

Downloads (Last 12 months)95
Downloads (Last 6 weeks)7

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qing TStrauch CVan Maanen LVan der Stigchel S(2024)Shifting reliance between the internal and external world: A meta-analysis on visual-working memory usePsychonomic Bulletin & Review10.3758/s13423-024-02623-zOnline publication date: 4-Dec-2024
https://doi.org/10.3758/s13423-024-02623-z
Krongauz DAyali AKaminka G(2024)Vision-based collective motion: A locust-inspired reductionist modelPLOS Computational Biology10.1371/journal.pcbi.101179620:1(e1011796)Online publication date: 29-Jan-2024
https://doi.org/10.1371/journal.pcbi.1011796
Neuland JFox AGrüning M(2024)Picture Content in Annual Reports Matters: Framing as a Strategy for Impressing Potential InvestorsAustralian Accounting Review10.1111/auar.12435Online publication date: 20-Oct-2024
https://doi.org/10.1111/auar.12435
D’Amelio APatania SBursic SCuculo VBoccignone G(2023)Using Gaze for Behavioural BiometricsSensors10.3390/s2303126223:3(1262)Online publication date: 22-Jan-2023
https://doi.org/10.3390/s23031262
Ioannou CSeernani DStefanou MBiscaldi-Schaefer MTebartz Van Elst LFleischhaker CBoccignone GKlein C(2020)Social Visual Perception Under the Eye of Bayesian Theories in Autism Spectrum Disorder Using Advanced Modeling of Spatial and Temporal ParametersFrontiers in Psychiatry10.3389/fpsyt.2020.58514911Online publication date: 23-Sep-2020
https://doi.org/10.3389/fpsyt.2020.585149
Klose EFunk JHegenberg JSchmidt L(2019)Feldstudie zur physiologischen und videogestützten Analyse von Nutzerzustand und -verhalten bei der Orientierung und Navigation am Frankfurter FlughafenField study with a physiological and video based analysis of user state and behavior during orientation and navigation at Frankfurt AirportZeitschrift für Arbeitswissenschaft10.1007/s41449-018-00145-873:1(45-57)Online publication date: 3-Jan-2019
https://doi.org/10.1007/s41449-018-00145-8
Yao ZYi W(2019)Bionic vision system and its application in license plate recognitionNatural Computing10.1007/s11047-019-09746-6Online publication date: 6-Jun-2019
https://doi.org/10.1007/s11047-019-09746-6
Boccignone G(2019)Advanced Statistical Methods for Eye Movement Analysis and Modelling: A Gentle IntroductionEye Movement Research10.1007/978-3-030-20085-5_9(309-405)Online publication date: 17-Oct-2019
https://doi.org/10.1007/978-3-030-20085-5_9
Amrouche SGollan BFerscha AHeftberger J(2018)Activity Segmentation and Identification based on Eye Gaze FeaturesProceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference10.1145/3197768.3197775(75-82)Online publication date: 26-Jun-2018
https://dl.acm.org/doi/10.1145/3197768.3197775
Coleman PCoon DKim RChung CBass RRegan BAnderson R(2017) Promoting Constructive Multicultural Attractors: Fostering Unity and Fairness From Diversity and Conflict The Journal of Applied Behavioral Science10.1177/002188631770213353:2(180-211)Online publication date: 19-Apr-2017
https://doi.org/10.1177/0021886317702133
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents