More Web Proxy on the site http://driver.im/

Article

Visual feature extraction via eye tracking for saliency driven 2D/3D registration

Authors:

Adrian James Chung,

Fani Deligianni,

Guang-Zhong YangAuthors Info & Claims

ETRA '04: Proceedings of the 2004 symposium on Eye tracking research & applications

Pages 49 - 54

https://doi.org/10.1145/968363.968371

Published: 22 March 2004 Publication History

Abstract

This paper presents a new technique for extracting visual saliency from experimental eye tracking data. An eye-tracking system is employed to determine which features that a group of human observers considered to be salient when viewing a set of video images. With this information, a biologically inspired saliency map is derived by transforming each observed video image into a feature space representation. By using a feature normalisation process based on the relative abundance of visual features within the background image and those dwelled on eye tracking scan paths, features related to visual attention are determined. These features are then back projected to the image domain to determine spatial areas of interest for unseen video images. The strengths and weaknesses of the method are demonstrated with feature correspondence for 2D to 3D image registration of endoscopy videos with computed tomography data. The biologically derived saliency map is employed to provide an image similarity measure that forms the heart of the 2D/3D registration method. It is shown that by only processing selective regions of interest as determined by the saliency map, rendering overhead can be greatly reduced. Significant improvements in pose estimation efficiency can be achieved without apparent reduction in registration accuracy when compared to that of using a non-saliency based similarity measure.

References

[1]

AYOUB, A. F., SIEBERT, J. P., WRAY, D., AND MOOS, K. F. 1998. A vision-based three dimensional capture system for maxillofacial assessment and surgical planning. Brit. J. Oral Maxillofacial Surg. 36, 353-357.

[2]

DEY, D., SLOMKA, P. J., GOBBI, D. G., AND PETERS, T. M. 2000. Mixed reality merging of endoscopic images and 3-d surfaces. 796-803. In Medical Image Computing and Computer-Assisted Intervention -MICCAI 2000, Third International Conference, Pittsburgh, Pennsylvania, USA, October 11-14, 2000, Proceedings, Springer, vol. 1935 of Lecture Notes in Computer Science, 796-803.

Digital Library

[3]

DEY, D., GOBBI, D., SLOMKA, P., SURRY, K., AND PETERS, T. 2002. Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: Creating stereoscopic panoramas. IEEE Transactions on Medical Imaging 21, 1, 23-30.

[4]

EINHÄUSER, W., AND KÖNIG, P. 2003. Does luminance-contrast contribute to a saliency map for overt visual attention? European Journal of Neuroscience 17, 5, 1089-1097.

[5]

FAISAL, A. A., FISLAGE, M., POMPLUN, M., RAE, R., AND RITTER, H. 1998. Observation of human eye movements to simulate visual exploration of complex scenes. Tech. rep., University of Bielfeld.

[6]

HELFERTY, J. P., AND HIGGINS, W. E. 2001. Technique for registering 3d virtual ct images to endoscopic video. In IEEE International Conference on Image Processing, ??-??

[7]

HILL, D. L. G., BATCHELOR, P. G., HOLDEN, M., AND HAWKES, D. J. 2001. Medical image registration. Physics in Medicine and Biology 46, R1-R45.

[8]

HU, X.-P., DEMPERE-MARCO, L., AND YANG, G.-Z. 2003. Hot spot detection based on feature space representation of visual search. IEEE Transactions on Medical Imaging to appear.

[9]

ITTI, L., AND KOCH, C. 1999. A comparison of feature combination strategies for saliency-based visual attention systems. In SPIE Human Vision and Electronic Imaging IV (HVEI 1999), San Jose, CA, vol. 3644, 373-382.

[10]

ITTI, L., KOCH, C., AND NIEBUR, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 11, 1254-1259.

Digital Library

[11]

JÄGERSAND, M., 1993. Saliency maps and attention selection in scale and spatial coordinates: An information theoretic approach.

[12]

JOSKOWICZ, L. 2000. Fluoroscopy-based navigation in computer-aided orthopaedic surgery. In Proc. of the IFAC Conference on Mechatronic Systems.

[13]

K. MORI, Y. SUENAGA, J. TORIWAKI, J. HASEGAWA, K. KATAA, H. TAKABATAKE, AND H. NATORI. 2000-02. A method for tracking camera motion of real endoscope by using virtual endoscopy system. In Proc.of SPIE, 3978, 134-145.

[14]

LEVENTON, M. E., AND GRIMSON, W. E. L. 1998. Multi-modal volume registration using joint intensity distributions. Lecture Notes in Computer Science 1496, 1057-??

Digital Library

[15]

MAINTZ, J. B. A., AND VIERGEVER, M. A. 1998. A survey of medical image registration. Medical Image Analysis 2, 1, 1-36.

[16]

MORI, K., DEGUCHI, D., SUGIYAMA, J., SUENAGA, Y., TORIWAKI, J., JR., C. R. M., TAKABATAKE, H., AND NATORI, H. 2002. Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images. Medical Image Analysis 6, 321-336.

[17]

PENNEY, G. P., WEESE, J., LITTLE, J. A., DESMEDT, P., HILL, D. L. G., AND HAWKES, D. J. 1998. A comparison of similarity measures for use in 2D-3D medical image registration. Lecture Notes in Computer Science 1496, 1153-??

Digital Library

[18]

PETKOV, N., AND KRUIZINGA, P. 1997. Computational models of visual neurons specialised in the detection of periodic and aperiodic oriented visual stimuli: bar and grating cells. Biological Cybernetics 76, 2, 83-96.

[19]

PRESS, W. H., TEUKOLSKY, S. A., VETTERLING, W. T., AND FLANNERY, B. P. 1992. Numerical Recipes in C, 2nd. edition. Cambridge University Press.

Digital Library

[20]

ROCHE, A., MALANDAIN, G., PENNEC, X., AND AYACHE, N. 1998. The correlation ratio as a new similarity measure for multimodal image registration. Lecture Notes in Computer Science 1496, 1115-??

Digital Library

[21]

SHOKOUFANDEH, A., MARSIC, I., AND DICKINSON, S. J. 1999. View-based object recognition using saliency maps. Image and Vision Computing 17, 5-6 (April), 445-460.

[22]

TENDICK, F., POLLY, D., BLEZEK, D., BURGESS, J., CARIGNAN, C., HIGGINS, G., LATHAN, C., AND REINIG, K. 1999. Final Report of the Technical Requirements for Image-Guided Spine Procedures. Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University Medical Center, Georgetown University Medical Center, 2115 Wisconsin Avenue, N.W., Suite 603, Washington, DC 20007, ch.3. Operative Planning and Surgical Simulators.

[23]

TSOTSOS, J., CULHANE, S., WAI, W., LAI, Y., DAVIS, N., AND NUFLO, F. 1995. Modelling visual attention via selective tuning. Artificial Intelligence 78, 1-2 (Oct.), 507-545.

Digital Library

[24]

VIOLA, P., AND WELLS III, W. M. 1997. Alignment by maximization of mutual information. International Journal of Computer Vision.

Digital Library

[25]

WAGNER, D., WEGENKITTL, R., AND GRÖLLER, E. 2002. Endoview: A phantom study of a tracked virtual bronchoscopy. In Journal of WSCG, V. Skala, Ed., vol. 10.

[26]

WHITTED, T. 1979. An improved illumination model for shaded display. In Computer Graphics (Special SIGGRAPH '79 Issue), vol. 13, 1-14.

Digital Library

[27]

YANG, G.-Z., DEMPERE-MARCO, L., HU, X.-P., AND ROWE, A. 2002. Visual search: psychophysical models and practical applications. Image and Vision Computing 20, 4, 291-305.

Cited By

Mahfoud ELu AHancock MMarquardt NSchöning JTory M(2016)Gaze-directed Immersive Visualization of Scientific EnsemblesProceedings of the 2016 ACM Companion on Interactive Surfaces and Spaces10.1145/3009939.3009952(77-82)Online publication date: 6-Nov-2016
https://dl.acm.org/doi/10.1145/3009939.3009952
Sundstedt V(2012)Gazing at Games: An Introduction to Eye Tracking ControlSynthesis Lectures on Computer Graphics and Animation10.2200/S00395ED1V01Y201111CGR0145:1(1-113)Online publication date: 5-Mar-2012
https://doi.org/10.2200/S00395ED1V01Y201111CGR014
Lu AMaciejewski REbert D(2010)Volume composition and evaluation using eye-tracking dataACM Transactions on Applied Perception10.1145/1658349.16583537:1(1-20)Online publication date: 18-Jan-2010
https://dl.acm.org/doi/10.1145/1658349.1658353
Show More Cited By

Index Terms

Visual feature extraction via eye tracking for saliency driven 2D/3D registration
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Animation
      1. Motion capture
      2. Motion processing

Recommendations

Extraction of visual features with eye tracking for saliency driven 2D/3D registration

This paper presents a new technique for deriving information on visual saliency with experimental eye-tracking data. The strength and potential pitfalls of the method are demonstrated with feature correspondence for 2D to 3D image registration. With ...
Human Visual Scanpath Prediction Based on RGB-D Saliency
ICIGP '18: Proceedings of the 2018 International Conference on Image and Graphics Processing

Human visual perception is considered as a dynamic process of information acquisition, while the visual scanpath can clearly reflect the shift of our eye fixations. In the previous study of visual attention, researchers generally do the saliency ...
Incorporating visual field characteristics into a saliency map
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

Characteristics of the human visual field are well known to be different in central (fovea) and peripheral areas. Existing computational models of visual saliency, however, do not take into account this biological evidence. The existing models compute ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '04: Proceedings of the 2004 symposium on Eye tracking research & applications

March 2004

154 pages

ISBN:1581138253

DOI:10.1145/968363

Conference Chairs:
Andrew T. Duchowski
Clemson University
,
Roel Vertegaal
Queens University, Canada

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 March 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ETRA04

Sponsor:

ETRA04: Eye Tracking Research and Applications 2004

March 22 - 24, 2004

Texas, San Antonio

Acceptance Rates

ETRA '04 Paper Acceptance Rate 18 of 40 submissions, 45%;

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
1,354
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mahfoud ELu AHancock MMarquardt NSchöning JTory M(2016)Gaze-directed Immersive Visualization of Scientific EnsemblesProceedings of the 2016 ACM Companion on Interactive Surfaces and Spaces10.1145/3009939.3009952(77-82)Online publication date: 6-Nov-2016
https://dl.acm.org/doi/10.1145/3009939.3009952
Sundstedt V(2012)Gazing at Games: An Introduction to Eye Tracking ControlSynthesis Lectures on Computer Graphics and Animation10.2200/S00395ED1V01Y201111CGR0145:1(1-113)Online publication date: 5-Mar-2012
https://doi.org/10.2200/S00395ED1V01Y201111CGR014
Lu AMaciejewski REbert D(2010)Volume composition and evaluation using eye-tracking dataACM Transactions on Applied Perception10.1145/1658349.16583537:1(1-20)Online publication date: 18-Jan-2010
https://dl.acm.org/doi/10.1145/1658349.1658353
Liu Y(2010)Automatic Range Image Registration in the Markov ChainIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2008.28032:1(12-29)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1109/TPAMI.2008.280
Liang ZFu HChi ZFeng D(2010)Refining a region based attention model using eye tracking data2010 IEEE International Conference on Image Processing10.1109/ICIP.2010.5651804(1105-1108)Online publication date: Sep-2010
https://doi.org/10.1109/ICIP.2010.5651804
Lu AMaciejewski REbert D(2006)Volume composition using eye tracking dataProceedings of the Eighth Joint Eurographics / IEEE VGTC conference on Visualization10.5555/2384796.2384814(115-122)Online publication date: 8-May-2006
https://dl.acm.org/doi/10.5555/2384796.2384814

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents