research-article

Open access

Face Shows Your Intention: Visual Search Based on Full-face Gaze Estimation with Channel-spatial Attention

Authors:

Yu ShiAuthors Info & Claims

ICIAI '21: Proceedings of the 2021 5th International Conference on Innovation in Artificial Intelligence

Pages 76 - 81

https://doi.org/10.1145/3461353.3461362

Published: 04 September 2021 Publication History

All formats PDF

References

[1]

H. Sattar, S. Müller, M. Fritz, and A. Bulling. Prediction of search targets from fixations in open-world settings. In 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 981-990.

Google Scholar

[2]

Julian Steil, Philipp Müller, Yusuke Sugano, and Andreas Bulling. Forecasting User Attention During Everyday Mobile Interactions Using Device-integrated and Wearable Sensors. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (Barcelona, Spain) (MobileHCI’18). ACM, New York, NY, USA, Article 1, 13 pages.

Google Scholar

[3]

Chienming Huang and Bilge Mutlu. Anticipatory robot control for efficient human-robot collaboration. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, Piscataway, NJ, USA, 83-90.

Google Scholar

[4]

Takumi Toyama and Daniel Sonntag. Towards Episodic Memory Support for Dementia Patients by Recognizing Objects, Faces and Text in Eye Gaze. In KI 2015: Advances in Artificial Intelligence, Steffen Hölldobler, Rafael Peñaloza, and Sebastian Rudolph (Eds.). Springer International Publishing, Cham, 316-323.

Google Scholar

[5]

Daniel Sonntag. 2015. Kognit: Intelligent Cognitive Enhancement Technology by Cognitive Models and Mixed Reality for Dementia Patients. In 2015 AAAI Fall Symposia, Arlington, Virginia, USA, November 12-14, 2015. AAAI Press, 47-53.

Google Scholar

[6]

Jan Zacharias, Michael Barz, and Daniel Sonntag. A Survey on Deep Learning Toolkits and Libraries for Intelligent User Interfaces. CoRR (mar 2018). arXiv:1803.04818.

Google Scholar

[7]

Sharon. Oviatt, Björn Schuller, Philip R. Cohen, Daniel Sonntag, Gerasimos Potamianos, and Antonio Krüger (Eds.). 2017. The Handbook of Multimodal-Multisensor Interfaces: Foundations, User Modeling, and Common Modality Combinations (volume 1 ed.). Association for Computing Machinery and Morgan & Claypool, New York, NY, USA. 662 pages.

Google Scholar

[8]

Daniel Sonntag. Collaborative Multimodality. KI-Künstliche Intelligenz 26, 2 (2012), 161-168.

Google Scholar

[9]

Ali Borji, Andreas Lennartz, and Marc Pomplun. What do eyes reveal about the mind? Algorithmic inference of search targets from fixations. Neurocomputing 149, PB (2015), 788-799.

Google Scholar

[10]

Sven Stauden, Michael Barz, and Daniel Sonntag. Visual Search Target Inference Using Bag of Deep Visual Words. In KI 2018: Advances in Artificial Intelligence, Frank Trollmann and Anni-Yasmin Turhan (Eds.). Springer International Publishing, Cham, 297-304.

Google Scholar

[11]

Hosnieh Sattar, Andreas Bulling, and Mario Fritz. Predicting the Category and Attributes of Visual Search Targets Using Deep Gaze Pooling. In 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). IEEE, 2740-2748.

Google Scholar

[12]

D. W. Hansen and Q. Ji. In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3):478-500, 2010. 1

Digital Library

Google Scholar

[13]

Kellnhofer, P., Recasens, A., Stent, S., Matusik, W., Torralba, A.: Gaze360: physically unconstrained gaze estimation in the wild. In 2019 Proceedings of the IEEE International Conference on Computer Vision. pp. 6912-6921.

Google Scholar

[14]

Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. It's written all over your face: Full-face appearance-based gaze estimation. In CVPR - Workshops, 2017.

Crossref

Google Scholar

[15]

Xucong Zhang, Seonwook Park, Thabo Beeler, Derek Bradley, Siyu Tang and Otmar Hilliges. ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation. In ECCV, 2020.

Digital Library

Google Scholar

[16]

Kar-Han Tan, David J. Kriegman, and Narendra Ahuja. Appearance-based eye gaze estimation. In WACV, 2002.

Google Scholar

[17]

W. Sewell and O. Komogortsev. Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network. In SIGCHI, 2010.

Digital Library

Google Scholar

[18]

F. Lu, Y. Sugano, T. Okabe, and Y. Sato. Adaptive linear regression for appearance-based gaze estimation. PAMI, 2014.

Crossref

Google Scholar

[19]

F. Lu, T. Okabe, Y. Sugano, and Y. Sato. Learning gaze biases with head motion for head pose-free gaze estimation. Image and Vision Computing, 2014.

Digital Library

Google Scholar

[20]

D. Torricelli, S. Conforto, M. Schmid, and T. D'Alessio. A neuralbased remote eye gaze tracker under natural head motion. Computer methods and programs in biomedicine, 2008.

Google Scholar

[21]

S. Baluja and D. Pomerleau. Non-intrusive gaze tracking using artificial neural networks. Technical report, 1994.

Digital Library

Google Scholar

[22]

L.-Q. Xu, D. Machin, and P. Sheppard. A novel approach to real-time non-intrusive gaze finding. In Proc. BMVC, 1998.

Crossref

Google Scholar

[23]

S. Baluja and D. Pomerleau. Non-intrusive gaze tracking using artificial neural networks. Technical Report CMU-CS-94-102, School of Computer Science, Carnegie Mellon University, 1994.

Google Scholar

[24]

F. Lu, Y. Sugano, T. Okabe, and Y. Sato. Inferring human gaze from appearance via adaptive linear regression. In Proc. ICCV, pages 153-160, 2011

Digital Library

Google Scholar

[25]

O. Williams, A. Blake, and R. Cipolla. Sparse and semi-supervised visual mapping with the S3GP. In Proc. CVPR, volume 1, pages230-237, 2006.

Google Scholar

[26]

Y. Sugano, Y. Matsushita, and Y. Sato. Appearance-based gaze estimation using visual saliency. IEEE Trans. Pattern Anal. Mach. Intell., 35(2):329-341, 2013.

Digital Library

Google Scholar

[27]

Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part I. LNCS, vol. 8689, pp. 818-833. Springer, Cham (2014).

Google Scholar

[28]

D. E. King, Dlib-ml: A machine learning toolkit. J. Mach. Learning Research, vol. 10, pp. 1755-1758, 2009.

Digital Library

Google Scholar

[29]

Xucong Zhang, Yusuke Sugano, and Andreas Bulling. Revisiting data normalization for appearance-based gaze estimation. In ETRA, 2018.

Digital Library

Google Scholar

[30]

Q. Yu, F. Liu, Y.-Z. Song, T. Xiang, T. M. Hospedales, and C. C. Loy. Sketch me that shoe. In CVPR, 2016.

Crossref

Google Scholar

[31]

K. G. A. Kovashka, D. Parikh. Whittlesearch: Interactive image search with relative attribute feedback. International Journal of Computer Vision, 115(2), 2012.

Digital Library

Google Scholar

[32]

G. J. Zelinsky, Y. Peng, and D. Samaras. Eye can read your mind: Decoding gaze fixations to reveal categorical search targets. Journal of Vision, 13(14), 2013.

Crossref

Google Scholar

[33]

M. B. Neider and G. J. Zelinsky. Searching for camouflaged targets: Effects of target-background similarity on visual search. Vision Research, 46(14):2217-2235, 2006.

Crossref

Google Scholar

[34]

D. T. Levin, Y. Takarae, A. G. Miner, and F. Keil. Efficient visual search by category: Specifying the features that mark the difference between artifacts and animals in preattentive vision. Perception & Psychophysics, 63(4):676-697, 2001.

Crossref

Google Scholar

[35]

M. P. Eckstein. Visual search: A retrospective. Journal of Vision, 11(5):14, 2011.

Crossref

Google Scholar

[36]

X. Chen and G. J. Zelinsky. Real-world visual search is dominated by top-down guidance. Vision Research, 46(24):4118-4133, 2006.

Crossref

Google Scholar

[37]

N. Karessli, Z. Akata, B. Schiele, and A. Bulling. Gaze embeddings for zero-shot image classification. In Proc. Of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

Crossref

Google Scholar

[38]

S. Mathe and C. Sminchisescu. Multiple instance reinforcement learning for efficient weakly-supervised detection in images. arXiv:1412.0100, 2014.

Google Scholar

[39]

Y. Sugano and A. Bulling. Seeing with humans: Gaze-assisted neural image captioning. arxiv:1608.05203, 2016.

Google Scholar

Cited By

View all

Salim DSusanto JManalu SShiddiqi H(2024)Through the Lens: Unveiling the Power and Promise of Google's Visual Search Technology2024 International Conference on Information Management and Technology (ICIMTech)10.1109/ICIMTech63123.2024.10780912(207-211)Online publication date: 28-Aug-2024
https://doi.org/10.1109/ICIMTech63123.2024.10780912

Recommendations

Automatic face analysis system based on face recognition and facial physiognomy
ICHIT'06: Proceedings of the 1st international conference on Advances in hybrid information technology

An automatic face analysis system is proposed which uses face recognition and facial physiognomy. It first detects human's face, extracts its features, and classifies the shape of facial features. It will analyze the person's facial physiognomy and then ...
Lighting-aware face frontalization for unconstrained face recognition

Provide both lighting-recovered and lighting-normalized frontalized images.Basic frontalization with a generic 3D face model by the alignment of only five landmarks.Lighting recovered and normalized image filling by the symmetry of quotient image.LRFF ...
Age-Invariant Face Recognition

One of the challenges in automatic face recognition is to achieve temporal invariance. In other words, the goal is to come up with a representation and matching scheme that is robust to changes due to facial aging. Facial aging is a complex process that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICIAI '21: Proceedings of the 2021 5th International Conference on Innovation in Artificial Intelligence

March 2021

246 pages

ISBN:9781450388634

DOI:10.1145/3461353

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ICIAI 2021

ICIAI 2021: 2021 the 5th International Conference on Innovation in Artificial Intelligence

March 5 - 8, 2021

Xia men, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
252
Total Downloads

Downloads (Last 12 months)120
Downloads (Last 6 weeks)15

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Salim DSusanto JManalu SShiddiqi H(2024)Through the Lens: Unveiling the Power and Promise of Google's Visual Search Technology2024 International Conference on Information Management and Technology (ICIMTech)10.1109/ICIMTech63123.2024.10780912(207-211)Online publication date: 28-Aug-2024
https://doi.org/10.1109/ICIMTech63123.2024.10780912

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

References

Cited By

Recommendations

Automatic face analysis system based on face recognition and facial physiognomy

Lighting-aware face frontalization for unconstrained face recognition

Age-Invariant Face Recognition

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

HTML Format

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations