research-article

Automated Eye Motion Using Texture Synthesis

Authors:

Zhigang Deng,

J. P. Lewis,

Ulrich NeumannAuthors Info & Claims

IEEE Computer Graphics and Applications, Volume 25, Issue 2

Pages 24 - 30

https://doi.org/10.1109/MCG.2005.35

Published: 01 March 2005 Publication History

Publisher Site

Abstract

Modeling and animating human eyes requires special care, because, as the "windows to the soul", the eyes are particularly scrutinized by human observers.Our goal in this article is to simultaneously synthesize realistic eye gaze and blink motion, accounting for any possible correlations between the two. This problem of synthesizing signals that appear similar (but not identical) to a given sample is essentially the same problem as texture synthesis, but in a one-dimensional (vector) context. We demonstrate that texture synthesis methods can be applied to this animation problem, providing an effective method for capturing both perceptible movement and blink statistics, and any correlations between them. The resulting method is simple to implement yet produces life-like and lively eye motion for applications where automated movement (for example, for game characters) or voiceless eye motions (such as listening avatars) is a requirement.

References

[1]

S.P. Lee J.B. Badler and N. Badler, "Eyes Alive," ACM Trans. Graphics, ACM Press, vol. 21, no. 3, 2002, pp. 637-644.

Digital Library

Google Scholar

[2]

D.D. Salvucci and J.H. Goldberg, "Identifying Fixations and Saccades in Eye-Tracking Protocols," Proc. Symp. Eye Tracking Research & Applications (ETRA), ACM Press, Nov. 2000, pp. 71-78.

Digital Library

Google Scholar

[3]

L. Liang, et al., "Real-Time Texture Synthesis by Patch-Based Sampling," ACM Trans. Graphics, ACM Press, vol. 20, no. 3, 2001, pp. 127-150.

Digital Library

Google Scholar

[4]

J. Cassell, et al., "Animated Conversation: Ruled-Based Generation of Facial Expression Gesture and Spoken Intonation for Multiple Conversational Agents," Proc. Siggraph, ACM Press, 1994, pp. 413-420.

Digital Library

Google Scholar

[5]

J. Cassell H. Vilhjalmsson and T. Bickmore, "BEAT: The Behavior Expression Animation Toolkit," Proc. Siggraph, ACM Press, 2001, pp. 477-486.

Digital Library

Google Scholar

[6]

S. Chopra-Khullar and N. Badler, "Where to Look? Automating Visual Attending Behaviors of Virtual Human Characters," Proc. 3rd ACM Conf. Autonomous Agents, ACM Press, 1999, pp. 16-23.

Digital Library

Google Scholar

[7]

R. Vertegaal G.V. Derveer and H. Vons, "Effects of Gaze on Multiparty Mediated Communication," Proc. Graphics Interface, Morgan Kaufmann, 2000, pp. 95-102.

Google Scholar

[8]

R. Vertegaal, et al., "Eye Gaze Patterns in Conversations: There is More to Conversational Agents than Meets the Eyes," Proc. ACM CHI Conf. Human Factors in Computing Systems, ACM Press, 2001, pp. 301-308.

Digital Library

Google Scholar

[9]

S.P. Lee J.B. Badler and N. Badler, "Eyes Alive," ACM Trans. Graphics, vol. 21, no. 3, 2002, pp. 637-644.

Digital Library

Google Scholar

[10]

O. Arikan and D. Forsythe, "Interactive Motion Generation from Examples," ACM Trans. Graphics, ACM Press, vol. 21, no. 3, 2002, pp. 483-490.

Digital Library

Google Scholar

[11]

L. Kovar, et al., "Motion Graphs," ACM Trans. Graphics, ACM Press, vol. 21, no. 3, 2002, pp. 473-482.

Digital Library

Google Scholar

[12]

Y. Li T. Wang and H.-Y. Shum, "Motion Texture: A Two-Level Statistical Model for Character Motion Synthesis," ACM Trans. Graphics, ACM Press, vol. 21, no. 3, 2002, pp. 465-472.

Digital Library

Google Scholar

[13]

A. Efros and T.K. Leung, "Texture Synthesis by Non-Parametric Sampling," Proc. Int'l Conf. Computer Vision (ICCV), IEEE CS Press, 1999, pp. 1033-1038.

Digital Library

Google Scholar

[14]

L. Liang, et al., "Real-Time Texture Synthesis by Patch-Based Sampling," ACM Trans. Graphics, 2001, vol. 20, no. 3, pp. 127-150.

Digital Library

Google Scholar

Cited By

View all

Jin ADeng QDeng Z(2022)S2M-Net: Speech Driven Three-party Conversational Motion Synthesis NetworksProceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3561975.3562954(1-10)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3561975.3562954
Eom HHan DShin JNoh J(2019)Model Predictive Control with a Visuomotor System for Physics-based Character AnimationACM Transactions on Graphics10.1145/336090539:1(1-11)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360905
Klein AYumak ZBeij Avan der Stappen A(2019)Data-driven Gaze Animation using Recurrent Neural NetworksProceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3359566.3360054(1-11)Online publication date: 28-Oct-2019
https://dl.acm.org/doi/10.1145/3359566.3360054
Show More Cited By

Index Terms

Automated Eye Motion Using Texture Synthesis
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        Motion capture
  2. Computer graphics
2. Mathematics of computing
  1. Continuous mathematics
    1. Topology
      1. Geometric topology

Recommendations

Perceptual Comparison of Procedural and Data-Driven Eye Motion Jitter
SAP '19: ACM Symposium on Applied Perception 2019

Research has shown that keyframed eye motions are perceived as more realistic when some noise is added to eyeball motions and to pupil size changes. We investigate whether this noise, in contrast to being motion captured, can be synthesized with ...
Audio-based head motion synthesis for Avatar-based telepresence systems
ETP '04: Proceedings of the 2004 ACM SIGMM workshop on Effective telepresence

In this paper, a data-driven audio-based head motion synthesis technique is presented for avatar-based telepresence systems. First, head motion of a human subject speaking a custom corpus is captured, and the accompanying audio features are extracted. ...
Live Speech Driven Head-and-Eye Motion Generators

This paper describes a fully automated framework to generate realistic head motion, eye gaze, and eyelid motion simultaneously based on live (or recorded) speech input. Its central idea is to learn separate yet interrelated statistical models for each ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Computer Graphics and Applications

IEEE Computer Graphics and Applications Volume 25, Issue 2

March 2005

105 pages

ISSN:0272-1716

Issue’s Table of Contents

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 March 2005

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Jin ADeng QDeng Z(2022)S2M-Net: Speech Driven Three-party Conversational Motion Synthesis NetworksProceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3561975.3562954(1-10)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3561975.3562954
Eom HHan DShin JNoh J(2019)Model Predictive Control with a Visuomotor System for Physics-based Character AnimationACM Transactions on Graphics10.1145/336090539:1(1-11)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360905
Klein AYumak ZBeij Avan der Stappen A(2019)Data-driven Gaze Animation using Recurrent Neural NetworksProceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3359566.3360054(1-11)Online publication date: 28-Oct-2019
https://dl.acm.org/doi/10.1145/3359566.3360054
Jörg SDuchowski AKrejtz KNiedzielska A(2019)Perceptual Comparison of Procedural and Data-Driven Eye Motion JitterACM Symposium on Applied Perception 201910.1145/3343036.3343130(1-5)Online publication date: 19-Sep-2019
https://dl.acm.org/doi/10.1145/3343036.3343130
Jin ADeng QZhang YDeng Z(2019)A Deep Learning-Based Model for Head and Eye Motion Generation in Three-party ConversationsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/33402502:2(1-19)Online publication date: 26-Jul-2019
https://dl.acm.org/doi/10.1145/3340250
Huang DChandra EYang XZhou YMing HLin WDong MLi HHuang DZhao SSchuller BYao HTao JXu MXie LHuang QYang J(2018)Visual Speech Emotion Conversion using Deep Learning for 3D Talking HeadProceedings of the Joint Workshop of the 4th Workshop on Affective Social Multimedia Computing and first Multi-Modal Affective Computing of Large-Scale Multimedia Data10.1145/3267935.3267950(7-13)Online publication date: 19-Oct-2018
https://dl.acm.org/doi/10.1145/3267935.3267950
Jörg SDuchowski AKrejtz KNiedzielska A(2018)Perceptual Adjustment of Eyeball Rotation and Pupil Size Jitter for Virtual CharactersACM Transactions on Applied Perception10.1145/323830215:4(1-13)Online publication date: 18-Oct-2018
https://dl.acm.org/doi/10.1145/3238302
Xiong JPeng ZXu WLiang QWang LWang Q(2016)Helmet-mounted eye control system for pupil recognition and positionInternational Journal of Computer Applications in Technology10.1504/IJCAT.2016.07360553:1(91-103)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1504/IJCAT.2016.073605
Pejsa TRakita DMutlu BGleicher M(2016)Authoring directed gaze for full-body motion captureACM Transactions on Graphics10.1145/2980179.298244435:6(1-11)Online publication date: 5-Dec-2016
https://dl.acm.org/doi/10.1145/2980179.2982444
Schulz SLier FKipp AWachsmuth SYau WOmori TMetta GOsawa HZhao S(2016)HumotionProceedings of the Fourth International Conference on Human Agent Interaction10.1145/2974804.2974827(207-214)Online publication date: 4-Oct-2016
https://dl.acm.org/doi/10.1145/2974804.2974827
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

Perceptual Comparison of Procedural and Data-Driven Eye Motion Jitter

Audio-based head motion synthesis for Avatar-based telepresence systems

Live Speech Driven Head-and-Eye Motion Generators

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations