Abstract
We present a scenario whereby an agent senses, interprets and copies a range of facial and gesture expression from a person in the real-world. Input is obtained via a video camera and processed initially using computer vision techniques. It is then processed further in a framework for agent perception, planning and behaviour generation in order to perceive, interpret and copy a number of gestures and facial expressions corresponding to those made by the human. By perceive, we mean that the copied behaviour may not be an exact duplicate of the behaviour made by the human and sensed by the agent, but may rather be based on some level of interpretation of the behaviour. Thus, the copied behaviour may be altered and need not share all of the characteristics of the original made by the human.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Martin, J.C., Abrilian, S., Devillers, L., Lamolle, M., Mancini, M., Pelachaud, C.: Levels of representation in the annotation of emotion for the specification of expressivity in ecas. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS, vol. 3661, pp. 405–417. Springer, Heidelberg (2005)
Peters, C.: Direction of attention perception for conversation initiation in virtual environments. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS, vol. 3661, pp. 215–228. Springer, Heidelberg (2005)
Scherer, K., Ekman, P.: Approaches to Emotion. Lawrence Erlbaum Associates, Mahwah (1984)
Ekman, P., Friesen, W.: The Facial Action Coding System. Consulting Psychologists Press, San Francisco (1978)
Tian, Y., Kanade, T., Cohn, J.: Facial expression analysis. In: Li, S.Z., Jain, A.K. (eds.) Handbook of face recognition (2003)
Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573 (2005)
Morency, L.P., Sidner, C., Lee, C., Darrell, T.: Contextual recognition of head gestures. In: Proceedings of ICMI 2005, Trento, Italy (2005)
Gratch, J., Marsella, S., Maatman, M., Gratch, J., Marsella, S.: Natural behavior of a listening agent. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661. Springer, Heidelberg (2005)
Donato, G., Bartlett, M., Hager, J., Ekman, P., Sejnowski, T.: Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999)
Wu, Y., Huang, T.: Hand modeling, analysis, and recognition for vision-based human computer interaction. IEEE Signal Processing Magazine 18, 51–60 (2001)
Wexelblat, A.: An approach to natural gesture in virtual environments. ACM Transactions on Computer-Human Interaction 2, 179–200 (1995)
Chi, D., Costa, M., Zhao, L., Badler, N.: The EMOTE model for effort and shape. In: ACM SIGGRAPH 2000, New Orleans, LA, pp. 173–182 (2000)
Byun, M., Badler, N.: FacEMOTE: Qualitative parametric modifiers for facial animations. In: Symposium on Computer Animation, San Antonio, TX (2002)
Kopp, S., Sowa, T., Wachsmuth, I.: Imitation games with an artificial agent: From mimicking to understanding shape-related iconic gestures. In: Gesture Workshop, pp. 436–447 (2003)
Rapantzikos, K., Avrithis, Y.: An enhanced spatiotemporal visual attention model for sports video analysis. In: International Workshop on content-based Multimedia indexing (CBMI), Riga, Latvia (2005)
Pelachaud, C., Bilvi, M.: Computational model of believable conversational agents. In: Huget, M.-P. (ed.) Communication in Multiagent Systems. LNCS (LNAI), vol. 2650, pp. 300–317. Springer, Heidelberg (2003)
Hartmann, B., Mancini, M., Pelachaud, C.: Implementing expressive gesture synthesis for embodied conversational agents. In: Gesture Workshop, Vannes (2005)
Johansson, G.: Visual perception of biological motion adn a model for its analysis. Perception and Psychophysics 14, 201–211 (1973)
Wallbott, H.G., Scherer, K.R.: Cues and channels in emotion recognition. Journal of Personality and Social Psychology 51, 690–699 (1986)
Gallaher, P.E.: Individual differences in nonverbal behavior: Dimensions of style. Journal of Personality and Social Psychology 63, 133–145 (1992)
Ball, G., Breese, J.: Emotion and personality in a conversational agent. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Characters. MITpress, Cambridge (2000)
Pollick, F.E.: The features people use to recognize human movement style. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 10–19. Springer, Heidelberg (2004)
Hartmann, B., Mancini, M., Buisine, S., Pelachaud, C.: Design and evaluation of expressive gesture synthesis for embodied conversational agents. In: Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), Utretch (2005)
Tekalp, A., Ostermann, J.: Face and 2-d mesh animation in MPEG-4. Signal Processing: Image Communication 15, 387–421 (2000)
Raouzaiou, A., Tsapatsoulis, N., Karpouzis, K., Kollias, S.: Parameterized facial expression synthesis based on mpeg-4. EURASIP Journal on Applied Signal Processing 2002, 1021–1038 (2002)
Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)
Harrigan, J.A.: Listener’s body movements and speaking turns. Communication Research 12, 233–250 (1985)
Hartmann, B., Mancini, M., Pelachaud, C.: Formational parameters and adaptive prototype instantiation for MPEG-4 compliant gesture synthesis. In: Computer Animation 2002, Geneva, Switzerland. IEEE Computer Society Press, Los Alamitos (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bevacqua, E. et al. (2006). Multimodal Sensing, Interpretation and Copying of Movements by a Virtual Agent. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Weber, M. (eds) Perception and Interactive Technologies. PIT 2006. Lecture Notes in Computer Science(), vol 4021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11768029_16
Download citation
DOI: https://doi.org/10.1007/11768029_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34743-9
Online ISBN: 978-3-540-34744-6
eBook Packages: Computer ScienceComputer Science (R0)