research-article

Dancing the night away: controlling a virtual karaoke dancer by multimodal expressive cues

Authors:

Matthias Rehm,

Thurid Vogt,

Michael Wissner,

Nikolaus BeeAuthors Info & Claims

AAMAS '08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3

Pages 1249 - 1252

Published: 12 May 2008 Publication History

Get Access

Abstract

In this article, we propose an approach of nonverbal interaction with virtual agents to control agents' behavioral expressivity by extracting and combining acoustic and gestural features. The goal for this approach is twofold, (i) expressing individual features like situated arousal and personal style and (ii) transmitting this information in an immersive 3D environment by suitable means.

References

[1]

E. Bevacqua, A. Raouzaiou, C. Peters, G. Caridakis, K. Karpouzis, C. Pelachaud, and M. Mancini. Multimodal sensing, interpretation and copying of movements by a virtual agent. In PIT, pages 164--174, 2006.

Digital Library

Google Scholar

[2]

F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss. A Database of German Emotional Speech. In Proceedings of Interspeech 2005, 2005.

Google Scholar

[3]

G. Castellano, R. Bresin, A. Camurri, and G. Volpe. User-Centered Control of Audio and Visual Expressive Feedback by Full-Body Movements. In Affective Computing and Intelligent Interaction, pages 501--510. Springer, Berlin, Heidelberg, 2007.

Digital Library

Google Scholar

[4]

E. Crane and M. Gross. Motion Capture and Emotion: Affect Detection in Whole Body Movement. In Affective Computing and Intelligent Interaction, pages 95--101. Springer, Berlin, Heidelberg, 2007.

Digital Library

Google Scholar

[5]

P. E. Gallaher. Individual Differences in Nonverbal Behavior: Dimensions of Style. Journal of Personality and Social Psychology, 63(1):133--145, 1992.

Crossref

Google Scholar

[6]

J. James, T. Ingalls, G. Qian, L. Olsen, D. Whiteley, S. Wong, and T. Rikakis. Movement-based interactive dance performance. In Proceedings of ACM Multimedia, pages 470--480, New York, NY, USA, 2006. ACM Press.

Digital Library

Google Scholar

[7]

C. Jones and J. Sutherland. Acoustic emotion recognition for affective computer gaming. In C. Peter and R. Beale, editors, Affect and emotion in human-computer interaction, Heidelberg, Berlin, 2007. Springer.

Digital Library

Google Scholar

[8]

P. Y. Oudeyer. The production and recognition of emotions in speech: features and algorithms. International Journal of Human-Computer Studies, 59(1--2):157--183, 2003.

Digital Library

Google Scholar

[9]

C. Pelachaud. Multimodal expressive embodied conversational agents. In Proceedings of ACM Multimedia, pages 683--689, 2005.

Digital Library

Google Scholar

[10]

R. Price, C. Douther, and M. A. Jack. An investigation of the effectiveness of choreography for the portrayal of mood in virtual environments. In Proceedings of AGENTS '00, pages 54--55, New York, NY, USA, 2000. ACM Press.

Digital Library

Google Scholar

[11]

M. Rehm, N. Bee, B. Endrass, M. Wissner, and E. André. Too close for comfort? Adapting to the user's cultural background. In Workshop on Human-Centered Multimedia, ACM Multimedia, 2007.

Digital Library

Google Scholar

[12]

D. Reidsma, A. Nijholt, R. Poppe, R. Rienks, and H. Hondorp. Virtual rap dancer: invitation to dance. In CHI '06: CHI '06 extended abstracts on Human factors in computing systems, pages 263--266, New York, NY, USA, 2006. ACM Press.

Digital Library

Google Scholar

[13]

D. Rubine. Specifying gestures by example. In SIGGRAPH '91: Proceedings of the 18th annual conference on Computer graphics and interactive techniques, pages 329--337, New York, NY, USA, 1991. ACM Press.

Digital Library

Google Scholar

[14]

T. Vogt and E. André. Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition. In IEEE International Conference on Multimedia & Expo (ICME), Amsterdam, The Netherlands, 2005.

Crossref

Google Scholar

Cited By

View all

Richards DRoxas JBilgin AHanna NWeiss GYolum PBordini RElkind E(2015)A Dancing Virtual Agent to Evoke Human EmotionsProceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems10.5555/2772879.2773393(1701-1702)Online publication date: 4-May-2015
https://dl.acm.org/doi/10.5555/2772879.2773393
Cavazza MPizzi DCharles FVogt TAndré ESierra CCastelfranchi CDecker KSichman J(2009)Emotional input for character-based interactive storytellingProceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 110.5555/1558013.1558056(313-320)Online publication date: 10-May-2009
https://dl.acm.org/doi/10.5555/1558013.1558056

Index Terms

Dancing the night away: controlling a virtual karaoke dancer by multimodal expressive cues
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools

Recommendations

Changes in verbal and nonverbal conversational behavior in long-term interaction
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interaction

We present an empirical investigation of conversational behavior in dyadic interaction spanning multiple conversations, in the context of a developing interpersonal relationship between a health counselor and her clients. Using a longitudinal video ...
Prediction of Various Backchannel Utterances Based on Multimodal Information
IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents

The listener's backchannels are an important part of dialogues. With appropriate backchannels, people are able to smoothly promote dialogues. Thus, backchannels are considered to be important in dialogues between not only humans but also humans and ...
Virtual rap dancer: invitation to dance
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

This paper presents a virtual rap dancer that is able to dance to the beat of music coming in from music recordings, beats obtained from music, voice or other input through a microphone, motion beats detected in the video stream of a human dancer, or ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

AAMAS '08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3

May 2008

503 pages

ISBN:9780981738123

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 12 May 2008

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AAMAS08

Sponsor:

ACM
AAAI

AAMAS08: 7th International Conference on Autonomous Agents and Multi Agent Systems

May 12 - 16, 2008

Estoril, Portugal

Acceptance Rates

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
156
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Richards DRoxas JBilgin AHanna NWeiss GYolum PBordini RElkind E(2015)A Dancing Virtual Agent to Evoke Human EmotionsProceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems10.5555/2772879.2773393(1701-1702)Online publication date: 4-May-2015
https://dl.acm.org/doi/10.5555/2772879.2773393
Cavazza MPizzi DCharles FVogt TAndré ESierra CCastelfranchi CDecker KSichman J(2009)Emotional input for character-based interactive storytellingProceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 110.5555/1558013.1558056(313-320)Online publication date: 10-May-2009
https://dl.acm.org/doi/10.5555/1558013.1558056

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Changes in verbal and nonverbal conversational behavior in long-term interaction

Prediction of Various Backchannel Utterances Based on Multimodal Information

Virtual rap dancer: invitation to dance