More Web Proxy on the site http://driver.im/

Article

Integrating backchannel prediction models into embodied conversational agents

Authors:

Dirk HeylenAuthors Info & Claims

IVA'12: Proceedings of the 12th international conference on Intelligent Virtual Agents

Pages 268 - 274

https://doi.org/10.1007/978-3-642-33197-8_28

Published: 12 September 2012 Publication History

Abstract

In this paper we will present our design for generating listening behavior for embodied conversational agents. It uses a corpus based prediction model to predict the timing of backchannels. The design of the system iterates on a previous design (Huang et al.[5]) on which we propose improvements in terms of robustness and personalization. For robustness we propose a variable threshold determined at run-time to regulate the amount of backchannels being produced by the system. For personalization we propose a character specification interface where the typical type of head nods to be displayed by the agent can be specified and ways to generate slight variations during runtime.

References

[1]

Bavelas, J.B., Coates, L., Johnson, T.: Listeners as co-narrators. Journal of Personality and Social Psychology 79(6), 941-952 (2000).

[2]

Bevacqua, E., McRorie, M., Pammi, S., Pelachaud, C., Schröder, M., Sneddon, I., de Sevin, E.: SAL multimodal generation component with customised SAL characters and visual mimicking behaviour. Tech. rep., SEMAINE Project (2009).

[3]

Fujie, S., Fukushima, K., Kobayashi, T.: A conversation robot with back-channel feedback function based on linguistic and nonlinguistic information. In: Proc. Int. Conference on Autonomous Robots and Agents, pp. 379-384 (2004).

[4]

Gratch, J., Wang, N., Gerten, J., Fast, E., Duffy, R.: Creating Rapport with Virtual Agents. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 125-138. Springer, Heidelberg (2007).

Digital Library

[5]

Huang, L., Morency, L.-P., Gratch, J.: Virtual Rapport 2.0. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 68-79. Springer, Heidelberg (2011).

Digital Library

[6]

de Kok, I., Heylen, D.: The MultiLis Corpus - Dealing with Individual Differences in Nonverbal Listening Behavior. In: Esposito, A., Esposito, A.M., Martone, R., Müller, V.C., Scarpetta, G. (eds.) COST 2010. LNCS, vol. 6456, pp. 362-375. Springer, Heidelberg (2011).

Digital Library

[7]

de Kok, I., Ozkan, D., Heylen, D., Morency, L.-P.: Learning and Evaluating Response Prediction Models using Parallel Listener Consensus. In: Proceeding of International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (2010).

Digital Library

[8]

de Kok, I., Poppe, R., Heylen, D.: Iterative Perceptual Learning for Social Behavior Synthesis. Tech. rep., Centre for Telematics and Information Technology University of Twente (2012).

[9]

Kopp, S., Allwood, J., Grammer, K., Ahlsen, E., Stocksmeier, T.: Modeling Embodied Feedback with Virtual Humans. In: Wachsmuth, I., Knoblich, G. (eds.) ZiF Research Group International Workshop. LNCS (LNAI), vol. 4930, pp. 18-37. Springer, Heidelberg (2008).

Digital Library

[10]

Kraut, R.E., Lewis, S.H., Swezey, L.W.: Listener responsiveness and the coordination of conversation. Journal of Personality and Social Psychology 43(4), 718-731 (1982).

[11]

Maatman, R.M., Gratch, J., Marsella, S.: Natural Behavior of a Listening Agent. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 25-36. Springer, Heidelberg (2005).

Digital Library

[12]

Morency, L.P., de Kok, I., Gratch, J.: A probabilistic multimodal approach for predicting listener backchannels. Autonomous Agents and Multi-Agent Systems 20(1), 70-84 (2011).

Digital Library

[13]

Ozkan, D., Sagae, K., Morency, L.: Latent Mixture of Discriminative Experts for Multimodal Prediction Modeling. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 860-868. Association for Computational Linguistics (2010).

Digital Library

[14]

Poppe, R., Truong, K.P., Heylen, D.: Backchannels: Quantity, Type and Timing Matters. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 228-239. Springer, Heidelberg (2011).

Digital Library

[15]

Vilhjálmsson, H., Cantelmo, N., Cassell, J., Chafai, N.E., Kipp, M., Kopp, S., Mancini, M., Marsella, S., Marshall, A.N., Pelachaud, C., Ruttkay, Z., Thórisson, K.R., van Welbergen, H., van der Werf, R.J.: The Behavior Markup Language: Recent Developments and Challenges. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 99-111. Springer, Heidelberg (2007).

Digital Library

[16]

Ward, N., Tsukahara, W.: Prosodic features which cue back-channel responses in English and Japanese. Journal of Pragmatics 32(8), 1177-1207 (2000).

Cited By

Lala DInoue KKawahara TSawada K(2022)Backchannel Generation Model for a Third Party Listener AgentProceedings of the 10th International Conference on Human-Agent Interaction10.1145/3527188.3561926(114-122)Online publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1145/3527188.3561926

Recommendations

Embodied conversational agents: computing and rendering realistic gaze patterns
PCM'06: Proceedings of the 7th Pacific Rim conference on Advances in Multimedia Information Processing

We describe here our efforts for modeling multimodal signals exchanged by interlocutors when interacting face-to-face. This data is then used to control embodied conversational agents able to engage into a realistic face-to-face interaction with human ...
Multimodal backchannels for embodied conversational agents
IVA'10: Proceedings of the 10th international conference on Intelligent virtual agents

One of the most desirable characteristics of an Embodied Conversational Agent (ECA) is the capability of interacting with users in a human-like manner. While listening to a user, an ECA should be able to provide backchannel signals through visual and ...
Personality Analysis of Embodied Conversational Agents
IVA '18: Proceedings of the 18th International Conference on Intelligent Virtual Agents

People tend to personify machines. Giving machines the ability to actually produce social information can help improve human-machine interactions. Embodied Conversational Agents (ECAs) are virtual software agents that can process and produce speech, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

IVA'12: Proceedings of the 12th international conference on Intelligent Virtual Agents

September 2012

528 pages

ISBN:9783642331961

Editors:
Yukiko Nakano
Dept. of Computer and Information Science, Seikei University, Musashino-shi, Tokyo, Japan
,
Michael Neff
Department of Computer Science and Program for Technocultural Studies, University of California, 1 Shields Avenue, Davis, CA
,
Ana Paiva
Baskin School of Engineering, University of California Santa Cruz, 1156 N. High SOE-3, Santa Cruz, CA
,
Marilyn Walker
Baskin School of Engineering, University of California Santa Cruz, 1156 N. High SOE-3, Santa Cruz, CA

Sponsors

Univ. of California,: University of California at Santa Cruz

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 12 September 2012

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lala DInoue KKawahara TSawada K(2022)Backchannel Generation Model for a Third Party Listener AgentProceedings of the 10th International Conference on Human-Agent Interaction10.1145/3527188.3561926(114-122)Online publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1145/3527188.3561926

View Options

View options

Media

Figures

Other

Tables

View Table of Contents