More Web Proxy on the site http://driver.im/

research-article

A User Simulator Architecture for Socially-Aware Conversational Agents

Authors:

Florian Pecune,

Yoichi Matsuyama,

Justine CassellAuthors Info & Claims

IVA '18: Proceedings of the 18th International Conference on Intelligent Virtual Agents

Pages 133 - 140

https://doi.org/10.1145/3267851.3267916

Published: 05 November 2018 Publication History

Abstract

Over the last two decades, Reinforcement Learning (RL) has emerged the method of choice for data-driven dialog management. However, one of the limitations of RL methods for the optimization of dialog managers in the context of virtual conversational agents, is that they require a large amount of data, which is often unavailable, particularly when the dialog deals with complex discourse phenomena. User simulators help address this problem by generating synthetic data to train RL agents in an online fashion. In this work, we extend user simulators to the case of socially-aware conversational agents, that combine task and social functions. We propose a novel architecture that takes into consideration the user's conversational goals and generates both task and social behaviour. Our proposed architecture is general enough to be useful for training socially-aware conversational agents in any domain. As a proof of concept, we construct a user simulator for training a conversational recommendation agent and provide evidence towards the effectiveness of the approach.

References

[1]

Nalini Ambady and Robert Rosenthal. 1992. Thin slices of expressive behaviour as predictors of interpersonal consequences: a meta-analysis. Psychological Bulletin 111, 2 (1992), 256--274.

[2]

Timothy W Bickmore, Laura M Pfeifer Vardoulakis, and Daniel Schulman. 2013. Tinker: a relational agent museum guide. Autonomous agents and multi-agent systems 27, 2 (2013), 254--276.

[3]

James H Bray, Scott E Maxwell, and Scott E Maxwell. 1985. Multivariate analysis of variance. Number 54. Sage.

[4]

Justine Cassell and Timothy Bickmore. 2003. Negotiated collusion: Modeling social language and its relationship effects in intelligent agents. User modeling and user-adapted interaction 13, 1-2 (2003), 89--132.

Digital Library

[5]

Senthilkumar Chandramohan, Matthieu Geist, Fabrice Lefevre, and Olivier Pietquin. 2011. User simulation in dialogue systems using inverse reinforcement learning. In Conference of the International Speech Communication Association.

[6]

Nancy L Collins and Lynn Carol Miller. 1994. Self-disclosure and liking: a meta-analytic review. Psychological bulletin 116, 3 (1994), 457.

[7]

David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, et al. 2014. SimSensei Kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 international conference on Autonomous agents and multiagent systems. International Foundation for Autonomous Agents and Multiagent Systems, 1061--1068.

Digital Library

[8]

Aimee L Drolet and Michael W Morris. 2000. Rapport in conflict resolution: Accounting for how face-to-face contact fosters mutual cooperation in mixed-motive conflicts. Journal of Experimental Social Psychology 36, 1 (2000), 26--50.

[9]

Emmanuel Ferreira and Fabrice Lefèvre. 2015. Reinforcement-learning based dialogue system for human--robot interactions with socially-inspired rewards. Computer Speech and Language 34, 1 (2015), 256--274.

Digital Library

[10]

Norman M Fraser and G Nigel Gilbert. 1991. Simulating speech systems. Computer Speech and Language 5, 1 (1991), 81--99.

[11]

Kallirroi Georgila, Maria K Wolters, and Johanna D Moore. 2010. Learning dialogue strategies from older and younger simulated users. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Association for Computational Linguistics, 103--106.

Digital Library

[12]

Goren Gordon, Samuel Spaulding, Jacqueline Kory Westlund, Jin Joo Lee, Luke Plummer, Marayna Martinez, Madhurima Das, and Cynthia Breazeal. 2016. Affective personalization of a social robot tutor for children's second language skills. In Proceedings of the Conference on Artificial Intelligence (AAAI-16). 3951--3957.

Digital Library

[13]

Matt Stuttle Steve Young Jost Schatzmann, Karl Weilhammer. 2006. A Survey of Statistical User Simulation Techniques for Reinforcement-Learning of Dialogue Management Strategies. The Knowledge Engineering Review 1, 24 (2006).

Digital Library

[14]

Simon Keizer, Stéphane Rossignol, Senthilkumar Chandramohan, and Olivier Pietquin. 2012. User simulation in the development of statistical spoken dialogue systems. In Data-Driven Methods for Adaptive Spoken Dialogue Systems. 39--73.

[15]

Iolanda Leite, André Pereira, Ginevra Castellano, Samuel Mascarenhas, Carlos Martinho, and Ana Paiva. 2011. Modelling Empathy in Social Robotic Companions. In International Conference on User Modeling, Adaptation, and Personalization. 135--147.

Digital Library

[16]

Oliver Lemon and Olivier Pietquin. 2012. Data-Driven Methods for Adaptive Spoken Dialogue Systems: Computational Learning for Conversational Interfaces. Springer-Verlag New York.

Digital Library

[17]

Xiujun Li, Zachary C Lipton, Bhuwan Dhingra, Lihong Li, Jianfeng Gao, and Yun-Nung Chen. 2016. A user simulator for task-completion dialogues. arXiv preprint arXiv:1612.05688 (2016).

[18]

Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE transactions on information theory 28, 2 (1982), 129--137.

Digital Library

[19]

Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. 2013. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, Vol. 30. 3.

[20]

Yoichi Matsuyama, Arjun Bhardwaj, Ran Zhao, Oscar Romeo, Sushma Akoju, and Justine Cassell. 2016. Socially-aware animated intelligent personal assistant agent. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 224--227.

[21]

Florian Pecune, Jingya Chen, Yoichi Matsuyama, and Justine Cassell. 2018. Field Trial Analysis of Socially Aware Robot Assistant. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 1241--1249.

Digital Library

[22]

Olivier Pietquin and Helen Hastie. 2013. A survey on metrics for the evaluation of user simulations. The Knowledge Engineering Review 28, 1 (2013), 59--73.

[23]

David V Pynadath and Stacy C Marsella. 2005. PsychSim: modeling theory of mind with decision-theoretic agents. In Proceedings of the 19th international joint conference on Artificial intelligence. Morgan Kaufmann Publishers Inc., 1181--1186.

Digital Library

[24]

Charles Rich and Candace L Sidner. 2012. Using collaborative discourse theory to partially automate dialogue tree authoring. In International Conference on Intelligent Virtual Agents. Springer, 327--340.

Digital Library

[25]

Verena Rieser and Oliver Lemon. 2011. Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation. Springer-Verlag Berlin Heidelberg.

Digital Library

[26]

Hannes Ritschel, Tobias Baur, and Elisabeth Andre. 2017. Adapting a Robot's Linguistic Style Based on Socially-Aware Reinforcement Learning.

[27]

Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20 (1987), 53--65.

Digital Library

[28]

Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young. 2007. Agenda-based user simulation for bootstrapping a POMDP dialogue system. In NAACL-Short '07 Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers. Association for Computational Linguistics, Stroudsburg, PA, USA, 149--152.

Digital Library

[29]

Jost Schatzmann, Blaise Thomson, and Steve Young. 2007. Statistical user simulation with a hidden agenda. Proc SIGDial, Antwerp 273282, 9 (2007).

[30]

Satinder P Singh, Michael J Kearns, Diane J Litman, and Marilyn A Walker. 2000. Reinforcement learning for spoken dialogue systems. In Advances in Neural Information Processing Systems. 956--962.

Digital Library

[31]

Tanmay Sinha and Justine Cassell. 2015. We click, we align, we learn: Impact of influence and convergence processes on student learning and rapport building. In Proceedings of the 1st Workshop on Modeling INTERPERsonal SynchrONy And infLuence. ACM, 13--20.

Digital Library

[32]

Helen Spencer-Oatey. 2005. (Im) Politeness, face and perceptions of rapport: unpackaging their bases and interrelationships.

[33]

Karen Tracy and Nikolas Coupland. 1990. Multiple goals in discourse: An overview of issues. Journal of Language and Social Psychology 9, 1-2 (1990), 1--13.

[34]

Bernard L Welch. 1947. The generalization of student's' problem when several different population variances are involved. Biometrika 34, 1/2 (1947), 28--35.

[35]

Yorick Wilks. 2010. Close engagements with artificial companions: key social, psychological, ethical and design issues. Vol. 8. John Benjamins Publishing.

[36]

Jason D Williams. 2008. Evaluating user simulations with the Cramér--von Mises divergence. Speech communication 50, 10 (2008), 829--846.

Digital Library

[37]

Steve Young. 2010. Still talking to machines (cognitively speaking). In Eleventh Annual Conference of the International Speech Communication Association.

[38]

Zhou Yu, Alan W Black, and Alexander I. Rudnicky. 2017. Learning Conversational Systems that Interleave Task and Non-Task Content. In International Joint Conference on Artificial Intelligence. arXiv:1703.00099.

Digital Library

[39]

Ran Zhao, Alexandros Papangelis, and Justine Cassell. 2014. Towards a dyadic computational model of rapport management for human-virtual agent interaction. In International Conference on Intelligent Virtual Agents. Springer, 514--527.

[40]

Ran Zhao, Tanmay Sinha, Alan W Black, and Justine Cassell. 2016. Socially-Aware Virtual Agents: Automatically Assessing Dyadic Rapport from Temporal Patterns of Behavior. In International Conference on Intelligent Virtual Agents. 218--233.

Cited By

Kraus MWagner NRiekenbrauck RMinker W(2023)Improving Proactive Dialog Agents Using Socially-Aware Reinforcement LearningProceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3565472.3595611(146-155)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3565472.3595611
Kraus MRiekenbrauck RMinker W(2023)Development of a Trust-Aware User Simulator for Statistical Proactive Dialog Modeling in Human-AI TeamsAdjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3563359.3597403(38-43)Online publication date: 26-Jun-2023
https://dl.acm.org/doi/10.1145/3563359.3597403
Ashktorab ZHoover BAgarwal MDugan CGeyer WYang HYurochkin M(2023)Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group FairnessProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581227(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581227
Show More Cited By

Index Terms

A User Simulator Architecture for Socially-Aware Conversational Agents
1. Computing methodologies

Recommendations

A framework to co-optimize task and social dialogue policies using Reinforcement Learning
IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

One of the main challenges for conversational agents is to select the optimal dialogue policy based on the state of the interaction. This challenge becomes even harder when the conversational agent not only has to achieve a specific task, but also aims ...
Exploring Effects of Conversational Fillers on User Perception of Conversational Agents
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Through technological advancements in various areas of our lives, Conversational Agents progressed in their human-likeness. In the field of HCI, however, the use of conversational fillers (e.g., "um," "uh," etc.) by Conversational Agents have not been ...
Enabling Embodied Conversational Agents to Respond to Nonverbal Behavior of the Communication Partner
Human-Computer Interaction. User Experience and Behavior
Abstract
Humans communicate on three levels: words, paralanguage, and nonverbal. While conversational agents focus mainly on the interpretation of words that are being spoken, recently the focus has also shifted to how we say those words with our tone, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IVA '18: Proceedings of the 18th International Conference on Intelligent Virtual Agents

November 2018

381 pages

ISBN:9781450360135

DOI:10.1145/3267851

Conference Chairs:
Anton Bogdanovych
Western Sydney University
,
Deborah Richards
Macquarie University
,
Simeon Simoff
Western Sydney University
,
Program Chairs:
Catherine Pelachaud
CNRS - ISIR, Université Pierre et Marie Curie
,
Dirk Heylen
University of Twente
,
Tomas Trescak
Western Sydney University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IVA '18

Sponsor:

SIGAI

IVA '18: International Conference on Intelligent Virtual Agents

November 5 - 8, 2018

NSW, Sydney, Australia

Acceptance Rates

IVA '18 Paper Acceptance Rate 17 of 82 submissions, 21%;

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
414
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kraus MWagner NRiekenbrauck RMinker W(2023)Improving Proactive Dialog Agents Using Socially-Aware Reinforcement LearningProceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3565472.3595611(146-155)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3565472.3595611
Kraus MRiekenbrauck RMinker W(2023)Development of a Trust-Aware User Simulator for Statistical Proactive Dialog Modeling in Human-AI TeamsAdjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3563359.3597403(38-43)Online publication date: 26-Jun-2023
https://dl.acm.org/doi/10.1145/3563359.3597403
Ashktorab ZHoover BAgarwal MDugan CGeyer WYang HYurochkin M(2023)Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group FairnessProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581227(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581227
Galland LPelachaud CPecune F(2022)Adapting conversational strategies in information-giving human-agent interactionFrontiers in Artificial Intelligence10.3389/frai.2022.10293405Online publication date: 25-Oct-2022
https://doi.org/10.3389/frai.2022.1029340
Janowski KRitschel HAndré E(2022)Adaptive Artificial PersonalitiesThe Handbook on Socially Interactive Agents10.1145/3563659.3563666(155-194)Online publication date: 27-Oct-2022
https://dl.acm.org/doi/10.1145/3563659.3563666
Lee JLee DLee J(2022)Influence of Rapport and Social Presence with an AI Psychotherapy Chatbot on Users’ Self-DisclosureInternational Journal of Human–Computer Interaction10.1080/10447318.2022.214622740:7(1620-1631)Online publication date: 20-Nov-2022
https://doi.org/10.1080/10447318.2022.2146227
Soni MCowan BWade V(2022)Enhancing Self-disclosure In Open-Domain Dialogue By Candidate Re-rankingConversational AI for Natural Human-Centric Interaction10.1007/978-981-19-5538-9_17(243-252)Online publication date: 1-Nov-2022
https://doi.org/10.1007/978-981-19-5538-9_17
Callejas ZLugrin BMartin JMcTear MMiehle J(2021)Adaptive Systems for Multicultural and Ageing SocietiesMultimodal Agents for Ageing and Multicultural Societies10.1007/978-981-16-3476-5_1(1-20)Online publication date: 10-Oct-2021
https://doi.org/10.1007/978-981-16-3476-5_1
Pecune FCallebert LMarsella SObaid MMubin ONagai YOsawa HAbdelrahman YFjeld M(2020)A Socially-Aware Conversational Recommender System for Personalized Recipe RecommendationsProceedings of the 8th International Conference on Human-Agent Interaction10.1145/3406499.3415079(78-86)Online publication date: 10-Nov-2020
https://dl.acm.org/doi/10.1145/3406499.3415079
Gunson NSieińska WWalsh CDondrup CLemon O(2020)It's Good to Chat?Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents10.1145/3383652.3423889(1-8)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3383652.3423889
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents