More Web Proxy on the site http://driver.im/

research-article

It's Good to Chat?: Evaluation and Design Guidelines for Combining Open-Domain Social Conversation with Task-Based Dialogue in Intelligent Buildings

Authors:

Weronika Sieińska,

Christopher Walsh,

Christian Dondrup,

Oliver LemonAuthors Info & Claims

IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

Article No.: 24, Pages 1 - 8

https://doi.org/10.1145/3383652.3423889

Published: 19 October 2020 Publication History

Abstract

We present and evaluate a deployed conversational AI system that acts as a host of a working public building on a university campus. The system combines open-domain social chat with task-based conversation regarding navigation in the building, live resource updates (e.g. available computers), and events in the building. We investigated the impact of open-domain social chat on task completion and user preferences by comparing the combined system with a task-only version. We find that there is no significant difference in task completion or several aspects of user preference between the two systems, but that users would be significantly happier to talk to the task-only system in the future. This suggests that the "walk-up" public setting and workplace nature of the environment creates a markedly different use case to the in-home, and more individual and private "companion/assistant" setting which is commonly assumed for systems like Alexa. We discuss the implications for the design of conversational systems in other public settings.

References

[1]

Bayan Abushawar and Eric Atwell. 2015. ALICE chatbot: Trials and outputs. Computación y Sistemas 19 (12 2015). https://doi.org/10.13053/cys-19-4-2326

[2]

Timothy Bickmore and Justine Cassell. 2001. Relational Agents: A Model and Implementation of Building User Trust. SIGCHI (01 2001). https://doi.org/10.1145/365024.365304

Digital Library

[3]

Timothy Bickmore, Laura Vardoulakis, and Daniel Schulman. 2013. Tinker: A relational agent museum guide. Autonomous Agents and Multi-Agent Systems 27 (12 2013). https://doi.org/10.1007/s10458-012-9216-7

[4]

Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. 2017. A Survey on Dialogue Systems: Recent Advances and New Frontiers. SIGKDD Explorations 19 (2017), 25--35.

Digital Library

[5]

Amanda Cercas Curry, Ioannis Papaioannou, Alessandro Suglia, Shubham Agarwal, Igor Shalyminov, Xinnuo Xu, Ondřej Dušek, Arash Eshghi, Ioannis Konstas, Verena Rieser, et al. 2018. Alana v2: Entertaining and informative open-domain social dialogue using ontologies and entity linking. Alexa Prize Proceedings (2018).

[6]

Mary Ellen Foster, Bart Craenen, Amol Deshmukh, Oliver Lemon, Emanuele Bastianelli, Christian Dondrup, Ioannis Papaioannou, Andrea Vanzo, Jean-Marc Odobez, Olivier Canévet, Yuanzhouhan Cao, Weipeng He, Angel Martínez-González, Petr Motlicek, Rémy Siegfried, Rachid Alami, Kathleen Belhassein, Guilhem Buisan, Aurélie Clodic, Amandine Mayima, Yoan Sallami, Guillaume Sarthou, Phani-Teja Singamaneni, Jules Waldhart, Alexandre Mazel, Maxime Caniot, Marketta Niemelä, Päivi Heikkilä, Hanna Lammi, and Antti Tammela. 2019. MuMMER: Socially Intelligent Human-Robot Interaction in Public Spaces.

[7]

A. L. Gorin, G. Riccardi, and J. H. Wright. 1997. How May I Help You? Speech Commun. 23, 1-2 (Oct. 1997), 113--127. https://doi.org/10.1016/S0167-6393(97)00040-X

Digital Library

[8]

Alankar Jain, Florian Pecune, Yoichi Matsuyama, and Justine Cassell. 2018. A User Simulator Architecture for Socially-Aware Conversational Agents. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (Sydney, NSW, Australia) (IVA '18). Association for Computing Machinery, New York, NY, USA, 133--140. https://doi.org/10.1145/3267851.3267916

Digital Library

[9]

Saba Khashe, Gale Lucas, Burcin Becerik-Gerber, and Jonathan Gratch. 2019. Establishing Social Dialog between Buildings and Their Users. International Journal of Human-Computer Interaction 35, 17 (2019), 1545--1556. https://doi.org/10.1080/10447318.2018.1555346

[10]

Chandra Khatri, Behnam Hedayatnia, Anu Venkatesh, Jeff Nunn, Yi Pan, Qing Liu, Han Song, Anna Gottardi, Sanjeev Kwatra, Sanju Pancholi, Ming Cheng, Qinglang Chen, Lauren Stubel, Karthik Gopalakrishnan, Kate Bland, Raefer Gabriel, Arindam Mandal, Dilek Hakkani-Tur, Gene Hwang, Nate Michel, Eric King, and Rohit Prasad. 2018. Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize. In Alexa Prize Proceedings. 1--28. arXiv:1812.10757 http://arxiv.org/abs/1812.10757

[11]

Oliver Lemon and Olivier Pietquin. 2007. Machine learning for spoken dialogue systems. 8th Annual Conference of the International Speech Communication Association, Interspeech 2007 3 (2007), 1761--1764.

[12]

Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of psychology 140 (1932).

[13]

Irene Lopatovska and Heyrling Oropeza. 2018. User interactions with "Alexa" in public academic space. Proceedings of the Association for Information Science and Technology 55, 1 (2018), 309--318. https://doi.org/10.1002/pra2.2018.14505501034 arXiv:https://asistdl.onlinelibrary.wiley.com/doi/pdf/10.1002/pra2.2018.14505501034

[14]

Ioannis Papaioannou, Amanda Cercas Curry, Jose Part, Igor Shalyminov, Xu Xinnuo, Yanchao Yu, Ondřej Dušek, Verena Rieser, and Oliver Lemon. 2017. An Ensemble Model with Ranking for Social Dialogue. NIPS 2017 Conversational AI Workshop; Conference date: 08-12-2017 Through 08-12-2017.

[15]

Ioannis Papaioannou, Amanda Cercas Curry, Jose L. Part, Igor Shalyminov, Xinnuo Xu, Yanchao Yu, Ondřej Dušek, Verena Rieser, et al. 2017. Alana: Social Dialogue using an Ensemble Model and a Ranker trained on User Feedback. Alexa Prize Proceedings (2017).

[16]

Ioannis Papaioannou, Christian Dondrup, Jekaterina Novikova, and Oliver Lemon. 2017. Hybrid chat and task dialogue for more engaging HRI using reinforcement learning. In 26th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2017, Lisbon, Portugal, August 28 - Sept. 1, 2017. IEEE, 593--598.

Digital Library

[17]

Florian Pecune, Jingya Chen, Yoichi Matsuyama, and Justine Cassell. 2018. Field Trial Analysis of Socially Aware Robot Assistant. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (Stockholm, Sweden) (AAMAS '18). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1241--1249.

Digital Library

[18]

Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '17 (2017), 2853--2859. https://doi.org/10.1145/3027063.3053246

Digital Library

[19]

Antoine Raux, Brian Langner, Dan Bohus, Alan W. Black, and Maxine Eskenazi. 2005. Let's Go Public! Taking a spoken dialog system to the real world. 9th European Conference on Speech Communication and Technology (2005), 885--888.

[20]

Emanuel A. Schegloff. 1968. Sequencing in Conversational Openings. American Anthropologist 70, 6 (1968), 1075--1095.

[21]

Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I. Hong. 2018. "Hey Alexa, What's Up?": Studies of In-Home Conversational Agent Usage. Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS '18 (2018), 857--868. https://doi.org/10.1145/3196709.3196772

Digital Library

[22]

Igor Shalyminov, Ondřej Dušek, and Oliver Lemon. 2018. Neural Response Ranking for Social Conversation: A Data-Efficient Approach. In Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI. Association for Computational Linguistics, Brussels, Belgium.

[23]

Weronika Sieińska, Nancie Gunson, Christopher Walsh, Christian Dondrup, and Oliver Lemon. 2020. Conversational Agents for Intelligent Buildings. Proceedings of the 2020 SIGDial Conference (2020).

[24]

Marilyn A Walker, Diane J Litman, Candace A Kamm, and Alicia Abella. 1997. PARADISE: A Framework for Evaluating Spoken Dialogue Agents. In Proceedings of the 8th Conference on European Chapter of the Association of Computational Linguistics, ACL '97. 271--280.

[25]

Steve Young, Milica Gašić, Blaise Thomson, and Jason Williams. 2012. POMDP-based Statistical Spoken Dialogue Systems: a Review. In Proceedings of the IEEE (proceedings of the ieee ed.). 1--20. https://www.microsoft.com/en-us/research/publication/pomdp-based-statistical-spoken-dialogue-systems-a-review/ DOI 10.1109/JPROC.2012.2225812.

[26]

Zhou Yu, Alan W. Black, and Alexander I. Rudnicky. 2017. Learning conversational systems that interleave task and non-task content. IJCAI International Joint Conference on Artificial Intelligence (2017), 4214--4220. https://doi.org/10.24963/ijcai.2017/589 arXiv:1703.00099

[27]

Zhou Yu, Dan Bohus, and Eric Horvitz. 2015. Incremental coordination: Attention-centric speech production in a physically situated conversational agent. SIGDIAL 2015 September (2015), 402--406.

Cited By

Gunson NGarcía DSieińska WDondrup CLemon O(2022)Developing a Social Conversational Robot for the Hospital waiting room2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN53752.2022.9900827(1352-1357)Online publication date: 29-Aug-2022
https://dl.acm.org/doi/10.1109/RO-MAN53752.2022.9900827
Pantazopoulos GBruyere JNikandrou MBoissier THemanthage SSachish BShah VDondrup CLemon O(2021)ViCA: Combining visual, social, and task-oriented conversational AI in a healthcare settingProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479909(71-79)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479909
Gunson NSieińska WYu YGarcia DPart JDondrup CLemon O(2021)CoronabotProceedings of the Conference on Information Technology for Social Good10.1145/3462203.3475874(265-270)Online publication date: 9-Sep-2021
https://dl.acm.org/doi/10.1145/3462203.3475874

Index Terms

It's Good to Chat?: Evaluation and Design Guidelines for Combining Open-Domain Social Conversation with Task-Based Dialogue in Intelligent Buildings
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

The Effects of Anthropomorphism and Non-verbal Social Behaviour in Virtual Assistants
IVA '19: Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents

The adoption of virtual assistants is growing at a rapid pace. However, these assistants are not optimised to simulate key social aspects of human conversational environments. Humans are intellectually biased toward social activity when facing ...
Effects of Age-Related Cognitive Decline on Elderly User Interactions with Voice-Based Dialogue Systems
Human-Computer Interaction – INTERACT 2019
Abstract
Cognitive functioning that affects user behaviors is an important factor to consider when designing interactive systems for the elderly, including emerging voice-based dialogue systems such as smart speakers and voice assistants. Previous studies ...
Is chatting with a sophisticated chatbot as good as chatting online or FTF with a stranger?
Abstract
Emotionally-responsive chatbots are marketed as agents with which one can form emotional connections. They can also become weak ties in the outer layers of one's acquaintance network and available for social support. In this experiment,...
Highlights
- Adults had positive reactions after talking to the emotionally responsive Replika chatbot.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

October 2020

394 pages

ISBN:9781450375863

DOI:10.1145/3383652

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020 Framework Programme

Conference

IVA '20

Sponsor:

SIGAI

IVA '20: ACM International Conference on Intelligent Virtual Agents

October 20 - 22, 2020

Scotland, Virtual Event, UK

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
231
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gunson NGarcía DSieińska WDondrup CLemon O(2022)Developing a Social Conversational Robot for the Hospital waiting room2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN53752.2022.9900827(1352-1357)Online publication date: 29-Aug-2022
https://dl.acm.org/doi/10.1109/RO-MAN53752.2022.9900827
Pantazopoulos GBruyere JNikandrou MBoissier THemanthage SSachish BShah VDondrup CLemon O(2021)ViCA: Combining visual, social, and task-oriented conversational AI in a healthcare settingProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479909(71-79)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479909
Gunson NSieińska WYu YGarcia DPart JDondrup CLemon O(2021)CoronabotProceedings of the Conference on Information Technology for Social Good10.1145/3462203.3475874(265-270)Online publication date: 9-Sep-2021
https://dl.acm.org/doi/10.1145/3462203.3475874

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents