More Web Proxy on the site http://driver.im/

extended-abstract

Towards Designing Enthusiastic AI Agents

Authors:

Malihe AlikhaniAuthors Info & Claims

IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents

Pages 203 - 205

https://doi.org/10.1145/3472306.3478366

Published: 14 September 2021 Publication History

Abstract

Immersive virtual worlds are increasingly being used for education, training, and entertainment, and virtual humans that can interact with human users in these worlds play many important roles. Understating the emotional constructs of the user and generating multimodal forms of communications that are aligned with the user's needs and input is key to designing AI agents. Most virtual agents and communicative systems lack the ability to understand enthusiasm or generate multimodal enthusiastic communicative presentations. In this work, we argue for the importance of including enthusiasm in the design of human-AI collaboration and communication and review the existing datasets and models that can be used to bridge the gap in this area.

References

[1]

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (2020), 82--115.

Digital Library

[2]

Eve Ash. 2018. The 15 traits of ideal co-workers: How many do you know? https://www.smartcompany.com.au/people-human-resources/fifteen-traits-of-ideal-co-workers/

[3]

Paolo Baggia, Paul Bagshaw, Michael Bodell, De Zhi Huang, Lou Xiaoyan, Scott McGlashan, Jianhua Tao, Yan Jun, Hu Fang, Yongguo Kang, et al. 2010. Speech synthesis markup language (SSML) version 1.1. World Wide Web Consortium, Recommendation REC-speechsynthesis11-20100907 (2010).

[4]

Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2016. Openface: an open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1--10.

[5]

Edward M Bettencourt, Maxwell H Gillett, Meredith Damien Gall, and Ray E Hull. 1983. Effects of teacher enthusiasm training on student on-task behavior and achievement. American educational research journal 20, 3 (1983), 435--450.

[6]

Susana Castillo, Katharina Legde, and Douglas W Cunningham. 2018. The semantic space for motion-captured facial expressions. Computer Animation and Virtual Worlds 29, 3--4 (2018), e1823.

[7]

Chaona Chen, Oliver GB Garrod, Robin AA Ince, Mary Ellen Foster, Philippe G Schyns, and Rachael E Jack. 2020. Building Culturally-Valid Dynamic Facial Expressions for a Conversational Virtual Agent Using Human Perception. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. 1--3.

Digital Library

[8]

Chaona Chen, Oliver GB Garrod, Jiayu Zhan, Jonas Beskow, Philippe G Schyns, and Rachael E Jack. 2018. Reverse engineering psychologically valid facial expressions of emotion into social robots. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 448--452.

Digital Library

[9]

Douglas W Cunningham, Mario Kleiner, Christian Wallraven, and Heinrich H Bülthoff. 2005. Manipulating video sequences to determine the components of conversational facial expressions. ACM Transactions on Applied Perception (TAP) 2, 3 (2005), 251--269.

Digital Library

[10]

Kevin Daum. 2021. 5 Desirable Traits of Great Employees. https://www.inc.com/kevin-daum/5-tests-hiring-best-employees.html

[11]

Mira El Kamali, Leonardo Angelini, Maurizio Caon, Giuseppe Andreoni, Omar Abou Khaled, and Elena Mugellini. 2018. Towards the NESTORE e-Coach: a tangible and embodied conversational agent for older adults. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 1656--1663.

Digital Library

[12]

"enthusiasm". 2021. Merriam-Webster.com. https://www.merriam-webster.com

[13]

Florian Eyben, Klaus R Scherer, Björn W Schuller, Johan Sundberg, Elisabeth André, Carlos Busso, Laurence Y Devillers, Julien Epps, Petri Laukka, Shrikanth S Narayanan, et al. 2015. The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE transactions on affective computing 7, 2 (2015), 190--202.

[14]

E Friesen and Paul Ekman. 1978. Facial action coding system: a technique for the measurement of facial movement. Palo Alto 3, 2 (1978), 5.

[15]

Jonathan Gratch, Ning Wang, Jillian Gerten, Edward Fast, and Robin Duffy. 2007. Creating rapport with virtual agents. In International workshop on intelligent virtual agents. Springer, 125--138.

Digital Library

[16]

Dwayne D Gremler and Kevin P Gwinner. 2008. Rapport-building behaviors used by retail employees. Journal of Retailing 84, 3 (2008), 308--324.

[17]

Michimasa Inaba, Fujio Toriumi, and Kenichiro Ishii. 2011. Automatic detection of "enthusiasm" in non-task-oriented dialogues using word co-occurrence. In 2011 IEEE Workshop on Affective Computational Intelligence (WACI). IEEE, 1--7.

[18]

Shaidah Jusoh. 2018. Intelligent conversational agent for online sales. In 2018 10th International Conference on Electronics, Computers and Artificial Intelligence (ECAI). IEEE, 1--4.

[19]

Melanie M Keller, Thomas Goetz, Eva S Becker, Vinzenz Morger, and Lauren Hensley. 2014. Feeling and showing: A new conceptualization of dispositional teacher enthusiasm and its relation to students' interest. Learning and Instruction 33 (2014), 29--38.

[20]

Melanie M Keller, Anita Woolfolk Hoy, Thomas Goetz, and Anne C Frenzel. 2016. Teacher enthusiasm: Reviewing and redefining a complex construct. Educational Psychology Review 28, 4 (2016), 743--769.

[21]

Mohammad Rami Koujan, Michail Christos Doukas, Anastasios Roussos, and Stefanos Zafeiriou. 2020. Head2head: Video-based neural head synthesis. arXiv preprint arXiv:2005.10954 (2020).

[22]

J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. Biometrics (1977), 159--174.

[23]

David L Largent. 2016. Measuring and understanding team development by capturing self-assessed enthusiasm and skill levels. ACM Transactions on Computing Education (TOCE) 16, 2 (2016), 1--27.

Digital Library

[24]

Tze Wei Liew, Su-Mae Tan, and Chin Lay Gan. 2018. Interacting With Motivational Virtual Agent: The Effects of Message Framing and Regulatory Fit in an E-Learning Environment. In 2018 Thirteenth International Conference on Digital Information Management (ICDIM). IEEE, 136--141.

[25]

Tze Wei Liew, Su-Mae Tan, Teck Ming Tan, and Si Na Kew. 2020. Does speaker's voice enthusiasm affect social cue, cognitive load and transfer in multimedia learning? Information and Learning Sciences (2020).

[26]

Tze Wei Liew, Nor Azan Mat Zin, and Noraidah Sahari. 2017. Exploring the affective, motivational and cognitive effects of pedagogical agent enthusiasm in a multimedia learning environment. Human-centric Computing and Information Sciences 7, 1 (2017), 9.

Digital Library

[27]

Penny Loretto. 2019. Eight Qualities of a Good Mentor. https://www.thebalancecareers.com/qualities-of-a-good-mentor-1986663

[28]

Scott Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07874 (2017).

Digital Library

[29]

Tetsuya Matsui and Seiji Yamada. 2019. The design method of the virtual teacher. In Proceedings of the 7th International Conference on Human-Agent Interaction. 97--101.

Digital Library

[30]

Reo Matsumura, Masahiro Shiomi, and Norihiro Hagita. 2017. Does an animation character robot increase sales?. In Proceedings of the 5th International Conference on Human Agent Interaction. 479--482.

Digital Library

[31]

Kelly Mitchell. 2018. #1 Desired Trait? ENTHUSIASM! http://www.socialjusticesolutions.org/2018/04/18/1-desired-trait-enthusiasm/

[32]

Simon Provoost, Jeroen Ruwaard, Koen Neijenhuijs, Tibor Bosse, and Heleen Riper. 2018. Mood mirroring with an embodied virtual agent: a pilot study on the relationship between personalized visual feedback and adherence. In International Conference on Practical Applications of Agents and Multi-Agent Systems. Springer, 24--35.

[33]

Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. 2019. Autovc: Zero-shot voice style transfer with only autoencoder loss. In International Conference on Machine Learning. PMLR, 5210--5219.

[34]

Birgitta Sandberg. 2007. Enthusiasm in the development of radical innovations. Creativity and Innovation Management 16, 3 (2007), 265--273.

[35]

Lisa M Schreiber, Gregory D Paul, and Lisa R Shibley. 2012. The development and test of the public speaking competence rubric. Communication Education 61, 3 (2012), 205--233.

[36]

Ameneh Shamekhi, Ha Trinh, Timothy W Bickmore, Tamara R DeAngelis, Theresa Ellis, Bethlyn V Houlihan, and Nancy K Latham. 2016. A virtual self-care coach for individuals with spinal cord injury. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility. 327--328.

Digital Library

[37]

Sinan Sonlu, Uğur Güdükbay, and Funda Durupinar. 2021. A Conversational Agent Framework with Multi-modal Personality Expression. ACM Transactions on Graphics (TOG) 40, 1 (2021), 1--16.

Digital Library

[38]

Ryoko Tokuhisa and Ryuta Terashima. 2006. Relationship between utterances and "enthusiasm" in non-task-oriented conversational dialogue. In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue. 161--167.

Digital Library

[39]

Pascal van Opzeeland. 2017. The 8 Customer Service Skills And Traits You Should Look For. https://www.userlike.com/en/blog/customer-service-skills-traits

[40]

Carla Viegas and Malihe Alikhani. 2021. Entheos: A Multimodal Dataset for Studying Enthusiasm. In Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021).

[41]

Isaac Wang and Jaime Ruiz. 2021. Examining the Use of Nonverbal Communication in Virtual Agents. International Journal of Human-Computer Interaction (2021), 1--26.

[42]

Klaus Weber, Lukas Tinnes, Tobias Huber, Alexander Heimerl, Marc-Leon Reinecker, Eva Pohlen, and Elisabeth André. 2020. Towards demystifying subliminal persuasiveness: using XAI-techniques to highlight persuasive markers of public speeches. In International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems. Springer, 113--128.

[43]

Wei Xue, Catia Cucchiarini, RWNM van Hout, and Helmer Strik. 2019. Acoustic correlates of speech intelligibility. The usability of the eGeMAPS feature set for atypical speech. (2019).

[44]

Qin Zhang. 2014. Assessing the effects of instructor enthusiasm on classroom engagement, learning goal orientation, and academic self-efficacy. Communication Teacher 28, 1 (2014), 44--56.

[45]

Zhenjie Zhao and Xiaojuan Ma. 2020. Situated Learning of Soft Skills with an Interactive Agent in Virtual Reality via Multimodal Feedback. In Adjunct Publication of the 33rd Annual ACM Symposium on User Interface Software and Technology. 25--27.

Digital Library

[46]

Kun Zhou, Berrak Sisman, Rui Liu, and Haizhou Li. 2020. Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset. arXiv preprint arXiv:2010.14794 (2020).

Cited By

Dubiel MLeiva LBongard-Blanchy KSergeeva A(2024)“Hey Genie, You Got Me Thinking about My Menu Choices!” Impact of Proactive Feedback on User Perception and Reflection in Decision-making TasksACM Transactions on Computer-Human Interaction10.1145/368527431:5(1-30)Online publication date: 29-Jul-2024
https://dl.acm.org/doi/10.1145/3685274
Liew TTan SPang WKhan MKew S(2022)I am Alexa, your virtual tutor!: The effects of Amazon Alexa’s text-to-speech voice enthusiasm in a multimedia learning environmentEducation and Information Technologies10.1007/s10639-022-11255-628:2(1455-1489)Online publication date: 30-Jul-2022
https://dl.acm.org/doi/10.1007/s10639-022-11255-6

Index Terms

Towards Designing Enthusiastic AI Agents
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Adapted Foreigner-directed Communication towards Virtual Agents
IVA '18: Proceedings of the 18th International Conference on Intelligent Virtual Agents

People tend to adapt to their conversation partners. In mixed-cultural settings, with a native and a non-native speaker, adaptation can manifest itself in the usage of simplified language or increased usage of nonverbal scaffolding to foster ...
Towards a smiling ECA: studies on mimicry, timing and types of smiles
SSPW '10: Proceedings of the 2nd international workshop on Social signal processing

Smile is one of the most often used nonverbal signals. Depending on when, how and where it is displayed, it may convey various meanings. We believe that introducing the variety of smiles may improve the communicative skills of embodied conversational ...
Towards healthcare personal agents
RFMIR '14: Proceedings of the 2014 Workshop on Roadmapping the Future of Multimodal Interaction Research including Business Opportunities and Challenges

For a long time, the research on human-machine conversation and interaction has inspired futuristic visions created by film directors and science fiction writers. Nowadays, there has been great progress towards this end by the extended community of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents

September 2021

238 pages

ISBN:9781450386197

DOI:10.1145/3472306

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021

Check for updates

Badges

Honorable Mention Short Paper

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

IVA '21

Sponsor:

SIGAI

IVA '21: ACM International Conference on Intelligent Virtual Agents

September 14 - 17, 2021

Virtual Event, Japan

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
196
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)7

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dubiel MLeiva LBongard-Blanchy KSergeeva A(2024)“Hey Genie, You Got Me Thinking about My Menu Choices!” Impact of Proactive Feedback on User Perception and Reflection in Decision-making TasksACM Transactions on Computer-Human Interaction10.1145/368527431:5(1-30)Online publication date: 29-Jul-2024
https://dl.acm.org/doi/10.1145/3685274
Liew TTan SPang WKhan MKew S(2022)I am Alexa, your virtual tutor!: The effects of Amazon Alexa’s text-to-speech voice enthusiasm in a multimedia learning environmentEducation and Information Technologies10.1007/s10639-022-11255-628:2(1455-1489)Online publication date: 30-Jul-2022
https://dl.acm.org/doi/10.1007/s10639-022-11255-6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents