More Web Proxy on the site http://driver.im/

research-article

Open access

The Role of Lexical Alignment in Human Understanding of Explanations by Conversational Agents

Authors:

Sumit Srivastava,

Mariët Theune,

Alejandro CatalaAuthors Info & Claims

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

Pages 423 - 435

https://doi.org/10.1145/3581641.3584086

Published: 27 March 2023 Publication History

All formats PDF

Abstract

Explainable Artificial Intelligence (XAI) focuses on research and technology that can explain an AI system’s functioning and its underlying methods, and also on making these explanations better through personalization. Our research study investigates a natural language personalization method called lexical alignment in understanding an explanation provided by a conversational agent. The study setup was online and navigated the participants through an interaction with a conversational agent. Participants faced either an agent designed to align its responses to those of the participants, a misaligned agent, or a control condition that did not involve any dialogue. The dialogue delivered an explanation based on a pre-defined set of causes and effects. The recall and understanding of the explanations was evaluated using a combination of Yes-No questions, a Cloze test (fill-in-the-blanks), and What-style questions. The analysis of the test scores revealed a significant advantage in information recall for those who interacted with an aligning agent against the participants who either interacted with a non-aligning agent or did not go through any dialogue. The Yes-No type questions that included probes on higher-order inferences (understanding) also reflected an advantage for the participants who had an aligned dialogue against both non-aligned and no dialogue conditions. The results overall suggest a positive effect of lexical alignment on understanding of explanations.

References

[1]

[1] Sabita Acharya, Andrew Dallas Boyd, Richard Cameron, Karen Dunn Lopez, Pamela Martyn-Nemeth, Carolyn Dickens, Amer Ardati, Jose D Flores, Matthew Baumann, Betty Welland, and Barbara Maria Di Eugenio. 2019. What Happened to Me while I Was in the Hospital? Challenges and Opportunities for Generating Patient-Friendly Hospitalization Summaries. Journal of Healthcare Informatics Research 3 (2019), 107–123.

[2]

[2] Imran Ahmed, Gwanggil Jeon, and Francesco Piccialli. 2022. From artificial intelligence to explainable artificial intelligence in Industry 4.0: a survey on what, how, and where. IEEE Transactions on Industrial Informatics 18, 8 (2022), 5031–5042.

[3]

[3] Gulsum Alicioglu and Bo Sun. 2022. A survey of visual analytics for Explainable Artificial Intelligence methods. Computers & Graphics 102 (2022), 502–520. https://doi.org/10.1016/j.cag.2021.09.002

Digital Library

[4]

[4] Lorin W. Anderson, David R. Krathwohl, Peter W. Airasian, Kathleen A. Cruikshank, Richard E. Mayer, Paul R. Pintrich, James Raths, and Merlin C. Wittrock. 2000. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Longman, New York.

[5]

[5] A Arnellis, E Z Jamaan, and N Amalita. 2018. Efforts to Improve Mathematics Teacher Competency Through Training Program on Design Olympiad Mathematics Problems Based on Higher Order Thinking Skills in The Junior High School. IOP Conference Series: Materials Science and Engineering 335, 1 (apr 2018), 012118. https://doi.org/10.1088/1757-899X/335/1/012118

[6]

[6] Simran Arora, Avner May, Jian Zhang, and Christopher Ré. 2020. Contextual Embeddings: When Are They Worth It?. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 2650–2663. https://doi.org/10.18653/v1/2020.acl-main.236

[7]

[7] Benjamin Samuel Bloom. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals.Longmans, Green, New York.

[8]

[8] Holly P. Branigan, Martin John Pickering, Jamie Pearson, and Janet McLean. 2010. Linguistic alignment between people and computers. Journal of Pragmatics 42 (2010), 2355–2368.

[9]

[9] Erik Cambria, Lorenzo Malandri, Fabio Mercorio, Mario Mezzanzanica, and Navid Nobani. 2023. A survey on XAI and natural language explanations. Information Processing & Management 60, 1 (2023), 103111.

Digital Library

[10]

[10] Giuseppe Carenini and Johanna D. Moore. 2000. An Empirical Study of the Influence of Argument Conciseness on Argument Effectiveness. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics (Hong Kong) (ACL ’00). Association for Computational Linguistics, USA, 150–157. https://doi.org/10.3115/1075218.1075238

Digital Library

[11]

[11] Shuo Chang, F. Maxwell Harper, and Loren Gilbert Terveen. 2016. Crowd-Based Personalized Natural Language Explanations for Recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (Boston, Massachusetts, USA) (RecSys ’16). Association for Computing Machinery, New York, NY, USA, 175–182. https://doi.org/10.1145/2959100.2959153

Digital Library

[12]

[12] Amy Cheng, Vaishnavi Raghavaraju, Jayanth Kanugo, Yohanes P. Handrianto, and Yi Shang. 2018. Development and evaluation of a healthy coping voice interface application using the Google home for elderly patients with type 2 diabetes. In 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC). 1–5. https://doi.org/10.1109/CCNC.2018.8319283

Digital Library

[13]

[13] Guillaume Dubuisson Duplessis, Caroline Langlet, Chloé Clavel, and Frédéric Landragin. 2021. Towards alignment strategies in human-agent interactions based on measures of lexical repetitions. Language Resources and Evaluation 55 (2021), 353 – 388.

Digital Library

[14]

[14] Mohammadamin Erfanmanesh, Abdullah Abrizah, and Noor Al-Huda Abdul Karim. 2014. The prevalence and correlates of information seeking anxiety in postgraduate students. Malaysian Journal of Library & Information Science 19 (2014).

[15]

[15] Stefan Th. Gries. 2005. Syntactic Priming: A Corpus-based Approach. Journal of Psycholinguistic Research 34 (2005), 365–399.

[16]

[16] David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. 2019. XAI—Explainable artificial intelligence. Science Robotics 4, 37 (2019), eaay7120. https://doi.org/10.1126/scirobotics.aay7120 arXiv:https://www.science.org/doi/pdf/10.1126/scirobotics.aay7120

[17]

[17] Benjamin Heinzerling and Michael Strube. 2018. BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Nicoletta Calzolari (Conference chair), Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Koiti Hasida, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis, and Takenobu Tokunaga (Eds.). European Language Resources Association (ELRA), Miyazaki, Japan.

[18]

[18] Rens Hoegen, Deepali Aneja, Daniel McDuff, and Mary Czerwinski. 2019. An End-to-End Conversational Style Matching Agent. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (Paris, France) (IVA ’19). Association for Computing Machinery, New York, NY, USA, 111–118. https://doi.org/10.1145/3308532.3329473

Digital Library

[19]

[19] Regina Jucks and Elisabeth Paus. 2013. Different Words for the Same Concept: Learning Collaboratively From Multiple Documents. Cognition and Instruction 31, 2 (2013), 227–254. https://doi.org/10.1080/07370008.2013.769993 arXiv:https://doi.org/10.1080/07370008.2013.769993

[20]

[20] Jongkwang Kim and Thomas Wilhelm. 2008. What is a complex graph. Physica A-statistical Mechanics and Its Applications 387 (2008), 2637–2652.

[21]

[21] Ahmet Baki Kocaballi, Shlomo Berkovsky, Juan C Quiroz, Liliana Laranjo, Huong Ly Tong, Dana Rezazadegan, Agustina Briatore, and Enrico Coiera. 2019. The Personalization of Conversational Agents in Health Care: Systematic Review. J Med Internet Res 21, 11 (7 Nov 2019), e15360. https://doi.org/10.2196/15360

[22]

[22] David R. Krathwohl. 2002. A Revision of Bloom’s Taxonomy: An Overview. Theory Into Practice 41 (2002), 212 – 218.

[23]

[23] Diane J. Litman and Scott Silliman. 2004. ITSPOKE: An Intelligent Tutoring Spoken Dialogue System. In Demonstration Papers at HLT-NAACL 2004 (Boston, Massachusetts) (HLT-NAACL–Demonstrations ’04). Association for Computational Linguistics, USA, 5–8.

[24]

[24] Anna A. Meldo, Lev V. Utkin, Maxim S. Kovalev, and Ernest M. Kasimov. 2020. The natural language explanation algorithms for the lung cancer computer-aided diagnosis system. Artificial intelligence in medicine 108 (2020), 101952.

[25]

[25] Christian Meske, Enrico Bunde, Johannes Schneider, and Martin Gersch. 2020. Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities. Information Systems Management 39 (2020), 53 – 63.

[26]

[26] Elham Mousavinasab, Nahid Zarifsanaiey, Sharareh Rostam Niakan Kalhori, Mahnaz Rakhshan, Leila Keikha, and Marjan Ghazi Saeedi. 2021. Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments 29 (2021), 142 – 163.

[27]

[27] Van Bach Nguyen, Jörg Schlötterer, and Christin Seifert. 2022. Explaining Machine Learning Models in Natural Conversations: Towards a Conversational XAI Agent. ArXiv abs/2209.02552 (2022).

[28]

[28] Elisabeth Paus and Regina Jucks. 2008. Do We Really Mean the Same? The Relationship between Word Choices and Computer Mediated Cooperative Learning. In Proceedings of the 8th International Conference on International Conference for the Learning Sciences - Volume 2 (Utrecht, The Netherlands) (ICLS’08). International Society of the Learning Sciences, 172–179.

[29]

[29] Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1532–1543. https://doi.org/10.3115/v1/D14-1162

[30]

[30] Martin J. Pickering and Simon Garrod. 2004. Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences 27, 2 (2004), 169–190. https://doi.org/10.1017/S0140525X04000056

[31]

[31] Martin John Pickering and Simon Garrod. 2006. Alignment as the Basis for Successful Communication. Research on Language and Computation 4 (2006), 203–228.

[32]

[32] Chong Eun Rhee and Junho H. Choi. 2020. Effects of personalization and social role in voice shopping: An experimental study on product recommendation by a conversational voice agent. Computers in Human Behavior 109 (2020), 106359.

[33]

[33] Stuart J. Russell and Peter Norvig. 2009. Artificial Intelligence: a modern approach (3rd ed.). Pearson.

[34]

[34] Johannes Schneider and Joshua Peter Handali. 2019. Personalized Explanation for Machine Learning: a Conceptualization. In 27th European Conference on Information Systems (ECIS). Association for Information Systems, Stockholm & Uppsala, Sweden. https://aisel.aisnet.org/ecis2019_rp/171/

[35]

[35] Emre Sezgin, Garey Noritz, Simon Lin, and Yungui Huang. 2021. Feasibility of a Voice-Enabled Medical Diary App (SpeakHealth) for Caregivers of Children With Special Health Care Needs and Health Care Providers: Mixed Methods Study. JMIR Form Res 5, 5 (11 May 2021), e25503. https://doi.org/10.2196/25503

[36]

[36] Timo Speith. 2022. A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 2239–2250. https://doi.org/10.1145/3531146.3534639

Digital Library

[37]

[37] Laura Spillner and Nina Wenig. 2021. Talk to Me on My Level – Linguistic Alignment for Chatbots. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (Toulouse; Virtual, France) (MobileHCI ’21). Association for Computing Machinery, New York, NY, USA, Article 45, 12 pages. https://doi.org/10.1145/3447526.3472050

Digital Library

[38]

[38] Ilia Stepin, Jose M. Alonso, Alejandro Catalá, and Martin Pereira-Fariña. 2021. A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence. IEEE Access 9 (2021), 11974–12001.

[39]

[39] Ilia Stepin, Jose M. Alonso, Alejandro Gatala, and Martin Pereira-Fariña. 2020. Generation and Evaluation of Factual and Gounterfaetual Explanations for Decision Trees and Fuzzy Rule-Based Classifiers. In 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (Glasgow, United Kingdom). IEEE Press, 1–8. https://doi.org/10.1109/FUZZ48607.2020.9177629

Digital Library

[40]

[40] Marian Swope and Jeffrey Katzer. 1972. The Silent Majority: Why Don’t They Ask Questions?RQ 12, 2 (1972), 161–166.

[41]

[41] Deborah Tannen. 1987. Repetition in conversation: toward a poetics of talk. Language 63, 3 (1987), 574–605.

[42]

[42] Wilson L. Taylor. 1953. “Cloze Procedure”: A New Tool for Measuring Readability. Journalism & Mass Communication Quarterly 30 (1953), 415 – 433.

[43]

[43] Wilson L. Taylor. 1957. ’Cloze’ Readability Scores as Indices of Individual Differences in Comprehension and Aptitude: Erratum.Journal of Applied Psychology 41 (1957), 19–26.

[44]

[44] Paul Thomas, Daniel McDuff, Mary Czerwinski, and Nick Craswell. 2020. Expressions of Style in Information Seeking Conversation with an Agent. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, China) (SIGIR ’20). Association for Computing Machinery, New York, NY, USA, 1171–1180. https://doi.org/10.1145/3397271.3401127

Digital Library

[45]

[45] Cynthia A. Thompson, Mehmet H. Göker, and Pat Langley. 2004. A Personalized System for Conversational Recommendations. Journal of Artificial Intelligence Research 21, 1 (2004), 393–428.

Digital Library

[46]

[46] Lev V. Utkin, Anna A. Meldo, Maxim S. Kovalev, and Ernest M. Kasimov. 2020. A Simple General Algorithm for the Diagnosis Explanation of Computer-Aided Diagnosis Systems in Terms of Natural Language Primitives. In 2020 XXIII International Conference on Soft Computing and Measurements (SCM). 202–205. https://doi.org/10.1109/SCM50615.2020.9198764

[47]

[47] Laura H. Waite, Jason F Zupec, Diane H. Quinn, and Cathy Y. Poon. 2020. Revised Bloom’s taxonomy as a mentoring framework for successful promotion.Currents in pharmacy teaching & learning 12 11 (2020), 1379–1382.

[48]

[48] Arthur Ward and Diane J Litman. 2007. Automatically measuring lexical and acoustic/prosodic convergence in tutorial dialog corpora. In Seventh ISCA Workshop on Speech and Language Technology in Education (SLaTE). http://d-scholarship.pitt.edu/23210/

[49]

[49] Arthur Ward and Diane J. Litman. 2007. Measuring Convergence and Priming in Tutorial Dialog. Technical Report TR-07-148. University of Pittsburgh.

[50]

[50] Chen Henry Wu, Yinhe Zheng, Xiaoxi Mao, and Minlie Huang. 2021. Transferable Persona-Grounded Dialogues via Grounded Minimal Edits. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2368–2382. https://doi.org/10.18653/v1/2021.emnlp-main.183

[51]

[51] Mingzhi Yu, Diane J. Litman, Shuang Ma, and Jian Wu. 2021. A Neural Network-Based Linguistic Similarity Measure for Entrainment in Conversations. CoRR abs/2109.01924 (2021). arXiv:2109.01924https://arxiv.org/abs/2109.01924

[52]

[52] Guoshuai Zhao, Hao Fu, Ruihua Song, Tetsuya Sakai, Zhongxia Chen, Xing Xie, and Xueming Qian. 2019. Personalized Reason Generation for Explainable Song Recommendation. ACM Transactions on Intelligent Systems and Technology (TIST) 10 (2019), 1 – 21.

Digital Library

[53]

[53] Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu. 2019. Personalized Dialogue Generation with Diversified Traits. https://doi.org/10.48550/ARXIV.1901.09672

Cited By

Srivastava SWentzel SCatala ATheune M(2025)Measuring and implementing lexical alignment: A systematic literature reviewComputer Speech & Language10.1016/j.csl.2024.10173190(101731)Online publication date: Mar-2025
https://doi.org/10.1016/j.csl.2024.101731
Zhao ZTheune MSrivastava SBraun D(2024)Exploring Lexical Alignment in a Price Bargain ChatbotProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665576(1-7)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665576
Srivastava STheune MCatala AReed CBoratto LGena CMarras MGermanakos PPopescus E(2024)Trust in a Human-Computer Collaborative Task With or Without Lexical AlignmentAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664868(189-194)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3664868
Show More Cited By

Index Terms

The Role of Lexical Alignment in Human Understanding of Explanations by Conversational Agents
1. Computing methodologies
  1. Artificial intelligence
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Natural language interfaces

Index terms have been assigned to the content through auto-classification.

Recommendations

Talk to Me on My Level – Linguistic Alignment for Chatbots
MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction

Digital companions and conversational agents are becoming increasingly popular in our everyday lives. Natural language interfaces play an important role in ubiquitous computing: voice assistants are used to control smart home devices and smartphone ...
Lexical Entrainment in Multi-party Human–Robot Interaction
Social Robotics
Abstract
This paper reports lexical entrainment in a multi-party human–robot interaction, wherein one robot and two humans serve as participants. Humans tend to use the same terms as their interlocutors while making conversation. This phenomenon is called ...
Effects of social skills on lexical alignment in human-human interaction and human-computer interaction
Abstract
Over the past decade, people's language behaviors towards computer partners have been capturing growing interests with the prevalence of interaction with dialogue systems. However, it remains controversial whether people perceive and ...
Highlights
- People were more likely to align with a computer than a human in lexical use.
- ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

March 2023

972 pages

ISBN:9798400701061

DOI:10.1145/3581641

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

European Union?s Horizon 2020 Marie Sk?odowska-Curie

Conference

IUI '23

Sponsor:

IUI '23: 28th International Conference on Intelligent User Interfaces

March 27 - 31, 2023

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
1,965
Total Downloads

Downloads (Last 12 months)1,571
Downloads (Last 6 weeks)98

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Srivastava SWentzel SCatala ATheune M(2025)Measuring and implementing lexical alignment: A systematic literature reviewComputer Speech & Language10.1016/j.csl.2024.10173190(101731)Online publication date: Mar-2025
https://doi.org/10.1016/j.csl.2024.101731
Zhao ZTheune MSrivastava SBraun D(2024)Exploring Lexical Alignment in a Price Bargain ChatbotProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665576(1-7)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665576
Srivastava STheune MCatala AReed CBoratto LGena CMarras MGermanakos PPopescus E(2024)Trust in a Human-Computer Collaborative Task With or Without Lexical AlignmentAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664868(189-194)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3664868
Szczuka JMühl L(2024)The Usage of Voice in Sexualized Interactions with Technologies and Sexual Health Communication: An OverviewCurrent Sexual Health Reports10.1007/s11930-024-00383-416:2(47-57)Online publication date: 27-Mar-2024
https://doi.org/10.1007/s11930-024-00383-4

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents