Just Accepted
Impact of Tone-Aware Explanations in Recommender Systems
Abstract
In recommender systems, explanations are essential for supporting users’ decision-making processes. While many studies have focused on explanation content or user interface, the expression of textual explanations has been largely overlooked. The expression refers to textual styles such as formal or humorous, which we call tone in this paper. Although tone contributes to smooth human communication, its impact on users’ perceptions of recommender systems remains largely unexplored. In particular, it is unclear whether the perceived effects of explanation tone differ by domain or user attributes. Therefore, we investigate the effects of explanation tones through two online user studies considering domains and user attributes. In the first study with 470 participants, we generated datasets using a large language model to create fictional items and explanations with six tones across three domains: movies, hotels, and home products. The participants evaluated two explanations for an item, each presented in a different tone, and rated ten metrics. In the second study with 103 participants, we used a real-world dataset from the hotel domain and incorporated a simple personalized recommender system to examine effects of tone in a more realistic setting. The results revealed that the perceived effects of tones differ by domain and are significantly influenced by user attributes such as age and personality traits. Our findings suggest that appropriately adjusting the tone of explanations according to domains and user attributes can enhance the perceived effects of recommender systems.
References
[1]
Douglas Biber and Susan Conrad. 2019. Register, genre, and style. Cambridge University Press.
[2]
Nancy Bonvillain. 2019. Language, culture, and communication: The meaning of messages. Rowman & Littlefield.
[3]
Eleftheria Briakou, Sweta Agrawal, Ke Zhang, Joel Tetreault, and Marine Carpuat. 2021. A Review of Human Evaluation for Style Transfer. In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics. Association for Computational Linguistics, 58–67.
[4]
Shuo Chang, F Maxwell Harper, and Loren Gilbert Terveen. 2016. Crowd-based personalized natural language explanations for recommendations. In RecSys. ACM, 175–182.
[5]
Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Neural attentional rating regression with review-level explanations. In WWW. ACM, 1583–1592.
[6]
Jessie Chin, Smit Desai, Sheny Lin, and Shannon Mejia. 2024. Like My Aunt Dorothy: Effects of Conversational Styles on Perceptions, Acceptance and Metaphorical Descriptions of Voice Assistants during Later Adulthood. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1(2024), 1–21.
[7]
Robert B Cialdini. 2001. The science of persuasion. Scientific American 284, 2 (2001), 76–81.
[8]
Dan Cosley, Shyong K Lam, Istvan Albert, Joseph A Konstan, and John Riedl. 2003. Is seeing believing? How recommender system interfaces affect users’ opinions. In SIGCHI. ACM, 585–592.
[9]
Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten Van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. UMUAI (2008), 455–496.
[10]
Sahraoui Dhelim, Nyothiri Aung, Mohammed Amine Bouras, Huansheng Ning, and Erik Cambria. 2022. A survey on personality-aware recommendation systems. Artificial Intelligence Review(2022), 1–46.
[11]
Zeshan Fayyaz, Mahsa Ebrahimian, Dina Nawara, Ahmed Ibrahim, and Rasha Kashef. 2020. Recommendation systems: Algorithms, challenges, metrics, and business opportunities. applied sciences 10, 21 (2020), 7748.
[12]
Fatih Gedikli, Dietmar Jannach, and Mouzhi Ge. 2014. How should I explain? A comparison of different explanation types for recommender systems. International Journal of Human-Computer Studies 72, 4 (2014), 367–382.
[13]
Zohar Gilad, Ofra Amir, and Liat Levontin. 2021. The effects of warmth and competence perceptions on users’ choice of an AI system. In CHI. 1–13.
[14]
Howard Giles, Tania Ogay, et al. 2007. Communication accommodation theory. (2007).
[15]
Sofia Gkika and George Lekakos. 2014. The Persuasive Role of Explanations in Recommender Systems. In BCSS@ PERSUASIVE. 59–68.
[16]
Carlos A Gomez-Uribe and Neil Hunt. 2015. The Netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS) 6, 4(2015), 1–19.
[17]
Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. 2003. A very brief measure of the Big-Five personality domains. Journal of Research in personality 37, 6 (2003), 504–528.
[18]
Jonathan L Herlocker, Joseph A Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In CSCW. ACM, 241–250.
[19]
Stefan Hirschmeier and Detlef Schoder. 2020. An Approach to Explanations for Public Radio Recommendations. In UMAP adjunct. ACM, 237–240.
[20]
Daniel Horowitz, David Contreras, and Maria Salamó. 2018. EventAware: A mobile recommender system for events. Pattern Recognition Letters 105 (2018), 121–134.
[21]
Min Hou, Le Wu, Enhong Chen, Zhi Li, Vincent W Zheng, and Qi Liu. 2019. Explainable fashion recommendation: A semantic attribute region guided approach. arXiv preprint arXiv:1905.12862(2019).
[22]
Yunfeng Hou, Ning Yang, Yi Wu, and Philip S Yu. 2019. Explainable recommendation with fusion of aspect information. World Wide Web 22(2019), 221–240.
[23]
Yoyo Tsung-Yu Hou, Wen-Ying Lee, and Malte Jung. 2023. “Should I Follow the Human, or Follow the Robot?”—Robots in Power Can Have More Influence Than Humans on Decision-Making. In CHI. 1–13.
[24]
Eduard Hovy. 1987. Generating natural language under pragmatic constraints. Journal of Pragmatics 11, 6 (1987), 689–719.
[25]
Celestine Iwendi, Suleman Khan, Joseph Henry Anajemba, Ali Kashif Bashir, and Fazal Noor. 2020. Realizing an efficient IoMT-assisted patient diet recommendation system through machine learning model. IEEE access 8(2020), 28462–28474.
[26]
Sapna Jaiswal, Tejaswi Kharade, Nikita Kotambe, and Shilpa Shinde. 2020. Collaborative recommendation system for agriculture sector. In ITM web of conferences, Vol. 32. EDP Sciences, 03034.
[27]
Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: A survey. Computational Linguistics 48, 1 (2022), 155–205.
[28]
Oliver P John, Sanjay Srivastava, et al. 1999. The Big-Five trait taxonomy: History, measurement, and theoretical perspectives. (1999).
[29]
Ji-Youn Jung and Alessandro Bozzon. 2023. Are Female Chatbots More Empathic?-Discussing Gendered Conversational Agent through Empathic Design. In Proceedings of the 2nd Empathy-Centric Design Workshop. 1–5.
[30]
Ji-Youn Jung, Sihang Qiu, Alessandro Bozzon, and Ujwal Gadiraju. 2022. Great chain of agents: The role of metaphorical representation of agents in conversational crowdsourcing. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–22.
[31]
Patricia K Kahr, Gerrit Rooks, Martijn C Willemsen, and Chris CP Snijders. 2023. It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task. In IUI. 528–539.
[32]
Bart P Knijnenburg, Martijn C Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction 22 (2012), 441–504.
[33]
Hyeyoung Ko, Suyeon Lee, Yoonseo Park, and Anna Choi. 2022. A survey of recommendation systems: recommendation models, techniques, and application fields. Electronics 11, 1 (2022), 141.
[34]
Pigi Kouki, James Schaffer, Jay Pujara, John O’Donovan, and Lise Getoor. 2019. Personalized explanations for hybrid recommender systems. In IUI. ACM, 379–390.
[35]
Klaus Krippendorff. 2011. Computing Krippendorff’s alpha-reliability.
[36]
Johannes Kunkel, Tim Donkers, Lisa Michael, Catalin-Mihai Barbu, and Jürgen Ziegler. 2019. Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems. In CHI. ACM, 1–12.
[37]
Javier Lacasta, F Javier Lopez-Pellicer, Borja Espejo-García, Javier Nogueras-Iso, and F Javier Zarazaga-Soria. 2018. Agricultural recommendation system for crop protection. Computers and Electronics in Agriculture 152 (2018), 82–89.
[38]
Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2019. Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Transactions on Knowledge and Data Engineering 32, 8(2019), 1502–1516.
[39]
Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 7, 1 (2003), 76–80.
[40]
Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. 2010. Personalized news recommendation based on click behavior. In Proceedings of the 15th international conference on Intelligent user interfaces. 31–40.
[41]
Sebastian Lubos, Thi Ngoc Trang Tran, Alexander Felfernig, Seda Polat Erdeniz, and Viet-Man Le. 2024. LLM-generated Explanations for Recommender Systems. In UMAP adjunct. ACM, 276–285.
[42]
Stephan Ludwig, Ko De Ruyter, Mike Friedman, Elisabeth C Brüggen, Martin Wetzels, and Gerard Pfann. 2013. More than words: The influence of affective content and linguistic style matches in online reviews on conversion rates. Journal of marketing 77, 1 (2013), 87–103.
[43]
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. In ACL-HLT. ACL, 142–150.
[44]
David D McDonald and James Pustejovsky. 1985. A computational theory of prose style for natural language generation. In Second Conference of the European Chapter of the Association for Computational Linguistics.
[45]
Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38.
[46]
Anam Mustaqeem, Syed Muhammad Anwar, and Muhammad Majid. 2020. A modular cluster based collaborative recommender system for cardiac patients. Artificial intelligence in medicine 102 (2020), 101761.
[47]
Cataldo Musto, Pasquale Lops, Marco de Gemmis, and Giovanni Semeraro. 2019. Justifying recommendations through aspect-based sentiment analysis of users reviews. In UMAP. ACM, 4–12.
[48]
Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In EMNLP-IJCNLP. ACL, 188–197.
[49]
Kate G Niederhoffer and James W Pennebaker. 2002. Linguistic style matching in social interaction. Journal of Language and Social Psychology 21, 4 (2002), 337–360.
[50]
Ayano Okoso, Keisuke Otaki, Yoshinao Ishii, and Satoshi Koide. 2024. A Case Study on Recommender Systems in Online Conferences: Behavioral Analysis through A/B Testing. IEICE TRANSACTIONS on Information and Systems 107, 5 (2024), 650–658.
[51]
Ayano Okoso, Keisuke Otaki, Satoshi Koide, and Yukino Baba. 2024. Toward Tone-Aware Explanations in Recommender Systems. In UMAP. ACM, 261–266.
[52]
Grant Packard and Jonah Berger. 2021. How concrete language shapes customer satisfaction. Journal of Consumer Research 47, 5 (2021), 787–806.
[53]
Andrea Papenmeier, Dagmar Kern, Gwenn Englebienne, and Christin Seifert. 2022. It’s complicated: The relationship between user trust, model accuracy and explanations in AI. ACM Transactions on Computer-Human Interaction (TOCHI) 29, 4(2022), 1–33.
[54]
Haekyu Park, Hyunsik Jeon, Junghwan Kim, Beunguk Ahn, and U Kang. 2017. Uniwalk: Explainable and accurate recommendation for rating and network data. arXiv preprint arXiv:1710.07134(2017).
[55]
Lara Quijano-Sanchez, Christian Sauer, Juan A. Recio-Garcia, and Belen Diaz-Agudo. 2017. Make it personal: A social explanation system applied to group recommendations. Expert Systems with Applications 76 (2017), 36–48.
[56]
Amy Rechkemmer and Ming Yin. 2022. When confidence meets accuracy: Exploring the effects of multiple performance indicators on trust in machine learning models. In Proceedings of the 2022 chi conference on human factors in computing systems. 1–14.
[57]
Paul Resnick and Hal R Varian. 1997. Recommender systems. Commun. ACM 40, 3 (1997), 56–58.
[58]
J Ben Schafer, Joseph A Konstan, and John Riedl. 2001. E-commerce recommendation applications. Data mining and knowledge discovery 5 (2001), 115–153.
[59]
Klaus R Scherer. 2003. Vocal communication of emotion: A review of research paradigms. Speech communication 40, 1-2 (2003), 227–256.
[60]
Skipper Seabold and Josef Perktold. 2010. statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference.
[61]
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems 30 (2017).
[62]
Nava Tintarev and Judith Masthoff. 2010. Designing and evaluating explanations for recommender systems. In Recommender systems handbook. Springer, 479–510.
[63]
Nava Tintarev and Judith Masthoff. 2012. Evaluating the effectiveness of explanations for recommender systems: Methodological issues and empirical studies on the impact of personalization. User Modeling and User-Adapted Interaction 22 (2012), 399–439.
[64]
Suzanne Tolmeijer, Markus Christen, Serhiy Kandul, Markus Kneer, and Abraham Bernstein. 2022. Capable but amoral? Comparing AI and human expert collaboration in ethical decision making. In CHI. 1–17.
[65]
Thi Ngoc Trang Tran, Alexander Felfernig, Viet Man Le, Thi Minh Ngoc Chau, and Thu Giang Mai. 2023. User Needs for Explanations of Recommendations: In-depth Analyses of the Role of Item Domain and Personal Characteristics. In UMAP. ACM, 54–65.
[66]
Andreu Vall, Matthias Dorfer, Hamid Eghbal-Zadeh, Markus Schedl, Keki Burjorjee, and Gerhard Widmer. 2019. Feature-combination hybrid recommender systems for automated music playlist continuation. User Modeling and User-Adapted Interaction 29 (2019), 527–572.
[67]
Jesse Vig, Shilad Sen, and John Riedl. 2009. Tagsplanations: explaining recommendations using tags. In IUI. ACM, 47–56.
[68]
Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. 2018. Explainable recommendation via multi-task learning in opinionated text data. In SIGIR. ACM, 165–174.
[69]
Weiquan Wang and Izak Benbasat. 2007. Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. Journal of Management Information Systems 23, 4 (2007), 217–246.
[70]
George Yule. 2022. The study of language. Cambridge university press.
[71]
Yongfeng Zhang, Xu Chen, et al. 2020. Explainable recommendation: A survey and new perspectives. Foundations and Trends® in Information Retrieval 14, 1(2020), 1–101.
[72]
Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In SIGIR. ACM, 83–92.
Index Terms
- Impact of Tone-Aware Explanations in Recommender Systems
Recommendations
Toward Tone-Aware Explanations in Recommender Systems
UMAP '24: Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and PersonalizationIn recommender systems, the presentation of explanations plays a crucial role in supporting users’ decision-making processes. Although numerous existing studies have focused on the effects (e.g., transparency) of explanation content, explanation ...
User Needs for Explanations of Recommendations: In-depth Analyses of the Role of Item Domain and Personal Characteristics
UMAP '23: Proceedings of the 31st ACM Conference on User Modeling, Adaptation and PersonalizationExplanations can be provided with different goals, such as clarifying how the system works, how well the recommended item meets the user’s preferences, and how an explanation helps the user select an item faster. Although extensive research has been ...
LLM-generated Explanations for Recommender Systems
UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and PersonalizationUsers are often confronted with situations where they have to decide in favor or against an offered item, like a book, movie, or recipe. Those suggested items are commonly determined by a recommender system, which considers personal preferences to ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
EISSN:2770-6699
Table of ContentsCopyright © 2025 Copyright held by the owner/author(s).
This work is licensed under Creative Commons Attribution International 4.0.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Online AM: 15 February 2025
Accepted: 10 February 2025
Revised: 07 February 2025
Received: 09 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 36Total Downloads
- Downloads (Last 12 months)36
- Downloads (Last 6 weeks)36
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in