More Web Proxy on the site http://driver.im/

research-article

Understanding the Prediction Mechanism of Sentiments by XAI Visualization

Author:

Chaehan SoAuthors Info & Claims

NLPIR '20: Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval

Pages 75 - 80

https://doi.org/10.1145/3443279.3443284

Published: 01 February 2021 Publication History

Abstract

People often rely on online reviews to make purchase decisions. The present work aimed to gain an understanding of a machine learning model's prediction mechanism by visualizing the effect of sentiments extracted from online hotel reviews with explainable AI (XAI) methodology. Study 1 used the extracted sentiments as features to predict the review ratings by five machine learning algorithms (knn, CART decision trees, support vector machines, random forests, gradient boosting machines) and identified random forests as best algorithm. Study 2 analyzed the random forests model by feature importance and revealed the sentiments joy, disgust, positive and negative as the most predictive features. Furthermore, the visualization of additive variable attributions and their prediction distribution showed correct prediction in direction and effect size for the 5-star rating but partially wrong direction and insufficient effect size for the 1-star rating. These prediction details were corroborated by a what-if analysis for the four top features. In conclusion, the prediction mechanism of a machine learning model can be uncovered by visualization of particular observations. Comparing instances of contrasting ground truth values can draw a differential picture of the prediction mechanism and inform decisions for model improvement.

References

[1]

D. Tsui, "Predicting Stock Price Movement Using Social Media Analysis," 2017.

[2]

A. Mudinas, D. Zhang, and M. Levene, "Market Trend Prediction using Sentiment Analysis: Lessons Learned and Paths Forward," Arxiv, 2019.

[3]

Y. Kim, M. Kang, and S. R. Jeong, "Text mining and sentiment analysis for predicting box office success," KSII Trans. Internet Inf. Syst., vol. 12, no. 8, pp. 4090--4102, 2018.

[4]

S. De Kok, L. Punt, R. Van Den Puttelaar, K. Ranta, K. Schouten, and F. Frasincar, "Review-level aspect-based sentiment analysis using an ontology," Proc. ACM Symp. Appl. Comput., pp. 315--322, 2018.

Digital Library

[5]

Y. Yu, "Aspect-based Sentiment Analysis on Hotel Reviews," Arxiv Prepr., p. 10, 2016.

[6]

P. Ekman, "Are there basic emotions?," 1Psychological Rev., vol. 99, no. 3, pp. 550--553, 1992.

[7]

H. Saarimäki et al., "Discrete Neural Signatures of Basic Emotions," Cereb. Cortex, vol. 26, no. 6, pp. 2563--2573, 2016.

[8]

E. Diener and R. A. Emmons, "The independence of positive and negative affect," Clin. Endocrinol. (Oxf)., vol. 41, no. 4, pp. 421--424, 1994.

[9]

D. V. Carvalho, E. M. Pereira, and J. S. Cardoso, "Machine learning interpretability: A survey on methods and metrics," Electron., vol. 8, no. 8, pp. 1--34, 2019.

[10]

C. Rudin, "Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead," Nat. Mach. Intell., vol. 1, no. 5, pp. 206--215, Nov. 2019.

[11]

S. M. Mohammad and P. D. Turney, "Crowdsourcing a word-emotion association lexicon," Comput. Intell., vol. 29, no. 3, pp. 436--465, 2013.

[12]

S. M. Mohammad and P. D. Turney, "Emotions evoked by common words and phrases: using mechanical turk to create an emotion lexicon," CAAGET '10 Proc. NAACL HLT 2010 Work. Comput. Approaches to Anal. Gener. Emot. Text, no. June, pp. 26--34, 2010.

[13]

M. Kuhn, "caret: Classification and Regression Training." 2018.

[14]

A. Karatzoglou, A. Smola, K. Hornik, and A. Zeileis, "kernlab -- An S4 Package for Kernel Methods in R," J. Stat. Softw., vol. 11, no. 9, pp. 1--20, 2004.

[15]

T. Therneau and B. Atkinson, "rpart: Recursive Partitioning and Regression Trees." 2019.

[16]

A. Liaw and M. Wiener, "Classification and Regression by randomForest," R News, vol. 2, no. 3, pp. 18--22, 2002.

[17]

B. Greenwell, B. Boehmke, J. Cunningham, and G. B. M. Developers, "gbm: Generalized Boosted Regression Models." 2019.

[18]

F. Doshi-Velez and B. Kim, "Towards A Rigorous Science of Interpretable Machine Learning," no. Ml, pp. 1--13, 2017.

[19]

P. Biecek, "DALEX: Explainers for Complex Predictive Models in R," J. Mach. Learn. Res., vol. 19, no. 84, pp. 1--5, 2018.

[20]

A. Fisher, C. Rudin, and F. Dominici, "Model Class Reliance: Variable Importance Measures for any Machine Learning Model Class, from the 'Rashomon' Perspective," J. Mach. Learn. Res., vol. 20, 2019.

Cited By

Wei LMooney C(2024)An EEG-based Automatic Classification Model for Epilepsy with Explainable Artificial IntelligenceProceedings of the 2024 14th International Conference on Biomedical Engineering and Technology10.1145/3678935.3678943(44-50)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3678935.3678943
Aparicio-Aparicio IRuescas-Nicolau MDueñas LSánchez-Jiménez JSánchez-Sánchez MAlcántara E(2024)Discovering the relationship between attributes of facial masks and review rating in online customer reviews using Explainable Artificial Intelligence (XAI)Applied Artificial Intelligence10.1080/08839514.2024.241178038:1Online publication date: 9-Oct-2024
https://doi.org/10.1080/08839514.2024.2411780
Contreras VSchumacher MCalvaresi D(2024)Explanation of Deep Learning Models via Logic Rules Enhanced by Embeddings Analysis, and Probabilistic ModelsExplainable and Transparent AI and Multi-Agent Systems10.1007/978-3-031-70074-3_9(155-183)Online publication date: 25-Sep-2024
https://doi.org/10.1007/978-3-031-70074-3_9
Show More Cited By

Index Terms

Understanding the Prediction Mechanism of Sentiments by XAI Visualization
1. Human-centered computing
  1. Visualization
    1. Visualization application domains
      1. Visual analytics

Recommendations

What Emotions Make One or Five Stars? Understanding Ratings of Online Product Reviews by Sentiment Analysis and XAI
Artificial Intelligence in HCI
Abstract
When people buy products online, they primarily base their decisions on the recommendations of others given in online reviews. The current work analyzed these online reviews by sentiment analysis and used the extracted sentiments as features to ...
Does sentiment help requirement engineering: exploring sentiments in user comments to discover informative comments
Abstract
User comments are valuable resources for software improvement; however, owing to excessive volume, informative comments need to be selectively analyzed. We attempt to address this problem by sentiment analysis and expect sentiment can be a useful ...
Jointly Discovering Fine-grained and Coarse-grained Sentiments via Topic Modeling
MM '14: Proceedings of the 22nd ACM international conference on Multimedia

The ever-increasing user-generated contents in social media and other web services make it highly desirable to discover opinions of users on all kinds of topics. Motivated by the assumption that individual word and paragraph in documents will deliver ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

NLPIR '20: Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval

December 2020

217 pages

ISBN:9781450377607

DOI:10.1145/3443279

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

FernUniversität in Hagen

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 February 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Yonsei University Faculty Research Fund

Conference

NLPIR 2020

NLPIR 2020: 4th International Conference on Natural Language Processing and Information Retrieval

December 18 - 20, 2020

Seoul, Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
340
Total Downloads

Downloads (Last 12 months)139
Downloads (Last 6 weeks)4

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wei LMooney C(2024)An EEG-based Automatic Classification Model for Epilepsy with Explainable Artificial IntelligenceProceedings of the 2024 14th International Conference on Biomedical Engineering and Technology10.1145/3678935.3678943(44-50)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3678935.3678943
Aparicio-Aparicio IRuescas-Nicolau MDueñas LSánchez-Jiménez JSánchez-Sánchez MAlcántara E(2024)Discovering the relationship between attributes of facial masks and review rating in online customer reviews using Explainable Artificial Intelligence (XAI)Applied Artificial Intelligence10.1080/08839514.2024.241178038:1Online publication date: 9-Oct-2024
https://doi.org/10.1080/08839514.2024.2411780
Contreras VSchumacher MCalvaresi D(2024)Explanation of Deep Learning Models via Logic Rules Enhanced by Embeddings Analysis, and Probabilistic ModelsExplainable and Transparent AI and Multi-Agent Systems10.1007/978-3-031-70074-3_9(155-183)Online publication date: 25-Sep-2024
https://doi.org/10.1007/978-3-031-70074-3_9
de Santana VFucs ASegura Vde Moraes DCerqueira R(2023)Predicting the need for XAI from high-granularity interaction dataInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103029175:COnline publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103029
Maleszka B(2023)A Survey of Explainable Artificial Intelligence Approaches for Sentiment AnalysisIntelligent Information and Database Systems10.1007/978-981-99-5837-5_5(52-62)Online publication date: 5-Sep-2023
https://doi.org/10.1007/978-981-99-5837-5_5
Craig PNg RTefsen BLinsen SLiu YHendel J(2022)Information Visualisation for Antibiotic Detection Biochip Design and TestingProcesses10.3390/pr1012268010:12(2680)Online publication date: 13-Dec-2022
https://doi.org/10.3390/pr10122680
Alicioglu GSun B(2022)A survey of visual analytics for Explainable Artificial Intelligence methodsComputers and Graphics10.1016/j.cag.2021.09.002102:C(502-520)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1016/j.cag.2021.09.002

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents