Abstract
The Rashomon Effect describes the following phenomenon: for a given dataset there may exist many models with equally good performance but with different solution strategies. The Rashomon Effect has implications for Explainable Machine Learning, especially for the comparability of explanations. We provide a unified view on three different comparison scenarios and conduct a quantitative evaluation across different datasets, models, attribution methods, and metrics. We find that hyperparameter-tuning plays a role and that metric selection matters. Our results provide empirical support for previously anecdotal evidence and exhibit challenges for both scientists and practitioners.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Our code is available at github.com/lamarr-xai-group/RashomonEffect.
- 2.
See project page at github.com/pytorch/captum.
References
Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)
Alkhatib, A., Boström, H., Vazirgiannis, M.: Explaining predictions by characteristic rules. In: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD) (2022)
Alvarez-Melis, D., Jaakkola, T.S.: On the robustness of interpretability methods. In: Workshop on Human Interpretability in Machine Learning (WHI@ICML) (2018)
Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for deep neural networks. In: International Conference on Learning Representations, (ICLR) (2018)
Atanasova, P., Simonsen, J.G., Lioma, C., Augenstein, I.: A diagnostic study of explainability techniques for text classification. In: Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020)
Beckh, K., et al.: Harnessing prior knowledge for explainable machine learning: an overview. In: 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 450–463 (2023). https://doi.org/10.1109/SaTML54575.2023.00038
Bogun, A., Kostadinov, D., Borth, D.: Saliency diversified deep ensemble for robustness to adversaries. In: AAAI-22 Workshop on Adversarial Machine Learning and Beyond (2021)
Breiman, L.: Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat. Sci. 16(3), 199–231 (2001)
Burkart, N., Huber, M.F.: A survey on the explainability of supervised machine learning. J. Artif. Intell. Res. 70, 245–317 (2021)
DeYoung, J., et al.: ERASER: a benchmark to evaluate rationalized NLP models. In: Annual Meeting of the Association for Computational Linguistics (ACL) (2020)
ElShawi, R., Sherif, Y., Al-Mallah, M., Sakr, S.: Interpretability in healthcare: a comparative study of local machine learning interpretability techniques. Comput. Intell. 37(4), 1633–1650 (2021)
Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res. 20(177), 1–81 (2019)
Flora, M., Potvin, C., McGovern, A., Handler, S.: Comparing explanation methods for traditional machine learning models part 1: an overview of current methods and quantifying their disagreement. arXiv preprint arXiv:2211.08943 (2022)
Guidotti, R., Ruggieri, S.: Assessing the stability of interpretable models. arXiv preprint arXiv:1810.09352 (2018)
Han, T., Srinivas, S., Lakkaraju, H.: Which explanation should i choose? A function approximation perspective to characterizing post hoc explanations. In: Advances in Neural Information Processing Systems (NeurIPS) (2022)
Hancox-Li, L.: Robustness in machine learning explanations: does it matter? In: Conference on Fairness, Accountability, and Transparency (FAT*) (2020)
Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for interpretability methods in deep neural networks. In: Advances in Neural Information Processing Systems (NeurIPS) (2019)
Koklu, M., Özkan, I.A.: Multiclass classification of dry beans using computer vision and machine learning techniques. Comput. Electron. Agric. 174, 105507 (2020)
Krishna, S., et al.: The disagreement problem in explainable machine learning: a practitioner’s perspective. arXiv preprint arXiv:2202.01602 (2022)
Leventi-Peetz, A.M., Weber, K.: Rashomon effect and consistency in explainable artificial intelligence (XAI). In: Future Technologies Conference (FTC) (2022)
Liu, F., Avci, B.: Incorporating priors with feature attribution on text classification. In: Annual Meeting of the Association for Computational Linguistics (ACL) (2019)
Lundberg, S., Lee, S.I.: A Unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems (NeurIPS) (2017)
Marx, C.T., Calmon, F.P., Ustun, B.: Predictive multiplicity in classification. In: International Conference on Machine Learning (ICML) (2020)
Molnar, C.: Interpretable Machine Learning. 2nd edn. (2022)
Mücke, S., Pfahler, L.: Check Mate: a sanity check for trustworthy AI. In: Lernen. Wissen. Daten. Analysen. (LWDA) (2022)
Neely, M., Schouten, S.F., Bleeker, M.J., Lucic, A.: order in the court: explainable AI methods prone to disagreement. arXiv preprint arXiv:2105.03287 (2021)
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: explaining the predictions of any classifier. In: International Conference on Knowledge Discovery and Data Mining (KDD) (2016)
Roscher, R., Bohn, B., Duarte, M.F., Garcke, J.: Explainable machine learning for scientific insights and discoveries. IEEE Access 8, 42200–42216 (2020)
Schramowski, P., et al.: Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nat. Mach. Intell. 2(8), 476–486 (2020)
Semenova, L., Rudin, C., Parr, R.: On the existence of simpler machine learning models. In: Conference on Fairness, Accountability, and Transparency (FAccT) (2022)
Sigillito, V.G., Wing, S.P., Hutton, L.V., Baker, K.B.: Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Tech. Digest 10(3), 262–266 (1989)
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: International Conference on Learning Representations (ICLR) (2014)
Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International Conference on Machine Learning (ICML) (2017)
Watson, M., Hasan, B.A.S., Al Moubayed, N.: Agree to disagree: when deep learning models with identical architectures produce distinct explanations. In: Winter Conference on Applications of Computer Vision (WACV) (2022)
Wolberg, W., Street, N., Mangasarian, O.: Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository (1995)
Xin, R., Zhong, C., Chen, Z., Takagi, T., Seltzer, M., Rudin, C.: Exploring the whole rashomon set of sparse decision trees. In: Advances in Neural Information Processing Systems (NeurIPS) (2022)
Yeh, C., Hsieh, C., Suggala, A.S., Inouye, D.I., Ravikumar, P.: On the (In)fidelity and sensitivity of explanations. In: Advances in Neural Information Processing Systems (NeurIPS) (2019)
Zednik, C., Boelsen, H.: Scientific exploration and explainable artificial intelligence. Minds Mach. 32(1), 219–239 (2022)
Zhang, X., Zhao, J.J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems (NeurIPS) (2015)
Acknowledgments
This research has been funded by the Federal Ministry of Education and Research of Germany and the state of North-Rhine Westphalia as part of the Lamarr-Institute for Machine Learning and Artificial Intelligence Lamarr22B. Part of PWs work has been funded by the Vienna Science and Technology Fund (WWTF) project ICT22-059.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethical Statement
In critical contexts, where persons are directly or indirectly impacted by a model, and where explanations are used to verify that model behavior is compliant with a given standard, proper use of explanation methods is of utmost importance. Hyperparameter choices have to be validated for each model individually. For model testing and validation procedures to be reliable they have to integrate this knowledge. Our work demonstrated that it is unreasonable to expect an explanation computed for one model, to be valid for another model, however similar their performance otherwise may be. Re-using explanations from one model to give as an explanation of behavior for another model is not possible and has to be avoided in critical scenarios.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Müller, S., Toborek, V., Beckh, K., Jakobs, M., Bauckhage, C., Welke, P. (2023). An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14171. Springer, Cham. https://doi.org/10.1007/978-3-031-43418-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-43418-1_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43417-4
Online ISBN: 978-3-031-43418-1
eBook Packages: Computer ScienceComputer Science (R0)