More Web Proxy on the site http://driver.im/

research-article

Interactive Machine Learning and Explainability in Mobile Classification of Forest-Aesthetics

Authors:

Andreas Seiderer,

Katharina Weitz,

Ruben Schlagowski,

Elisabeth André,

Joachim RathmannAuthors Info & Claims

GoodTechs '20: Proceedings of the 6th EAI International Conference on Smart Objects and Technologies for Social Good

Pages 90 - 95

https://doi.org/10.1145/3411170.3411225

Published: 14 September 2020 Publication History

Abstract

This paper presents an application that classifies forest's aesthetics using interactive machine learning on mobile devices. Transfer learning is used to be able to build upon deep ANNs (MobileNet) using the limited resources available on smart-phones. We trained and evaluated a model using our application based on a data-set that is plausible to be created by a single user. In order to increase the comprehensibility of our model we explore the potential of incorporating explainable Artificial Intelligence (XAI) into our mobile application. To this end we use deep Taylor decomposition to generate saliency maps that highlight areas of the input that were relevant for the decision of the ANN and conducted a user study to evaluate the usefulness of this approach for end-users.

References

[1]

R.P. Abello and F.G. Bernaldez. 1986. Landscape preference and personality. Landscape and Urban Planning 13 (1986), 19 - 28.

[2]

Amina Adadi and Mohammed Berrada. 2018. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (2018), 52138--52160. https://doi.org/10.1109/ACCESS.2018.2870052

[3]

Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T Schütt, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, and Pieter-Jan Kindermans. 2018. iNNvestigate neural networks! arXiv preprint arXiv:1808.04260 (2018).

[4]

Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoe Juozapaitis, Evan Newman, Jed Irvine, Souti Chattopadhyay, Alan Fern, and Margaret Burnett. 2019. Explaining Reinforcement Learning to Mere Mortals: An Empirical Study. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 1328--1334. https://doi.org/10.24963/ijcai.2019/184

[5]

Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLOS ONE 10, 7 (July 2015). https://doi.org/10.1371/journal.pone.0130140

[6]

Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one 10, 7 (2015), e0130140.

[7]

Simon Bell. 2001. Landscape pattern, perception and visualisation in the visual management of forests. Landscape and Urban planning 54, 1--4 (2001), 201--211.

[8]

Rafael A Calvo and Dorian Peters. 2014. Positive computing: technology for wellbeing and human potential. MIT Press.

[9]

Yuelu CHEN, Tianzhong ZHAO, Gang WU, and Feixiang CHEN. 2018. Optimal Walking Path Analysis Method for Forest Region. Transactions of the Chinese Society for Agricultural Machinery 6 (2018), 23.

[10]

Terry C Daniel and Ron S Boster. 1976. Measuring landscape esthetics: the scenic beauty estimation method. Res. Pap. RM-RP-167. US Department of Agriculture, Forest Service, Rocky Mountain Range and Experiment Station. 66 p. 167 (1976).

[11]

Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2013. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. CoRR abs/1310.1531 (2013). arXiv:1310.1531

[12]

Jerry Alan Fails and Dan R Olsen Jr. 2003. Interactive machine learning. In Proceedings of the 8th international conference on Intelligent user interfaces. ACM, 39--45.

Digital Library

[13]

Simon Flutura, Andreas Seiderer, Ilhan Aslan, Chi-Tai Dang, Raphael Schwarz, Dominik Schiller, and Elisabeth André. 2018. DrinkWatch: A Mobile Wellbeing Application Based on Interactive and Cooperative Machine Learning. In Proceedings of the 2018 International Conference on Digital Health (DH '18). Association for Computing Machinery, New York, NY, USA, 65--74.

Digital Library

[14]

Simon Flutura, Andreas Seiderer, Ilhan Aslan, Michael Dietz, Dominik Schiller, Christoph Beck, Joachim Rathmann, and Elisabeth André. 2019. Mobile Sensing for Wellbeing Estimation of Urban Green Using Physiological Signals. In Proceedings of the 5th EAI International Conference on Smart Objects and Technologies for Social Good (GoodTechs '19). Association for Computing Machinery, New York, NY, USA, 249--254.

Digital Library

[15]

Francesco Fusco, Michalis Vlachos, Vasileios Vasileiadis, Kathrin Wardatzky, and Johannes Schneider. 2019. RecoNet: An Interpretable Neural Architecture for Recommender Systems. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 2343--2349.

[16]

Ali Jahani. 2019. Forest landscape aesthetic quality model (FLAQM): A comparative study on landscape modelling using regression analysis and artificial neural networks. Journal of Forest Science 65 (03 2019), 61--69.

[17]

Rachel Kaplan. 2001. The nature of the view from home: Psychological benefits. Environment and behavior 33, 4 (2001), 507--542.

[18]

Stephen Kaplan. 1995. The restorative benefits of nature: Toward an integrative framework. Journal of Environmental Psychology 15, 3 (1995), 169 - 182. Green Psychology.

[19]

Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th international conference on intelligent user interfaces. ACM, 126--137.

Digital Library

[20]

Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more?: the effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1--10.

Digital Library

[21]

Andrew Lothian. 1999. Landscape and the philosophy of aesthetics: is landscape quality inherent in the landscape or in the eye of the beholder? Landscape and urban planning 44, 4 (1999), 177--198.

[22]

Satomi Manzaki, Ayame Kano, Narihiro Haneda, Chihiro Sato, and Naohito Okude. 2018. Country Road Finder: Exploring Beauty when Driving Around. In Proceedings of the 2018 ACM Conference Companion Publication on Designing Interactive Systems. ACM, New York, NY, USA, 21--25.

Digital Library

[23]

Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, and Klaus-Robert Müller. 2017. Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognition 65 (2017), 211--222.

Digital Library

[24]

Bum Jin Park, Yuko Tsunetsugu, Tamami Kasetani, Takahide Kagawa, and Yoshifumi Miyazaki. 2010. The physiological effects of Shinrin-yoku (taking in the forest atmosphere or forest bathing): evidence from field experiments in 24 forests across Japan. Environmental health and preventive medicine 15, 1 (2010), 18.

[25]

Russ Parsons and Terry C Daniel. 2002. Good looking: in defense of scenic landscape aesthetics. Landscape and Urban Planning 60, 1 (2002), 43 - 56.

[26]

Yuen Peng Loh, Song Tong, Xuefeng Liang, Takatsune Kumada, and Chee Seng Chan. 2017. Understanding scenery quality: A visual attention measure and its computational model. In Proceedings of the IEEE International Conference on Computer Vision. 289--297.

[27]

Maaret Posti, Johannes Schöming, and Jonna Häkkilä. 2014. Unexpected Journeys with the HOBBIT - The Design and Evaluation of an Asocial Hiking App. In Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS.

[28]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1135--1144.

Digital Library

[29]

Pavel Samsonov, Florian Heller, and Johannes Schöning. 2017. Autobus: Selection of Passenger Seats Based on Viewing Experience for Touristic Tours. In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia. ACM, 321--326.

Digital Library

[30]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]

Andreas Seiderer, Michael Dietz, Ilhan Aslan, and Elisabeth André. 2018. Enabling Privacy with Transfer Learning for Image Classification DNNs on Mobile Devices. In Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good (Goodtechs '18). ACM, New York, NY, USA, 25--30.

Digital Library

[32]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. CoRR abs/1312.6034(2013). arXiv:1312.6034

[33]

J. G. Taylor, K.J. Czarnowski, N. R. Sexton, and S. Flick. 1995. The importance of water to Rocky Mountain National Park visitors: an adaptation of visitor-employed photography to natural resources management. Journal of Applied Recreation Research 20, 1 (1995), 61--85.

Cited By

Al-Ansari NAl-Thani DAl-Mansoori R(2024)User‐Centered Evaluation of Explainable Artificial Intelligence (XAI): A Systematic Literature ReviewHuman Behavior and Emerging Technologies10.1155/2024/46288552024:1Online publication date: 15-Jul-2024
https://doi.org/10.1155/2024/4628855

Index Terms

Interactive Machine Learning and Explainability in Mobile Classification of Forest-Aesthetics
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing
    2. Ubiquitous and mobile computing systems and tools
2. Information systems
  1. World Wide Web
    1. Web searching and information discovery
      1. Personalization

Recommendations

Artificial Intelligence for Humankind: A Panel on How to Create Truly Interactive and Human-Centered AI for the Benefit of Individuals and Society
Human-Computer Interaction – INTERACT 2021
Abstract
This panel discusses the role of human-computer interaction (HCI) in the conception, design, and implementation of human-centered artificial intelligence (AI). For us, it is important that AI and machine learning (ML) are ethical and create value ...
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
CSCW

The wide adoption of Machine Learning (ML) technologies has created a growing demand for people who can train ML models. Some advocated the term "machine teacher'' to refer to the role of people who inject domain knowledge into ML models. This "teaching''...
Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

With Artificial Intelligence (AI) becoming ubiquitous in every application domain, the need for explanations is paramount to enhance transparency and trust among non-technical users. Despite the potential shown by Explainable AI (XAI) for enhancing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

GoodTechs '20: Proceedings of the 6th EAI International Conference on Smart Objects and Technologies for Social Good

September 2020

286 pages

ISBN:9781450375597

DOI:10.1145/3411170

General Chairs:
Catia Prandi
University of Bologna, Italy
,
Johann Marquez-Barja
University of Antwerp - imec, Belgium

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

EAI: The European Alliance for Innovation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

GoodTechs '20

GoodTechs '20: 6th EAI International Conference on Smart Objects and Technologies for Social Good

September 14 - 16, 2020

Antwerp, Belgium

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
147
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Al-Ansari NAl-Thani DAl-Mansoori R(2024)User‐Centered Evaluation of Explainable Artificial Intelligence (XAI): A Systematic Literature ReviewHuman Behavior and Emerging Technologies10.1155/2024/46288552024:1Online publication date: 15-Jul-2024
https://doi.org/10.1155/2024/4628855

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents