Machine Learning and Knowledge Extraction

Research

Jump to: Other

20 pages, 1544 KiB

Open AccessArticle

Alternative Formulations of Decision Rule Learning from Neural Networks

by Litao Qiao, Weijia Wang and Bill Lin

Mach. Learn. Knowl. Extr. 2023, 5(3), 937-956; https://doi.org/10.3390/make5030049 - 3 Aug 2023

Viewed by 1932

Abstract

This paper extends recent work on decision rule learning from neural networks for tabular data classification. We propose alternative formulations to trainable Boolean logic operators as neurons with continuous weights, including trainable NAND neurons. These alternative formulations provide uniform treatments to different trainable [...] Read more.

This paper extends recent work on decision rule learning from neural networks for tabular data classification. We propose alternative formulations to trainable Boolean logic operators as neurons with continuous weights, including trainable NAND neurons. These alternative formulations provide uniform treatments to different trainable logic neurons so that they can be uniformly trained, which enables, for example, the direct application of existing sparsity-promoting neural net training techniques like reweighted

L_{1}

regularization to derive sparse networks that translate to simpler rules. In addition, we present an alternative network architecture based on trainable NAND neurons by applying De Morgan’s law to realize a NAND-NAND network instead of an AND-OR network, both of which can be readily mapped to decision rule sets. Our experimental results show that these alternative formulations can also generate accurate decision rule sets that achieve state-of-the-art performance in terms of accuracy in tabular learning applications. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

Figure 1
(a) An example of the DR-Net architecture with 4 AND neurons is shown. The blue lines to the AND neurons represent positive weights, while red lines represent negative weights. The dashed line indicates the exclusion of the corresponding input feature. Please note that we represent “NOT (GPA ≥ 3.0)” as “GPA < 3.0” in the third rule. Similarly, “NOT (SAT ≥ 1000)” is represented as “SAT < 1000”. For the output OR neuron, the blue line indicates that the corresponding rule is included in the rule set, and the dashed line indicates that the corresponding rule is excluded. (b) The network maps directly to the corresponding decision rule set shown in the box on the right. Full article ">Figure 2
(a) A variation of the example in <a href="#make-05-00049-f001" class="html-fig">Figure 1</a>, in which the red line to the output OR neuron indicates the negation of the corresponding rule “GPA < 3.0 AND SAT < 1000.” By De Morgan’s law, the negation of “GPA < 3.0 AND SAT < 1000” becomes “GPA ≥ 3.0 OR SAT ≥ 1000”, which results in the same decision rule set. (b) The corresponding decision rule set is shown on the right. Full article ">Figure 3
Training statistics (training loss, training accuracy, number of rules, and rule complexity) as functions of the number of epochs in the training process. Full article ">Figure 4
The relations between complexities (rule complexity and number of rules) and regularization parameters (<math display="inline"><semantics><msub><mi>λ</mi><mn>1</mn></msub></semantics></math> and <math display="inline"><semantics><msub><mi>λ</mi><mn>2</mn></msub></semantics></math>). All x-axes are on a log10 scale. All complexity values were averaged over five cross-validation partitions, and the vertical bars represent standard deviations. Full article ">Figure 5
Accuracy–Complexity trade-offs on all datasets for DR-Net and NN-Net trained using <math display="inline"><semantics><msub><mi>L</mi><mn>0</mn></msub></semantics></math> regularization. Pareto efficient points are connected by line segments. Full article ">Figure 6
Accuracy–Complexity trade-offs on all datasets for DR-Net and NN-Net trained using reweighted <math display="inline"><semantics><msub><mi>L</mi><mn>1</mn></msub></semantics></math> regularization. Pareto efficient points are connected by line segments. Full article ">

15 pages, 2157 KiB

Open AccessArticle

Achievable Minimally-Contrastive Counterfactual Explanations

by Hosein Barzekar and Susan McRoy

Mach. Learn. Knowl. Extr. 2023, 5(3), 922-936; https://doi.org/10.3390/make5030048 - 3 Aug 2023

Viewed by 2165

Abstract

Decision support systems based on machine learning models should be able to help users identify opportunities and threats. Popular model-agnostic explanation models can identify factors that support various predictions, answering questions such as “What factors affect sales?” or “Why did sales decline?”, but [...] Read more.

Decision support systems based on machine learning models should be able to help users identify opportunities and threats. Popular model-agnostic explanation models can identify factors that support various predictions, answering questions such as “What factors affect sales?” or “Why did sales decline?”, but do not highlight what a person should or could do to get a more desirable outcome. Counterfactual explanation approaches address intervention, and some even consider feasibility, but none consider their suitability for real-time applications, such as question answering. Here, we address this gap by introducing a novel model-agnostic method that provides specific, feasible changes that would impact the outcomes of a complex Black Box AI model for a given instance and assess its real-world utility by measuring its real-time performance and ability to find achievable changes. The method uses the instance of concern to generate high-precision explanations and then applies a secondary method to find achievable minimally-contrastive counterfactual explanations (AMCC) while limiting the search to modifications that satisfy domain-specific constraints. Using a widely recognized dataset, we evaluated the classification task to ascertain the frequency and time required to identify successful counterfactuals. For a 90% accurate classifier, our algorithm identified AMCC explanations in 47% of cases (38 of 81), with an average discovery time of 80 ms. These findings verify the algorithm’s efficiency in swiftly producing AMCC explanations, suitable for real-time systems. The AMCC method enhances the transparency of Black Box AI models, aiding individuals in evaluating remedial strategies or assessing potential outcomes. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

21 pages, 17177 KiB

Open AccessArticle

What about the Latent Space? The Need for Latent Feature Saliency Detection in Deep Time Series Classification

by Maresa Schröder, Alireza Zamanian and Narges Ahmidi

Mach. Learn. Knowl. Extr. 2023, 5(2), 539-559; https://doi.org/10.3390/make5020032 - 18 May 2023

Cited by 3 | Viewed by 2444

Abstract

Saliency methods are designed to provide explainability for deep image processing models by assigning feature-wise importance scores and thus detecting informative regions in the input images. Recently, these methods have been widely adapted to the time series domain, aiming to identify important temporal [...] Read more.

Saliency methods are designed to provide explainability for deep image processing models by assigning feature-wise importance scores and thus detecting informative regions in the input images. Recently, these methods have been widely adapted to the time series domain, aiming to identify important temporal regions in a time series. This paper extends our former work on identifying the systematic failure of such methods in the time series domain to produce relevant results when informative patterns are based on underlying latent information rather than temporal regions. First, we both visually and quantitatively assess the quality of explanations provided by multiple state-of-the-art saliency methods, including Integrated Gradients, Deep-Lift, Kernel SHAP, and Lime using univariate simulated time series data with temporal or latent patterns. In addition, to emphasize the severity of the latent feature saliency detection problem, we also run experiments on a real-world predictive maintenance dataset with known latent patterns. We identify Integrated Gradients, Deep-Lift, and the input-cell attention mechanism as potential candidates for refinement to yield latent saliency scores. Finally, we provide recommendations on using saliency methods for time series classification and suggest a guideline for developing latent saliency methods for time series. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

18 pages, 3481 KiB

Open AccessArticle

Painting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting

by Federico Cabitza, Andrea Campagner, Chiara Natali, Enea Parimbelli, Luca Ronzio and Matteo Cameli

Mach. Learn. Knowl. Extr. 2023, 5(1), 269-286; https://doi.org/10.3390/make5010017 - 8 Mar 2023

Cited by 9 | Viewed by 3662

Abstract

The emergence of black-box, subsymbolic, and statistical AI systems has motivated a rapid increase in the interest regarding explainable AI (XAI), which encompasses both inherently explainable techniques, as well as approaches to make black-box AI systems explainable to human decision makers. Rather than [...] Read more.

The emergence of black-box, subsymbolic, and statistical AI systems has motivated a rapid increase in the interest regarding explainable AI (XAI), which encompasses both inherently explainable techniques, as well as approaches to make black-box AI systems explainable to human decision makers. Rather than always making black boxes transparent, these approaches are at risk of painting the black boxes white, thus failing to provide a level of transparency that would increase the system’s usability and comprehensibility, or even at risk of generating new errors (i.e., white-box paradox). To address these usability-related issues, in this work we focus on the cognitive dimension of users’ perception of explanations and XAI systems. We investigated these perceptions in light of their relationship with users’ characteristics (e.g., expertise) through a questionnaire-based user study involved 44 cardiology residents and specialists in an AI-supported ECG reading task. Our results point to the relevance and correlation of the dimensions of trust, perceived quality of explanations, and tendency to defer the decision process to automation (i.e., technology dominance). This contribution calls for the evaluation of AI-based support systems from a human–AI interaction-oriented perspective, laying the ground for further investigation of XAI and its effects on decision making and user experience. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

12 pages, 2247 KiB

Open AccessArticle

An Explainable Deep Learning Framework for Detecting and Localising Smoke and Fire Incidents: Evaluation of Grad-CAM++ and LIME

by Ioannis D. Apostolopoulos, Ifigeneia Athanasoula, Mpesi Tzani and Peter P. Groumpos

Mach. Learn. Knowl. Extr. 2022, 4(4), 1124-1135; https://doi.org/10.3390/make4040057 - 6 Dec 2022

Cited by 11 | Viewed by 3487

Abstract

Climate change is expected to increase fire events and activity with multiple impacts on human lives. Large grids of forest and city monitoring devices can assist in incident detection, accelerating human intervention in extinguishing fires before they get out of control. Artificial Intelligence [...] Read more.

Climate change is expected to increase fire events and activity with multiple impacts on human lives. Large grids of forest and city monitoring devices can assist in incident detection, accelerating human intervention in extinguishing fires before they get out of control. Artificial Intelligence promises to automate the detection of fire-related incidents. This study enrols 53,585 fire/smoke and normal images and benchmarks seventeen state-of-the-art Convolutional Neural Networks for distinguishing between the two classes. The Xception network proves to be superior to the rest of the CNNs, obtaining very high accuracy. Grad-CAM++ and LIME algorithms improve the post hoc explainability of Xception and verify that it is learning features found in the critical locations of the image. Both methods agree on the suggested locations, strengthening the abovementioned outcome. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

23 pages, 4081 KiB

Open AccessArticle

On the Dimensionality and Utility of Convolutional Autoencoder’s Latent Space Trained with Topology-Preserving Spectral EEG Head-Maps

by Arjun Vinayak Chikkankod and Luca Longo

Mach. Learn. Knowl. Extr. 2022, 4(4), 1042-1064; https://doi.org/10.3390/make4040053 - 18 Nov 2022

Cited by 13 | Viewed by 3126

Abstract

Electroencephalography (EEG) signals can be analyzed in the temporal, spatial, or frequency domains. Noise and artifacts during the data acquisition phase contaminate these signals adding difficulties in their analysis. Techniques such as Independent Component Analysis (ICA) require human intervention to remove noise and [...] Read more.

Electroencephalography (EEG) signals can be analyzed in the temporal, spatial, or frequency domains. Noise and artifacts during the data acquisition phase contaminate these signals adding difficulties in their analysis. Techniques such as Independent Component Analysis (ICA) require human intervention to remove noise and artifacts. Autoencoders have automatized artifact detection and removal by representing inputs in a lower dimensional latent space. However, little research is devoted to understanding the minimum dimension of such latent space that allows meaningful input reconstruction. Person-specific convolutional autoencoders are designed by manipulating the size of their latent space. A sliding window technique with overlapping is employed to segment varied-sized windows. Five topographic head-maps are formed in the frequency domain for each window. The latent space of autoencoders is assessed using the input reconstruction capacity and classification utility. Findings indicate that the minimal latent space dimension is

25 %

of the size of the topographic maps for achieving maximum reconstruction capacity and maximizing classification accuracy, which is achieved with a window length of at least 1 s and a shift of 125 ms, using the 128 Hz sampling rate. This research contributes to the body of knowledge with an architectural pipeline for eliminating redundant EEG data while preserving relevant features with deep autoencoders. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

22 pages, 923 KiB

Open AccessArticle

A Multi-Component Framework for the Analysis and Design of Explainable Artificial Intelligence

by Mi-Young Kim, Shahin Atakishiyev, Housam Khalifa Bashier Babiker, Nawshad Farruque, Randy Goebel, Osmar R. Zaïane, Mohammad-Hossein Motallebi, Juliano Rabelo, Talat Syed, Hengshuai Yao and Peter Chun

Mach. Learn. Knowl. Extr. 2021, 3(4), 900-921; https://doi.org/10.3390/make3040045 - 18 Nov 2021

Cited by 29 | Viewed by 6880

Abstract

The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success of modern machine learning methods, especially deep and reinforcement learning, have created high expectations for industrial, commercial, and social value. Second, the emerging [...] Read more.

The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success of modern machine learning methods, especially deep and reinforcement learning, have created high expectations for industrial, commercial, and social value. Second, the emerging and growing concern for creating ethical and trusted AI systems, including compliance with regulatory principles to ensure transparency and trust. These two threads have created a kind of “perfect storm” of research activity, all motivated to create and deliver any set of tools and techniques to address the XAI demand. As some surveys of current XAI suggest, there is yet to appear a principled framework that respects the literature of explainability in the history of science and which provides a basis for the development of a framework for transparent XAI. We identify four foundational components, including the requirements for (1) explicit explanation knowledge representation, (2) delivery of alternative explanations, (3) adjusting explanations based on knowledge of the explainee, and (4) exploiting the advantage of interactive explanation. With those four components in mind, we intend to provide a strategic inventory of XAI requirements, demonstrate their connection to a basic history of XAI ideas, and then synthesize those ideas into a simple framework that can guide the design of AI systems that require XAI. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

31 pages, 4782 KiB

Open AccessArticle

Explainable Artificial Intelligence for Human Decision Support System in the Medical Domain

by Samanta Knapič, Avleen Malhi, Rohit Saluja and Kary Främling

Mach. Learn. Knowl. Extr. 2021, 3(3), 740-770; https://doi.org/10.3390/make3030037 - 19 Sep 2021

Cited by 87 | Viewed by 11421

Abstract

In this paper, we present the potential of Explainable Artificial Intelligence methods for decision support in medical image analysis scenarios. Using three types of explainable methods applied to the same medical image data set, we aimed to improve the comprehensibility of the decisions [...] Read more.

In this paper, we present the potential of Explainable Artificial Intelligence methods for decision support in medical image analysis scenarios. Using three types of explainable methods applied to the same medical image data set, we aimed to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). In vivo gastral images obtained by a video capsule endoscopy (VCE) were the subject of visual explanations, with the goal of increasing health professionals’ trust in black-box predictions. We implemented two post hoc interpretable machine learning methods, called Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), and an alternative explanation approach, the Contextual Importance and Utility (CIU) method. The produced explanations were assessed by human evaluation. We conducted three user studies based on explanations provided by LIME, SHAP and CIU. Users from different non-medical backgrounds carried out a series of tests in a web-based survey setting and stated their experience and understanding of the given explanations. Three user groups (n = 20, 20, 20) with three distinct forms of explanations were quantitatively analyzed. We found that, as hypothesized, the CIU-explainable method performed better than both LIME and SHAP methods in terms of improving support for human decision-making and being more transparent and thus understandable to users. Additionally, CIU outperformed LIME and SHAP by generating explanations more rapidly. Our findings suggest that there are notable differences in human decision-making between various explanation support settings. In line with that, we present three potential explainable methods that, with future improvements in implementation, can be generalized to different medical data sets and can provide effective decision support to medical experts. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

47 pages, 6520 KiB

Open AccessArticle

Classification of Explainable Artificial Intelligence Methods through Their Output Formats

by Giulia Vilone and Luca Longo

Mach. Learn. Knowl. Extr. 2021, 3(3), 615-661; https://doi.org/10.3390/make3030032 - 4 Aug 2021

Cited by 92 | Viewed by 17535

Abstract

Machine and deep learning have proven their utility to generate data-driven models with high accuracy and precision. However, their non-linear, complex structures are often difficult to interpret. Consequently, many scholars have developed a plethora of methods to explain their functioning and the logic [...] Read more.

Machine and deep learning have proven their utility to generate data-driven models with high accuracy and precision. However, their non-linear, complex structures are often difficult to interpret. Consequently, many scholars have developed a plethora of methods to explain their functioning and the logic of their inferences. This systematic review aimed to organise these methods into a hierarchical classification system that builds upon and extends existing taxonomies by adding a significant dimension—the output formats. The reviewed scientific papers were retrieved by conducting an initial search on Google Scholar with the keywords “explainable artificial intelligence”; “explainable machine learning”; and “interpretable machine learning”. A subsequent iterative search was carried out by checking the bibliography of these articles. The addition of the dimension of the explanation format makes the proposed classification system a practical tool for scholars, supporting them to select the most suitable type of explanation format for the problem at hand. Given the wide variety of challenges faced by researchers, the existing XAI methods provide several solutions to meet the requirements that differ considerably between the users, problems and application fields of artificial intelligence (AI). The task of identifying the most appropriate explanation can be daunting, thus the need for a classification system that helps with the selection of methods. This work concludes by critically identifying the limitations of the formats of explanations and by providing recommendations and possible future research directions on how to build a more generally applicable XAI method. Future work should be flexible enough to meet the many requirements posed by the widespread use of AI in several fields, and the new regulations. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

17 pages, 1553 KiB

Open AccessArticle

Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability

by Muhammad Rehman Zafar and Naimul Khan

Mach. Learn. Knowl. Extr. 2021, 3(3), 525-541; https://doi.org/10.3390/make3030027 - 30 Jun 2021

Cited by 137 | Viewed by 12444

Abstract

Local Interpretable Model-Agnostic Explanations (LIME) is a popular technique used to increase the interpretability and explainability of black box Machine Learning (ML) algorithms. LIME typically creates an explanation for a single prediction by any ML model by learning a simpler interpretable model (e.g., [...] Read more.

Local Interpretable Model-Agnostic Explanations (LIME) is a popular technique used to increase the interpretability and explainability of black box Machine Learning (ML) algorithms. LIME typically creates an explanation for a single prediction by any ML model by learning a simpler interpretable model (e.g., linear classifier) around the prediction through generating simulated data around the instance by random perturbation, and obtaining feature importance through applying some form of feature selection. While LIME and similar local algorithms have gained popularity due to their simplicity, the random perturbation methods result in shifts in data and instability in the generated explanations, where for the same prediction, different explanations can be generated. These are critical issues that can prevent deployment of LIME in sensitive domains. We propose a deterministic version of LIME. Instead of random perturbation, we utilize Agglomerative Hierarchical Clustering (AHC) to group the training data together and K-Nearest Neighbour (KNN) to select the relevant cluster of the new instance that is being explained. After finding the relevant cluster, a simple model (i.e., linear model or decision tree) is trained over the selected cluster to generate the explanations. Experimental results on six public (three binary and three multi-class) and six synthetic datasets show the superiority for Deterministic Local Interpretable Model-Agnostic Explanations (DLIME), where we quantitatively determine the stability and faithfulness of DLIME compared to LIME. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

Figure 1
A block diagram of the LIME framework. Full article ">Figure 2
A high level block diagram of the DLIME framework. Full article ">Figure 3
Dendrograms of binary and multi-class datasets. Full article ">Figure 4
Explanations for neural network generated by DLIME (Linear) and LIME, and respective Jaccard distances over 10 iterations. Highlighted features with yellow color in (b,d) represents the difference in selected features for the same instance over 2 iterations. The order of features in (a–d) is higher to lower importance. (e,f) shows the Jaccard distance matrix among the features selected over 10 iterations. Full article ">Figure 5
Explanations generated for neural network by DLIME-Tree and LIME, and respective Jaccard distances over 10 iterations. Features outlined with red color in (a,c) represents insignificant features with 0 contribution. Highlighted features with yellow color in (b,d) represents the difference in selected features for the same instance over 2 iterations. The order of features in (a–d) is higher to lower importance. (e,f) shows the Jaccard distance matrix among the features selected over 10 iterations. Full article ">

Other

Jump to: Research

31 pages, 1054 KiB

Open AccessSystematic Review

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

by Tobias Clement, Nils Kemmerzell, Mohamed Abdelaal and Michael Amberg

Mach. Learn. Knowl. Extr. 2023, 5(1), 78-108; https://doi.org/10.3390/make5010006 - 11 Jan 2023

Cited by 51 | Viewed by 21014

Abstract

Currently, explainability represents a major barrier that Artificial Intelligence (AI) is facing in regard to its practical implementation in various application domains. To combat the lack of understanding of AI-based systems, Explainable AI (XAI) aims to make black-box AI models more transparent and [...] Read more.

Currently, explainability represents a major barrier that Artificial Intelligence (AI) is facing in regard to its practical implementation in various application domains. To combat the lack of understanding of AI-based systems, Explainable AI (XAI) aims to make black-box AI models more transparent and comprehensible for humans. Fortunately, plenty of XAI methods have been introduced to tackle the explainability problem from different perspectives. However, due to the vast search space, it is challenging for ML practitioners and data scientists to start with the development of XAI software and to optimally select the most suitable XAI methods. To tackle this challenge, we introduce XAIR, a novel systematic metareview of the most promising XAI methods and tools. XAIR differentiates itself from existing reviews by aligning its results to the five steps of the software development process, including requirement analysis, design, implementation, evaluation, and deployment. Through this mapping, we aim to create a better understanding of the individual steps of developing XAI software and to foster the creation of real-world AI applications that incorporate explainability. Finally, we conclude with highlighting new directions for future research. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

Journal Menu

Journal Browser

Advances in Explainable Artificial Intelligence (XAI)

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Related Special Issue

Published Papers (11 papers)

Research

Other

Further Information

Guidelines

MDPI Initiatives

Follow MDPI