Proposes a method for automatic feature identification and visual explanation of deep models.
problem Improving interpretability and explanation of deep neural networks.
method Automatic feature identification and visualization of relevant features for classes without additional annotations.
result Produces detailed explanations with good coverage of relevant features.
Study finds visual explanations do not significantly improve human accuracy or trust in model predictions.
problem Measuring the impact of visual explanations on human accuracy and trust in model predictions.
method Randomized controlled trial with image-based age prediction task, varying levels of explanation quality.
result Visual explanations do not significantly alter human accuracy or trust in the model.
New method evaluates visual explanations of deep models using adversarial perturbations.
problem Lack of objective evaluation of visual explanations of deep models.
method Proposes an adversarial perturbation approach to evaluate visual explanations of deep models.
result Demonstrates the effectiveness of the proposed approach through comparisons with existing methods.
New method to evaluate visual explanations from neural networks.
problem Lack of consensus on measuring effectiveness of visual explanations.
method Proposed a new procedure for evaluating explanations using a range of sources.
result Demonstrated the benefit of combining different sources and the impact of bias parameters.
Develops counterfactual visual explanations to show how images could change to classify differently.
problem Creating understandable explanations for vision system predictions.
method Selects a distractor image and identifies spatial regions to modify for different classification.
result Users trained with counterfactual explanations perform better in fine-grained bird classification.
VINE visualizes statistical interactions in complex models.
problem Lack of utilities for regional explanations in black box models.
method VINE algorithm to extract and visualize statistical interaction effects.
result VINE provides a novel evaluation metric for visualizations.
Grad-CAM visualizes CNN predictions to explain model decisions.
problem Making CNN models transparent and understandable.
method Using class-specific gradient information to localize important regions.
result Improved understanding of CNN-based models, including image captioning and VQA.
Develops visual explanations for Alzheimer's disease classification using 3D-CNNs.
problem Improving understanding of Alzheimer's disease classification using 3D-CNNs.
method Three approaches: sensitivity analysis and two activation visualization methods.
result Visual explanations identify important brain parts for Alzheimer's disease diagnosis.
DECE visualizes machine learning decisions with counterfactual explanations.
problem Making machine learning models transparent and explainable.
method Interactive visualization system supporting counterfactual explanations at instance- and subgroup-levels.
result DECE enables users to explore and understand machine learning model decisions.
Enhances visual explanations with logical rules for complex concepts.
problem Lack of explanatory power for deep learning models, especially for relational concepts.
method Combining LIME for highlighting and Aleph for logic rules.
result Generated relational rules can be explicitly linked to input images.
Framework for understanding deep learning models through visual explanations.
problem Challenges of explainability and debuggability in deep learning.
method Develops a preliminary framework called 'deep visual explanation' (DVE) for understanding deep neural network models.
result Initial results show potential for interpretability in image classification.
Teaches categories with visual explanations to improve learning.
problem Challenges of traditional machine teaching methods in providing clear explanations.
method Proposes a teaching framework that provides interpretable explanations as feedback.
result Participants achieve better test set performance with interpretable explanations.
Method generates visual explanations for similarity models without classification.
problem Lack of visual explanations for similarity models trained without classification loss.
method Gradient-based visual attention using learned feature embeddings.
result Attention maps improve model performance and can be used as constraints.
We create an interpretable credit risk model with transparent explanations.
problem Providing an explainable model for credit risk assessment.
method Two-layer additive risk model with globally consistent explanations.
result The model is as accurate as other neural networks and provides transparent explanations.
New method explains visual models by altering features causally.
problem Inability of pure observational data to compute reliable feature effects.
method Causal Counterfactuals and intervened causal models.
result Computes counterfactuals to show model reactions to feature changes.
Develops ACE to automatically identify meaningful concepts from neural network predictions.
problem Challenges in interpreting feature importance scores for machine learning models.
method Proposes concept-based explanation principles and develops ACE algorithm to extract visual concepts.
result Demonstrates ACE discovers human-meaningful, coherent concepts for neural network predictions.
DEMUD-VIS detects novel image content and explains it visually.
problem Detecting and explaining novel image content in large datasets.
method Uses CNN for feature extraction, reconstruction error for novelty detection, and up-convolutional networks for image reconstruction.
result Demonstrates visual explanations of novel image content on diverse datasets.
Paper presents a Rashomon Quartet to show different model explanations.
problem Models with similar performance can explain data differently.
method Synthetic dataset with four equally effective models.
result Models with similar performance can explain data differently.
Workflow aids experts in understanding binary classifier decisions.
problem Lack of transparency in machine learning model decisions.
method Instance-level explanations and visual representations.
result Workflow helps experts derive useful knowledge and hypotheses.
Manipulated explanations can be made imperceptible to the naked eye.
problem The trustworthiness and interpretability of neural networks can be compromised by manipulable explanations.
method The paper demonstrates that explanations can be altered by applying small, imperceptible input changes that do not affect the network's output.
result An upper bound on the susceptibility of explanations to manipulation has been derived, and effective mechanisms to enhance explanation robustness have been proposed.
Local explanation methods for deep networks are found to be insensitive to parameter values.
problem Local explanation methods lack sensitivity to parameter values in deep neural networks.
method Investigated the sensitivity of local explanation methods (e.g., integrated gradients) to parameter values in randomly-initialized and learned networks.
result IG attributions for a random network and the actual network are uncorrelated when both factors (signs and baseline pixels) are accounted for.
This paper improves neural network explanations by quantifying and visualizing semantic compositions.
problem Improving neural network explanations for natural language processing tasks.
method Proposes a formal way to quantify word and phrase importance, introduces SCD and SOC algorithms.
result Our algorithms outperform prior methods in explaining neural network predictions.
A CBR approach helps fraud analysts trust machine learning predictions.
problem Understanding the trustworthiness of machine learning predictions for fraud analysts.
method Case-based reasoning (CBR) approach to visualize similar previous instances and their local post-hoc explanations.
result Empirically, the visualization of similar previous instances is useful and easy to use for fraud analysts.
Visualizes deep network feature contributions in images.
problem Understanding information flow in deep networks.
method Forward-Backward approach for feature visualization.
result Numerical results show benefits over existing methods.
Gradient-based methods improve understanding of deep learning survival models.
problem Limited interpretability of deep learning survival models hinders their adoption.
method Gradient-based explanation methods tailored to survival neural networks.
result Gradient-based methods capture feature effects and temporal dynamics.
Extract class-specific subnetworks from neural models for better understanding and improved explanations.
problem Understanding and explaining the complex behavior of deep neural networks.
method For each semantic class, extract a class-specific subnetwork with a compressed structure that maintains comparable performance.
result Extracted subnetworks improve explanation saliency and adversarial example detection.
The paper reviews visual analytics for making deep learning models more interpretable.
problem Lack of explanation and control over deep learning models in critical applications.
method Review of visual analytics, information visualization, and machine learning perspectives.
result Discussion of challenges and future research directions in making deep learning models more interpretable.
Paper bridges generative models and explainability.
problem Lack of connection between generative models and explainability.
method Proposes a probabilistic framework for example-based explanations.
result Formally defines example-based explanations for deep generative models.
Extends local attributions to Bayesian Neural Networks for improved explanations.
problem Lack of explanations for Bayesian Neural Networks' predictions.
method Extend local attributions to a probabilistic explanation distribution of BNNs.
result Enriches standard explanations with uncertainty information and visualizes explanation stability.
Local explanations of DNNs are insensitive to parameter values.
problem Sensitivity of local explanations to DNN parameter values.
method Assessed sensitivity of local explanations to DNN parameter values.
result DNNs with randomly-initialized weights produce similar explanations to those with learned weights.
This chapter bridges the gap between expert and lay users in explainable AI.
problem Lack of user-friendly explanations in deep learning models.
method Analysis of user concerns, taxonomy of explanation methods, and evaluation of adequacy.
result Explanation methods are inadequate for lay users and address bias and unfair outcomes poorly.
InterpNET generates natural language explanations for deep learning classifications.
problem Deep learning models lack explainability.
method Inspired by human visual system, InterpNET designs interpretable neural networks.
result InterpNET achieves a high METEOR score of 37.9 for generating explanations.
TaylorPODA uses Taylor expansions to improve feature attributions for opaque models.
problem Lack of systematic framework for quantifying feature contributions in opaque models.
method Taylor expansion framework with postulates (precision, federation, zero-discrepancy, adaptation).
result TaylorPODA achieves competitive results and provides principled explanations.
SmoothGrad improves visual clarity of deep network sensitivity maps.
problem Visualizing deep network decision-making processes.
method Introducing SmoothGrad, a method to enhance gradient-based sensitivity maps.
result SmoothGrad helps in creating clearer, more interpretable sensitivity maps.
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
Paper presents adversarial method for explaining IDS misclassifications.
problem Limited interpretability of machine learning models in IDSs.
method Adversarial machine learning to generate feature modifications.
result Satisfactory explanations match expert knowledge.
Systematically compares methods for explaining RNN predictions.
problem Understanding the decisions of RNNs, particularly LSTMs, through relevance assignments.
method Systematic comparison of methods in various settings.
result Best method reveals linguistic phenomena in sentiment analysis.
StylEx trains a GAN to explain classifier decisions in StyleSpace.
problem Creating meaningful image-specific explanations for classifier decisions.
method Training a StyleGAN to learn a classifier-specific StyleSpace, incorporating the classifier model.
result StylEx finds attributes that align with semantic ones and generates human-interpretable explanations.
Method assesses neural network feature importance based on sensitivity.
problem Neural networks lack interpretability, especially in regulated industries.
method Sensitivity-based feature importance assessment.
result Method provides fast, global, and local explanations for various neural network architectures.
Bayesian framework for solar magnetogram super-resolution with uncertainty quantification.
problem Uncertainty in super-resolving solar magnetic field images.
method Bayesian decomposition of uncertainties into epistemic and aleatoric.
result Generation of maps measuring the range of possible high-resolution explanations.
GNNExplainer provides interpretable explanations for GNN predictions.
problem Explaining GNN predictions remains unsolved due to complex model structure.
method Formulates GNNExplainer as an optimization task maximizing mutual information between prediction and subgraph structures.
result Identifies crucial subgraph structures and node features for GNN predictions.
The paper evaluates saliency methods for image predictions, finding some inadequate.
problem The reliability of saliency methods in explaining model predictions.
method Proposed a methodology to evaluate saliency methods based on their independence from model and data.
result Some saliency methods are independent of both model and data, making them inadequate for certain tasks.
Archipelago provides interpretable explanations of feature interactions in machine learning models.
problem Interpreting the impact of feature interactions on predictions in machine learning models.
method Archipelago is a novel framework for extracting and attributing feature interactions in a scalable and interpretable manner.
result Archipelago provides significantly more interpretable explanations of feature interactions than comparable methods.
OptiLIME improves LIME explanations by balancing stability and adherence.
problem LIME's instability and lack of reliability in explanations.
method OptiLIME uses a deterministic sampling approach and feature selection to maximize stability while retaining predefined adherence.
result OptiLIME provides more reliable and interpretable explanations.
Study proposes a novel local explanation method for deep learning classifiers in process mining.
problem Lack of interpretability in deep learning models for process mining.
method Defines local regions using latent space representations and visualizes explanations.
result Deep learning classifier achieves high performance and local explanations increase user trust.
People can learn complex visual concepts from just a few examples.
problem Understanding how people learn and categorize visual concepts from limited data.
method Bayesian program learning model that searches for the best explanation of observations.
result People's judgments are broadly consistent with a Bayesian program learning model, indicating they can learn rich algorithmic abstractions from sparse input data.
GADGET framework decomposes global feature effects using recursive partitioning.
problem Misleading global feature effects when feature interactions are present.
method Generalized additive decomposition of global effects (GADGET) based on recursive partitioning.
result Minimizes interaction-related heterogeneity of local feature effects.
Manifold visualizes machine learning model outcomes without accessing their internal logic.
problem Lack of generic frameworks for interpreting and diagnosing different machine learning models.
method Generic framework that observes inputs and outputs, not model's internal logic.
result Supports interpretation, debugging, and comparison of machine learning models in a transparent and interactive manner.