Paper proposes metrics to evaluate AI explanations without ground truth.
problem Challenges in evaluating neural network explanations without ground truth.
method Designs four metrics to evaluate explanation results.
result New insights into neural network interpretation methods.
Combines neural network explanation methods to improve robustness and accuracy.
problem Lack of consensus on explaining neural network decisions and evaluating explanations.
method Investigates schemes to aggregate explanation methods and reduce model uncertainty.
result Aggregated explanations are better at identifying important features and more robust to adversarial attacks.
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
GAM generates global explanations of neural networks by mapping prediction landscapes.
problem Lack of interpretability in neural networks.
method GAM (Global Explanations Mapping) method that explains neural network predictions across subpopulations.
result GAM's global explanations match feature weights of interpretable models and are intuitive to practitioners.
New framework quantifies uncertainties in neural network explanations.
problem Lack of methods to quantify uncertainties in neural network explanations.
method Converts any explanation method into a Bayesian neural network method, modeling uncertainties.
result Allows quantification of explanation uncertainties and appropriate confidence levels.
Manipulated explanations can be made imperceptible to the naked eye.
problem The trustworthiness and interpretability of neural networks can be compromised by manipulable explanations.
method The paper demonstrates that explanations can be altered by applying small, imperceptible input changes that do not affect the network's output.
result An upper bound on the susceptibility of explanations to manipulation has been derived, and effective mechanisms to enhance explanation robustness have been proposed.
Extends local attributions to Bayesian Neural Networks for improved explanations.
problem Lack of explanations for Bayesian Neural Networks' predictions.
method Extend local attributions to a probabilistic explanation distribution of BNNs.
result Enriches standard explanations with uncertainty information and visualizes explanation stability.
Study finds discrepancies in model explanations for clean vs. adversarial inputs.
problem Identifying adversarial attacks on neural networks.
method Utilized experimental results to propose a framework for detecting adversarial inputs.
result Proposed framework can identify adversarial inputs based on model explanations.
New method to evaluate visual explanations from neural networks.
problem Lack of consensus on measuring effectiveness of visual explanations.
method Proposed a new procedure for evaluating explanations using a range of sources.
result Demonstrated the benefit of combining different sources and the impact of bias parameters.
A new method explains RNNs by decision lists over skipgrams, improving explanation fidelity and interpretability.
problem Lack of understanding how input segments combine to form patterns in neural network outputs.
method Proposes a pipeline to explain RNNs using decision lists over skipgrams, creating synthetic and real-world datasets for evaluation.
result Persistently achieves high explanation fidelity and interpretable rules.
DeConvNet, Guided BackProp, LRP, were invented to better understand deep neural networks. We show that these methods do not produce the theoretically correct explanation for a linear model. Yet they are used on multi-layer networks with millions of parameters. This is a cause for concern since linear models are simple …
SpArX creates faithful explanations of neural networks' decision-making.
problem Challenges in explaining neural networks' decisions.
method Sparsifies MLPs while maintaining structure, then translates into QAFs for argumentative explanations.
result SpArX provides more faithful explanations than existing methods.
Bayesian explanations are more resilient to adversarial attacks than deterministic ones.
problem Stability of saliency-based explanations under adversarial attacks in Neural Networks.
method Empirical and theoretical analysis of Bayesian vs deterministic Neural Networks.
result Bayesian explanations are more stable under adversarial perturbations and direct attacks.
Extract class-specific subnetworks from neural models for better understanding and improved explanations.
problem Understanding and explaining the complex behavior of deep neural networks.
method For each semantic class, extract a class-specific subnetwork with a compressed structure that maintains comparable performance.
result Extracted subnetworks improve explanation saliency and adversarial example detection.
PiNets provide faithful explanations for neural networks.
problem Lack of true explanations for neural network predictions.
method Pointwise-interpretable Networks (PiNets) that form linear models instance-wise.
result PiNets offer explanations that are meaningful, aligned, robust, and sufficient.
Method assesses neural network feature importance based on sensitivity.
problem Neural networks lack interpretability, especially in regulated industries.
method Sensitivity-based feature importance assessment.
result Method provides fast, global, and local explanations for various neural network architectures.
RAW-Explainer generates interpretable subgraph explanations for link predictions in knowledge graphs.
problem Interpreting GNN predictions for link prediction in heterogeneous settings is challenging.
method RAW-Explainer uses random walk objective and neural network to generate connected, concise subgraph explanations.
result RAW-Explainer strikes a balance between explanation quality and computational efficiency.
ID-ExpO fine-tunes neural networks for more faithful explanations.
problem Improving the faithfulness of explanations for complex machine learning models.
method Differentiable insertion/deletion metric-aware regularizers for optimization.
result Fine-tuned predictors produce more faithful explanations.
GRACE generates concise explanations for neural networks on tabular data.
problem Lack of explainable AI solutions for neural networks on tabular data.
method Borrowing ideas from causality and philosophy, GRACE generates contrastive samples to explain neural network predictions.
result GRACE explanations are more intuitive and lead to better decision-making.
This paper improves neural network explanations by quantifying and visualizing semantic compositions.
problem Improving neural network explanations for natural language processing tasks.
method Proposes a formal way to quantify word and phrase importance, introduces SCD and SOC algorithms.
result Our algorithms outperform prior methods in explaining neural network predictions.
Simple aggregation of multiple methods defends against adversarial attacks on neural networks.
problem Manipulation of machine learning models to hide discriminating features.
method Aggregation of multiple explanation methods to make the model robust against adversarial attacks.
result The model remains robust even when the attacker has knowledge of the model weights and explanation methods.
New method quantifies intrinsic causal contributions in neural networks.
problem Measuring the causal influence of input features in deep neural networks.
method Proposes an identifiable generative post-hoc framework to quantify intrinsic causal contributions (ICC) as structural causal models.
result ICC generates more intuitive and reliable explanations compared to existing global explanation techniques.
Develops ACE to automatically identify meaningful concepts from neural network predictions.
problem Challenges in interpreting feature importance scores for machine learning models.
method Proposes concept-based explanation principles and develops ACE algorithm to extract visual concepts.
result Demonstrates ACE discovers human-meaningful, coherent concepts for neural network predictions.
Proposes a new method for better explaining neural network decisions.
problem Challenges in explaining neural network decisions due to base-point choice.
method Introduces tangentially aligned integrated gradients to maximize explanation tangential alignment.
result Optimal base-point maximizes explanation tangential alignment, leading to more accurate interpretations.
Local explanations of DNNs are insensitive to parameter values.
problem Sensitivity of local explanations to DNN parameter values.
method Assessed sensitivity of local explanations to DNN parameter values.
result DNNs with randomly-initialized weights produce similar explanations to those with learned weights.
This paper introduces a method to find complete and interpretable concept-based explanations for deep neural networks.
problem Lack of complete and interpretable concept-based explanations in deep neural networks.
method Definition of completeness, concept discovery method, and importance score calculation using game-theoretic notions.
result The proposed method finds complete and interpretable concept-based explanations for deep neural networks.
The paper tackles one-for-many counterfactual explanations using column generation.
problem Minimizing the number of explanations needed for a group of instances with sparsity constraints.
method Developed a novel column generation framework to efficiently search for explanations for any black-box classifier.
result The column generation framework outperforms existing methods in scalability, computational performance, and solution quality.
Paper offers local neural network explanations by considering model architecture.
problem Creating true neighbourhoods for black box models.
method Penultimate layer decoding for local neighbourhood generation.
result Local explanations are more accurate and relevant to instances.
GraphLIME explains GNN models by selecting key features locally.
problem Explaining the effectiveness of GNN models is challenging due to complex nonlinear transformations.
method GraphLIME uses HSIC Lasso for nonlinear feature selection in GNN models.
result GraphLIME provides more descriptive explanations than existing methods.
Humans are able to explain their reasoning. On the contrary, deep neural networks are not. This paper attempts to bridge this gap by introducing a new way to design interpretable neural networks for classification, inspired by physiological evidence of the human visual system's inner-workings. This paper proposes a neu…
Formulates approach for guiding explanation types based on user specifications.
problem Creating explainable AI components from user-defined specifications.
method Develops a method for generating explanations based on user-defined specifications.
result Demonstrates feasibility of user-defined explanations for complex models like Bayesian networks and graph neural networks.
Adversarial training improves the sparsity and stability of neural network explanations.
problem Creating concise and stable explanations for neural network outputs.
method Theoretical exploration and empirical verification of adversarial training's impact on feature attributions.
result Adversarial training leads to sparser and more stable feature attributions in neural networks.
Understanding neural networks is becoming increasingly important. Over the last few years different types of visualisation and explanation methods have been proposed. However, none of them explicitly considered the behaviour in the presence of noise and distracting elements. In this work, we will show how noise and dis…
HSIC-based method explains GNN structures.
problem Interpreting GNNs' complex dynamics.
method Flexible model-agnostic explanation using HSIC.
result Identifies crucial structures in GNNs.
Paper compares two local explanation methods for machine learning models.
problem Comparing two local explanation methods for machine learning models.
method Integrated Gradients and Baseline Shapley methods.
result Additional insights on comparative behavior for tabular data and neural networks.
COMRECGC finds common recourse for global counterfactual explanations in GNNs.
problem Finding common recourse for global counterfactual explanations in GNNs.
method Formalized the common recourse explanation problem and designed COMRECGC algorithm.
result COMRECGC outperforms strong baselines on four real-world graph datasets.
This thesis trains machine learning models by penalizing their explanations, improving interpretability and generalization.
problem Neural networks' opacity and lack of explainability in critical applications.
method Regularizing explanations of machine learning models, penalizing inconsistency and complexity.
result Trained models provide more interpretable rationales and better generalize to new, confounded data.
OrphicX generates causal explanations for GNNs by isolating latent causal factors.
problem Generating interpretable causal explanations for complex graph neural networks.
method Develops a generative model and objective function to isolate latent causal factors, maximizing information flow.
result OrphicX effectively identifies causal semantics, significantly outperforming alternatives.
Efficiently generates and selects explanations for neural networks using GANs and FID.
problem Manual selection of hyper-parameters for generating interpretable neural network explanations is slow and requires qualitative evaluation.
method Proposes a novel metric using Fréchet Inception Distance (FID) and a GAN-based method for efficient search and realistic output generation.
result Successfully selects hyper-parameters leading to interpretable examples, avoiding manual evaluation.
We study the problem of explaining a rich class of behavioral properties of deep neural networks. Distinctively, our influence-directed explanations approach this problem by peering inside the network to identify neurons with high influence on a quantity and distribution of interest, using an axiomatically-justified in…
Unified understanding of neural networks on group operations verified.
problem Understanding and verifying neural networks trained on group operations.
method Investigated one-hidden-layer neural networks trained on binary operation of finite groups, revealing structure and providing a compact proof of model performance.
result Verified explanation applies to a large fraction of networks trained on the symmetric group S5, providing a >=95% accuracy bound for 45% of models.
Adversarial manipulation fools neural network interpretations.
problem Ensuring interpretation methods are robust to model manipulations.
method Integrating manipulation into fine-tuning objective function.
result State-of-the-art interpreters can be fooled with minimal accuracy loss.
Neural networks memorize exceptions, leading to poor generalization.
problem Memorization of exceptions hinders neural network generalization.
method Formalized memorization-generalization interplay, proposed MAT to shift logits.
result MAT improves generalization by learning robust patterns invariant across distributions.
GNNExplainer provides interpretable explanations for GNN predictions.
problem Explaining GNN predictions remains unsolved due to complex model structure.
method Formulates GNNExplainer as an optimization task maximizing mutual information between prediction and subgraph structures.
result Identifies crucial subgraph structures and node features for GNN predictions.
Develops masks to explain neural network predictions.
problem Improving neural network interpretability for various applications.
method Creates explanation masks for pre-trained networks using a secondary network.
result Demonstrates the effectiveness of the method across different types of networks.
DistShap parallelizes GNN explanation for large graphs.
problem Computational expense in attributing GNN predictions to specific edges or features.
method Distributed Shapley values across multiple GPUs for scalable GNN explanations.
result DistShap outperforms existing methods and scales to models with millions of features.
New method explains GNN predictions using walks.
problem GNNs are black-boxes and hard to explain.
method Nested attribution scheme using relevant walks.
result Extracts meaningful explanations from GNNs.
NeuroMask provides interpretable explanations for deep neural networks.
problem Understanding how deep neural networks make decisions.
method Applies a mask to reveal or hide parts of an image, tuning mask values to preserve classification results and produce interpretable explanations.
result NeuroMask successfully localizes the most relevant parts of an image to a deep neural network's decision.