This paper improves neural network explanations by quantifying and visualizing semantic compositions.
problem Improving neural network explanations for natural language processing tasks.
method Proposes a formal way to quantify word and phrase importance, introduces SCD and SOC algorithms.
result Our algorithms outperform prior methods in explaining neural network predictions.
New method explains classifiers trained on raw hierarchical data.
problem Lack of interpretability in classifiers trained on raw structured data.
method Treating classifiers as subset selection problems, generating interpretable explanations efficiently.
result Computational efficiency and higher-quality explanations compared to existing methods.
Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models. We propose Mahé, a novel approach to provide Model-agnostic hierarchical éxplanations of how powerful machine learning models, such as deep ne…
Improves hierarchical clustering in Euclidean space using autoencoders.
problem Lack of unsupervised methods for learning hierarchical structure in Euclidean space.
method Variational autoencoder with Gaussian mixture prior, rescaling latent space, and Ward's linkage.
result Improved dendrogram purity and Moseley-Wang cost function results.
Sharp theory of neural network scaling laws for hierarchical targets.
problem Learning hierarchical multi-index models in neural networks.
method Sharp information-theoretic scaling laws derived for two-layer neural networks.
result Optimal rates achieved by a simple spectral estimator.
Measures faithfulness of LLM explanations to reveal hidden biases and misleading claims.
problem LLM explanations can misrepresent the model's reasoning process, leading to over-trust and misuse.
method Defines faithfulness in terms of concept influence and uses counterfactuals and Bayesian models to estimate it.
result Can quantify and discover interpretable patterns of unfaithfulness in LLM explanations.
Interpretable Machine Learning (IML) has become increasingly important in many real-world applications, such as autonomous cars and medical diagnosis, where explanations are significantly preferred to help people better understand how machine learning systems work and further enhance their trust towards systems. Howeve…
Local Interpretable Model-Agnostic Explanations (LIME) is a popular technique used to increase the interpretability and explainability of black box Machine Learning (ML) algorithms. LIME typically generates an explanation for a single prediction by any ML model by learning a simpler interpretable model (e.g. linear cla…
Transformers prefer simpler explanations in hierarchical tasks.
problem Navigating tasks with varying complexity levels.
method Well-controlled testbeds based on Markov chains and linear regression.
result Transformers favor the least complex sufficient explanation when presented with simpler data.
A new framework quantifies how model explanations influence each other.
problem Understanding how different model explanations interact and influence each other.
method Introducing the metagame, a conceptual framework for measuring second-order interaction effects of model explanations using Shapley values.
result Meta-attributions provide directional insights into how feature interactions influence model explanations.
We develop three efficient approaches for generating visual explanations from 3D convolutional neural networks (3D-CNNs) for Alzheimer's disease classification. One approach conducts sensitivity analysis on hierarchical 3D image segmentation, and the other two visualize network activations on a spatial map. Visual chec…
U-Nets use belief propagation for efficient image denoising and classification.
problem Efficiently denoise and classify images using generative hierarchical models.
method Interpreted U-Nets as implementing belief propagation in tree-structured models.
result U-Nets can efficiently approximate denoising functions with sample complexity bounds.
Abstraction plays a key role in concept learning and knowledge discovery; this paper is concerned with computational abstraction. In particular, we study the nature of abstraction through a group-theoretic approach, formalizing it as symmetry-driven---as opposed to data-driven---hierarchical clustering. Thus, the resul…
INVERT connects neural representations to human-understandable concepts.
problem Lack of understanding and statistical significance in existing explainability methods.
method Inverse Recognition (INVERT) approach that connects learned representations to human-understandable concepts.
result INVERT provides interpretable metrics and statistical significance for representation alignment.
While generative models such as Latent Dirichlet Allocation (LDA) have proven fruitful in topic modeling, they often require detailed assumptions and careful specification of hyperparameters. Such model complexity issues only compound when trying to generalize generative models to incorporate human input. We introduce …
Residual networks' depth is mathematically equivalent to expanding an implicit ensemble size.
problem Understanding why deep residual networks are effective.
method Formal analysis of residual networks as ensembles of shallow models.
result Increasing network depth is equivalent to expanding the size of an implicit ensemble, revealing a hierarchical structure.
Strong regulations in the financial industry mean that any decisions based on machine learning need to be explained. This precludes the use of powerful supervised techniques such as neural networks. In this study we propose a new unsupervised and semi-supervised technique known as the topological hierarchical decomposi…
New method explains ML performance gaps without causal knowledge.
problem Understanding why ML algorithms perform differently across domains.
method Nonparametric hierarchical decomposition framework.
result Detailed variable-level explanations for performance gaps.
Vision models are interpretable when they classify objects on the basis of features that a person can directly understand. Recently, methods relying on visual feature prototypes have been developed for this purpose. However, in contrast to how humans categorize objects, these approaches have not yet made use of any tax…
With the advent of highly predictive but opaque deep learning models, it has become more important than ever to understand and explain the predictions of such models. Existing approaches define interpretability as the inverse of complexity and achieve interpretability at the cost of accuracy. This introduces a risk of …
Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based e…
AXE evaluates explanations to avoid misleading Rashomon set model selection.
problem Evaluating explanations for Rashomon set models to avoid false selection.
method Proposed AXE method to evaluate explanation quality.
result AXE detects adversarial fairwashing with 100% success rate.
The study initiates a theoretical analysis of dynamic benchmarking models.
problem Lack of theoretical foundation and empirical studies in dynamic benchmarks.
method Examined two realizations of dynamic benchmarking: sequential and hierarchical dependency models.
result Sequential dynamic benchmarks show initial performance improvement but can stall after three rounds due to label noise.
The paper proposes criteria and methods for evaluating and aggregating feature-based model explanations.
problem Lack of quantitative evaluation criteria for feature-based model explanations.
method Developed quantitative evaluation criteria (low sensitivity, high faithfulness, low complexity), devised a framework for aggregation, and derived a new aggregate Shapley value explanation function.
result A new aggregate Shapley value explanation function that minimizes sensitivity.
New definition reveals encoding explanations that retain predictive power.
problem Challenges in evaluating and identifying encoding explanations.
method Developed a definition of encoding based on conditional dependence.
result Existing evaluation scores do not rank non-encoding explanations correctly, but STRIPE-X does.
Advances in unsupervised learning enable reconstruction and generation of samples from complex distributions, but this success is marred by the inscrutability of the representations learned. We propose an information-theoretic approach to characterizing disentanglement and dependence in representation learning using mu…
Survey on efficient counterfactual explanations for various ML models.
problem Providing understandable explanations for machine learning predictions.
method Review and propose methods for computing counterfactual explanations.
result Efficient methods for various ML models and new methods for unconsidered models.
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Differentially private algorithms protect model explanations from leaking training data.
problem Model explanations can leak training data, compromising privacy.
method Adaptive differentially private gradient descent algorithm to produce accurate, private explanations.
result Privacy amplification and reduction of overall privacy loss on explanation data.
Paper proposes metrics to evaluate AI explanations without ground truth.
problem Challenges in evaluating neural network explanations without ground truth.
method Designs four metrics to evaluate explanation results.
result New insights into neural network interpretation methods.
We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fidelity, and sensitivity. We analyze optimal explanations with respect to both these measures, and while…
Paper explores a consumer-friendly approach to explain machine learning decisions.
problem Challenges in providing understandable explanations for machine learning predictions.
method Consumer-driven approach called TED that asks for explanations in training data.
result TED is robust to increasing numbers of explanations, noisy explanations, and missing explanations.
Formalizes explanations as blending input and model output.
problem Creating clear and consistent explanations for model predictions.
method Defines properties of explanation functions and links them to model layers.
result Consistency of activations across layers implies consistency of explanations.
Improves global counterfactual explanations for model recourse.
problem Inability to provide explanations beyond local instances.
method Investigates and improves Actionable Recourse Summaries (AReS) for global counterfactual explanations.
result Develops more efficient and interactive explainability tools.
New framework evaluates model explanations based on decision task improvement.
problem Evaluation of model explanations often misses practical value.
method Decision-theoretic framework quantifying three key values.
result Provides benchmarks and interprets human-AI decision support.
We define and compute plausible counterfactual explanations using density constraints.
problem Efficiently compute plausible counterfactual explanations for machine learning models.
method Propose and study a formal definition of plausible counterfactual explanations, use density estimators, and introduce convex density constraints.
result Convex density constraints ensure plausible and feasible counterfactual explanations.
G-SHAP generates multiple types of explanations for machine learning models.
problem Understanding model predictions and their differences across groups.
method Generalization of SHAP method to produce additional types of explanations.
result G-SHAP produces explanations for classification, intergroup differences, and model failure.
LLMs' explanations are often insufficient and vary with input distribution.
problem Evaluating the sufficiency of LLM explanations without predefined biases.
method Generalizing sufficiency to arbitrary explanations, using LLM's input beliefs, and introducing SCSuff metric.
result Explanation sufficiency can vary with input distribution and is weakly correlated with model size, accuracy, or output entropy.
Study finds visual explanations do not significantly improve human accuracy or trust in model predictions.
problem Measuring the impact of visual explanations on human accuracy and trust in model predictions.
method Randomized controlled trial with image-based age prediction task, varying levels of explanation quality.
result Visual explanations do not significantly alter human accuracy or trust in the model.
Improved VAEs by training a contrastive prior to match posterior.
problem Prior hole problem in VAEs, leading to poor image generation.
method Introduced a contrastive energy-based prior and trained it using noise contrastive estimation.
result Significant improvement in VAE generative performance on various datasets.
Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation. Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a sing…
The paper introduces a new framework for making machine learning explanations more understandable to humans.
problem Making machine learning explanations comprehensible and aligned with human preferences.
method Inspired by philosophy, cognitive science, and social sciences, the paper formalizes a framework using the concept of 'weight of evidence' from information theory.
result The framework produces intuitive and comprehensible explanations that align with human preferences.
Managing large-scale transportation infrastructure projects is difficult due to frequent misinformation about the costs which results in large cost overruns that often threaten the overall project viability. This paper investigates the explanations for cost overruns that are given in the literature. Overall, four categ…
This research investigates reliable local explanations for machine listening models.
problem Generating reliable local explanations for machine listening models.
method Investigates the sensitivity of SoundLIME explanations to input perturbations and proposes a novel method for identifying suitable content types.
result SoundLIME explanations are sensitive to the content in occluded input regions, and the average magnitude of input mel-spectrogram bins is the most suitable content type for temporal explanations.
Defines explanations for classifier outcomes using causal concepts.
problem Understanding classifier outcomes in a causal context.
method Proposes a new definition of explanation based on causality, compares it with existing notions, and evaluates it experimentally.
result Experimental evaluation shows the new definition's effectiveness on financial datasets.
This research improves interpretability in sequential explanations using mental models.
problem Improving interpretability in sequential explanations between two parties.
method A reinforcement learning framework that selects explanations based on the explainee's mental model.
result Mental model-based policies increase interpretability over random selection in multiple sequential explanations.
R package for counterfactual explanation methods.
problem Lack of unified interfaces for counterfactual explanation methods.
method Developed a modular R6-based interface for three existing counterfactual methods and proposed extensions.
result Comparison of implemented methods' quality and runtime behavior.
Explanation in machine learning and related fields such as artificial intelligence aims at making machine learning models and their decisions understandable to humans. Existing work suggests that personalizing explanations might help to improve understandability. In this work, we derive a conceptualization of personali…