Debias concept-based explanations by removing confounding information.
problem Correlation between concepts and confounding features.
method Causal prior graph and two-stage regression technique.
result Success in removing biases and improving concept ranking.
Proposes new methods for interpreting document classification models.
problem Interpretation fragility of attention-based neural networks.
method Corpus-level and concept-based explanation methods using attention weights.
result Extracts semantically meaningful keywords and concepts for model predictions.
New models handle survival analysis with concept-based learning.
problem Survival analysis tasks involving event times with censored data.
method SurvCBM and SurvRCM models integrating concept-based learning with survival analysis.
result SurvCBM outperforms traditional survival models in numerical experiments.
CB-SLICE identifies concept-based error slices in deep learning models.
problem Systematic errors in deep learning models on specific groups.
method Concept Bottleneck Models (CBMs) and concept representations.
result CB-SLICE outperforms state-of-the-art methods in error slice identification.
Interpretability has become an important topic of research as more machine learning (ML) models are deployed and widely used to make important decisions. Most of the current explanation methods provide explanations through feature importance scores, which identify features that are important for each individual input. …
Optimizes explanations for better listener understanding.
problem Insufficient consideration of listener preferences in concept-based explanations.
method Iterative training procedure based on direct preference optimization.
result Pragmatic explanations improve both model accuracy and user understanding.
Human explanations of high-level decisions are often expressed in terms of key concepts the decisions are based on. In this paper, we study such concept-based explainability for Deep Neural Networks (DNNs). First, we define the notion of completeness, which quantifies how sufficient a particular set of concepts is in e…
Interpretable ML methods for better decision-making with explanations.
problem Lack of transparency in black-box ML models.
method Use of Formal Concept Analysis and cooperative game theory to assess attribute importance and reduce attribute count.
result Developed methods to assess attribute importance and reduce attribute count in ML models.
Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.
problem CNNs lack accurate data representation in their built-in prototypes.
method Introduces k-means-based post-hoc explanations for CNNs, leveraging spatial consistency of convolutional receptive fields.
result Using shallower, less compressed feature activations improves semantic fidelity at the cost of slight predictive performance.
MCD offers a complete model understanding for high-stake decisions.
problem Local model understanding in XAI methods is not sufficient for high-stake decisions.
method MCD extends concept-based methods to ensure global model understanding via multi-dimensional subspaces.
result MCD provides a complete model understanding, ensuring the model reasoning is related to the actual model.
New framework quantifies and reduces concept-based models' leakage.
problem Information leakage in concept-based models reduces interpretability.
method Information-theoretic framework with CTL and ICL measures.
result Measures predict model behaviour and identify leakage causes.
LCBM model improves image classification without human supervision.
problem Improving interpretability and generalization of unsupervised concept-based models.
method LCBM models concepts as random variables in a Bernoulli latent space, reducing the number of concepts without sacrificing performance.
result LCBM outperforms existing models in generalization and interpretability.
DCR improves interpretability of concept-based models by using neural networks to build rule structures.
problem Inability of concept-based models to provide transparent decision processes.
method DCR uses neural networks to build syntactic rule structures using concept embeddings and executes these rules on concept truth degrees.
result DCR improves interpretability by up to 25% on challenging benchmarks and discovers meaningful logic rules.
A method for concept-based learning using probabilistic inference and expert rules.
problem Concept-based learning with limited training data.
method Divide images into patches, transform into embeddings, cluster, and use frequentist inference to find concepts.
result FI-CBL outperforms concept bottleneck model in small data scenarios.
COCKATIEL explains neural net models on NLP tasks by identifying meaningful concepts.
problem Transformer models are complex and hard to interpret.
method COCKATIEL uses NMF and sensitivity analysis to identify and rank concepts used by the model.
result COCKATIEL provides accurate and meaningful explanations without affecting model performance.
Combines neural networks and expert rules for concept-based learning.
problem Extending concept-based learning with machine learning models.
method Form constraints for joint probability distribution and represent feasible set as a convex polytope.
result Neural networks can be trained to satisfy expert rules without violating them.
Concept-driven OPE reduces variance in off-policy decision evaluation.
problem High variance in off-policy decision evaluation due to limited sample sizes.
method Integrating human-explainable concepts into OPE to reduce variance.
result Concept-based OPE estimators remain unbiased and reduce variance when concepts are known and predefined.
New method discovers concepts in hidden feature layers using sparse subspace clustering.
problem Local attribution methods fail to identify coherent model behavior across samples.
method Sparse Subspace Clustering (SSCC) for concept discovery.
result Empirically validated method for various image classification tasks.
Method makes non-interpretable models more intervenable.
problem Making non-interpretable models more understandable and controllable.
method Intervenability formalization and fine-tuning of black-box models.
result Fine-tuned black-box models are more intervenable and often better-calibrated.
This paper analyzes the variability of Concept Activation Vectors (CAVs).
problem The variability of CAVs in explaining AI models.
method Theoretical analysis and experiments on real-life datasets to quantify CAVs variability.
result The variance of CAVs decreases as 1/N, where N is the number of random examples.
STOOD-X detects out-of-distribution samples without distributional assumptions and provides explainable visualizations.
problem Challenges in OOD detection, including restrictive assumptions, scalability issues, and lack of interpretability.
method Two-stage methodology combining statistical nonparametric test and explainability enhancements.
result Achieves competitive performance in high-dimensional and complex settings, with explainability framework enabling human oversight.
The problem of detecting terms that can be interesting to the advertiser is considered. If a company has already bought some advertising terms which describe certain services, it is reasonable to find out the terms bought by competing companies. A part of them can be recommended as future advertising terms to the compa…
Researchers apply concept-based explainability to EEG data.
problem Understanding the internal states of complex EEG transformer models.
method Concept Activation Vectors (CAVs) adapted for EEG data, using externally labeled datasets and anatomically defined concepts.
result Both approaches to concept formation yield valuable insights into EEG model representations.
This is a survey on old and new results as well as an introduction to various related basic notions and concepts, based on two talks given at the International Workshop on Geometry and Analysis in Kemerovo (Sobolev Institute of Mathematics, Kemerovo State University) and at the University of Krasnojarsk in June 2011. W…
A concise review of recent few-shot meta-learning methods.
problem Mimicking human fast adaptation to new concepts based on prior knowledge.
method Categorized into four branches based on technical characteristics.
result Current challenges and future prospects identified.
The intuition of risk is based on two main concepts: loss and variability. In this paper, we present a composition of risk and deviation measures, which contemplate these two concepts. Based on the proposed Limitedness axiom, we prove that this resulting composition, based on properties of the two components, is a cohe…
ECBMs unify concept-based interpretations in deep learning models.
problem Suboptimal final accuracy and lack of concept interaction and conditional dependencies.
method ECBMs use a set of neural networks to define joint energy, enabling concept correction and conditional dependency quantification.
result ECBMs achieve higher accuracy and richer concept interpretations compared to state-of-the-art methods.
Geometric framework detects concept frustration between human concepts and machine representations.
problem Aligning human concepts with machine learning representations.
method Geometric framework and similarity measures for detecting concept frustration.
result Concept frustration affects machine learning model performance and reorganizes learned concept representations.
In an attempt to gather a deeper understanding of how convolutional neural networks (CNNs) reason about human-understandable concepts, we present a method to infer labeled concept data from hidden layer activations and interpret the concepts through a shallow decision tree. The decision tree can provide information abo…
Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based e…
A method to select validation data from a dataset using statistical criteria.
problem Selecting a validation basis from a full dataset for machine learning model validation.
method Adopting a 'design of experiments' point of view and using statistical criteria, particularly Maximum Mean Discrepancy criteria.
result The 'support points' concept is particularly relevant for selecting validation data.
AXE evaluates explanations to avoid misleading Rashomon set model selection.
problem Evaluating explanations for Rashomon set models to avoid false selection.
method Proposed AXE method to evaluate explanation quality.
result AXE detects adversarial fairwashing with 100% success rate.
The paper proposes criteria and methods for evaluating and aggregating feature-based model explanations.
problem Lack of quantitative evaluation criteria for feature-based model explanations.
method Developed quantitative evaluation criteria (low sensitivity, high faithfulness, low complexity), devised a framework for aggregation, and derived a new aggregate Shapley value explanation function.
result A new aggregate Shapley value explanation function that minimizes sensitivity.
New definition reveals encoding explanations that retain predictive power.
problem Challenges in evaluating and identifying encoding explanations.
method Developed a definition of encoding based on conditional dependence.
result Existing evaluation scores do not rank non-encoding explanations correctly, but STRIPE-X does.
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Differentially private algorithms protect model explanations from leaking training data.
problem Model explanations can leak training data, compromising privacy.
method Adaptive differentially private gradient descent algorithm to produce accurate, private explanations.
result Privacy amplification and reduction of overall privacy loss on explanation data.
We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fidelity, and sensitivity. We analyze optimal explanations with respect to both these measures, and while…
Paper explores a consumer-friendly approach to explain machine learning decisions.
problem Challenges in providing understandable explanations for machine learning predictions.
method Consumer-driven approach called TED that asks for explanations in training data.
result TED is robust to increasing numbers of explanations, noisy explanations, and missing explanations.
Formalizes explanations as blending input and model output.
problem Creating clear and consistent explanations for model predictions.
method Defines properties of explanation functions and links them to model layers.
result Consistency of activations across layers implies consistency of explanations.
Improves global counterfactual explanations for model recourse.
problem Inability to provide explanations beyond local instances.
method Investigates and improves Actionable Recourse Summaries (AReS) for global counterfactual explanations.
result Develops more efficient and interactive explainability tools.
New framework evaluates model explanations based on decision task improvement.
problem Evaluation of model explanations often misses practical value.
method Decision-theoretic framework quantifying three key values.
result Provides benchmarks and interprets human-AI decision support.
We define and compute plausible counterfactual explanations using density constraints.
problem Efficiently compute plausible counterfactual explanations for machine learning models.
method Propose and study a formal definition of plausible counterfactual explanations, use density estimators, and introduce convex density constraints.
result Convex density constraints ensure plausible and feasible counterfactual explanations.
G-SHAP generates multiple types of explanations for machine learning models.
problem Understanding model predictions and their differences across groups.
method Generalization of SHAP method to produce additional types of explanations.
result G-SHAP produces explanations for classification, intergroup differences, and model failure.
LLMs' explanations are often insufficient and vary with input distribution.
problem Evaluating the sufficiency of LLM explanations without predefined biases.
method Generalizing sufficiency to arbitrary explanations, using LLM's input beliefs, and introducing SCSuff metric.
result Explanation sufficiency can vary with input distribution and is weakly correlated with model size, accuracy, or output entropy.
Study finds visual explanations do not significantly improve human accuracy or trust in model predictions.
problem Measuring the impact of visual explanations on human accuracy and trust in model predictions.
method Randomized controlled trial with image-based age prediction task, varying levels of explanation quality.
result Visual explanations do not significantly alter human accuracy or trust in the model.
Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation. Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a sing…
Managing large-scale transportation infrastructure projects is difficult due to frequent misinformation about the costs which results in large cost overruns that often threaten the overall project viability. This paper investigates the explanations for cost overruns that are given in the literature. Overall, four categ…
This research investigates reliable local explanations for machine listening models.
problem Generating reliable local explanations for machine listening models.
method Investigates the sensitivity of SoundLIME explanations to input perturbations and proposes a novel method for identifying suitable content types.
result SoundLIME explanations are sensitive to the content in occluded input regions, and the average magnitude of input mel-spectrogram bins is the most suitable content type for temporal explanations.