Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based e…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “explanation methods”
R package for counterfactual explanation methods.
problem Lack of unified interfaces for counterfactual explanation methods.
method Developed a modular R6-based interface for three existing counterfactual methods and proposed extensions.
result Comparison of implemented methods' quality and runtime behavior.
GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanationsstat.ML
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation. Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a sing…
This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To …
AXE evaluates explanations to avoid misleading Rashomon set model selection.
problem Evaluating explanations for Rashomon set models to avoid false selection.
method Proposed AXE method to evaluate explanation quality.
result AXE detects adversarial fairwashing with 100% success rate.
Improves global counterfactual explanations for model recourse.
problem Inability to provide explanations beyond local instances.
method Investigates and improves Actionable Recourse Summaries (AReS) for global counterfactual explanations.
result Develops more efficient and interactive explainability tools.
This research investigates reliable local explanations for machine listening models.
problem Generating reliable local explanations for machine listening models.
method Investigates the sensitivity of SoundLIME explanations to input perturbations and proposes a novel method for identifying suitable content types.
result SoundLIME explanations are sensitive to the content in occluded input regions, and the average magnitude of input mel-spectrogram bins is the most suitable content type for temporal explanations.
Issues regarding explainable AI involve four components: users, laws & regulations, explanations and algorithms. Together these components provide a context in which explanation methods can be evaluated regarding their adequacy. The goal of this chapter is to bridge the gap between expert users and lay users. Different…
Proposes a method for multilevel explanations of black-box models.
problem Need for explanations at intermediate or group levels, especially for GDPR compliance.
method Meta-method that builds a multilevel explanation tree using local explainability methods.
result Effective multilevel explanations for groups of data points, including novel test points.
New method provides calibrated feature importance explanations for regression models.
problem Lack of uncertainty quantification in existing local explanation methods.
method Extension of Calibrated Explanations method to support regression and probabilistic regression.
result Calibrated Explanations for regression provides quantified uncertainty and robust explanations.
New method shows unfair explanations can be created for any classifier.
problem Fairness in black-box classifier explanations is not guaranteed.
method Constructs a new classifier with similar performance but altered explanations.
result Explanation methods can be manipulated to show unfair decisions.
Unified framework for explaining models by removing features.
problem Unclear relationships and preferences among existing model explanation methods.
method Removal-based explanations characterized by three dimensions.
result Unified framework unifies 26 existing methods.
The paper proposes criteria and methods for evaluating and aggregating feature-based model explanations.
problem Lack of quantitative evaluation criteria for feature-based model explanations.
method Developed quantitative evaluation criteria (low sensitivity, high faithfulness, low complexity), devised a framework for aggregation, and derived a new aggregate Shapley value explanation function.
result A new aggregate Shapley value explanation function that minimizes sensitivity.
Method trains deep models to explain predictions with fewer examples.
problem Difficulty in humans understanding deep model predictions.
method Simultaneously trains prediction and explanation models with sparse regularization.
result Improves faithfulness of explanations with fewer examples while maintaining predictive performance.
Optimizes explanations for better listener understanding.
problem Insufficient consideration of listener preferences in concept-based explanations.
method Iterative training procedure based on direct preference optimization.
result Pragmatic explanations improve both model accuracy and user understanding.
Due to the increasing use of machine learning in practice it becomes more and more important to be able to explain the prediction and behavior of machine learning models. An instance of explanations are counterfactual explanations which provide an intuitive and useful explanations of machine learning models. In this su…
Explanation methods aim to make neural networks more trustworthy and interpretable. In this paper, we demonstrate a property of explanation methods which is disconcerting for both of these purposes. Namely, we show that explanations can be manipulated arbitrarily by applying visually hardly perceptible perturbations to…
Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insightscs.LG
Paper compares two local explanation methods for machine learning models.
problem Comparing two local explanation methods for machine learning models.
method Integrated Gradients and Baseline Shapley methods.
result Additional insights on comparative behavior for tabular data and neural networks.
New method evaluates visual explanations of deep models using adversarial perturbations.
problem Lack of objective evaluation of visual explanations of deep models.
method Proposes an adversarial perturbation approach to evaluate visual explanations of deep models.
result Demonstrates the effectiveness of the proposed approach through comparisons with existing methods.
Unified framework for model explanation methods based on feature removal.
problem Unclear relationships and preferences among various model explanation methods.
method Characterizes removal-based explanations along three dimensions.
result Unified 26 existing methods, including widely used approaches.
Explainable Artificial Intelligence (XAI)has received a great deal of attention recently. Explainability is being presented as a remedy for the distrust of complex and opaque models. Model agnostic methods such as LIME, SHAP, or Break Down promise instance-level interpretability for any complex machine learning model. …
GLOBE-CE offers efficient global counterfactual explanations.
problem Lack of reliable and scalable global counterfactual explanations.
method Translation-based approach for global counterfactual explanations.
result GLOBE-CE performs significantly better than current methods across multiple metrics.
The paper introduces a method to learn models with built-in explanations.
problem Lack of interpretability in deep learning models.
method Formalizes learning with explanation constraints and provides a learning theoretic framework.
result Models that satisfy these constraints have reduced Rademacher complexities, improving their performance.
New method for explaining dialogue response generation models.
problem Interpreting sequence generation models, especially dialogue response generation.
method Local Explanation of Response Generation (LERG) method.
result LERG improves dialogue response generation explanations compared to existing methods.
We introduce a method, KL-LIME, for explaining predictions of Bayesian predictive models by projecting the information in the predictive distribution locally to a simpler, interpretable explanation model. The proposed approach combines the recent Local Interpretable Model-agnostic Explanations (LIME) method with ideas …
We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fidelity, and sensitivity. We analyze optimal explanations with respect to both these measures, and while…
New framework quantifies uncertainties in neural network explanations.
problem Lack of methods to quantify uncertainties in neural network explanations.
method Converts any explanation method into a Bayesian neural network method, modeling uncertainties.
result Allows quantification of explanation uncertainties and appropriate confidence levels.
Explains agent behavior through intended outcomes in reinforcement learning.
problem Proving impossibility of general post-hoc explanations in reinforcement learning.
method Derives local explanations based on intention for Q-function approximations, proving consistency with learned Q-values.
result Demonstrates the necessity of collecting information during training for accurate explanations.
This study analyzes counterfactual explanations for student success models.
problem Improving trust in machine learning models for student success prediction.
method Comparison of counterfactual generation methods (WhatIf, Multi-Objective, Nearest Instance) for student success prediction models.
result WhatIf Counterfactual Explanations are more effective for student success prediction models.
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecs.LG
For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective. Too often, the litany of proposed explainable deep learning methods stop at the first step, providing practitioners with insight into a model, but…
New metric assesses reliability of AI explanations.
problem Unreliable AI explanations under realistic conditions.
method Explanation Reliability Index (ERI) metrics quantifying stability under four axioms.
result Widespread reliability failures in popular explanation methods.
Study shows explanation disparities in machine learning models are influenced by data and model properties.
problem Disparities in post-hoc machine learning explanation methods across race and gender.
method Simulations and experiments on a real-world dataset to assess challenges to explanation disparities.
result Increased covariate shift, concept shift, and omission of covariates increase explanation disparities, especially for neural network models.
Modern instance-based model-agnostic explanation methods (LIME, SHAP, L2X) are of great use in data-heavy industries for model diagnostics, and for end-user explanations. These methods generally return either a weighting or subset of input features as an explanation of the classification of an instance. An alternative …
G-SHAP generates multiple types of explanations for machine learning models.
problem Understanding model predictions and their differences across groups.
method Generalization of SHAP method to produce additional types of explanations.
result G-SHAP produces explanations for classification, intergroup differences, and model failure.
New method to evaluate visual explanations from neural networks.
problem Lack of consensus on measuring effectiveness of visual explanations.
method Proposed a new procedure for evaluating explanations using a range of sources.
result Demonstrated the benefit of combining different sources and the impact of bias parameters.
Many proposed methods for explaining machine learning predictions are in fact challenging to understand for nontechnical consumers. This paper builds upon an alternative consumer-driven approach called TED that asks for explanations to be provided in training data, along with target labels. Using semi-synthetic data fr…
Develops causal explanations for black-box classifiers.
problem Creating understandable explanations for complex models.
method Generative model with information-theoretic measures of causal influence.
result Learned latent factors influence classifier outputs.
Paper proposes Coalitional BAE to improve explainability of unsupervised deep learning models.
problem Improving explainability of Autoencoder's predictions.
method Introduces Coalitional BAE, inspired by agent-based system theory, to reduce correlation in explanations.
result Improved quality of explanations using Coalitional BAE on publicly available datasets.
BRACE generates efficient counterfactual explanations by integrating causal reasoning.
problem Challenges in traditional counterfactual explanations, especially neglecting causal relationships.
method Backtracking counterfactuals with causal reasoning.
result Our method provides deeper insights into model outputs and is computationally efficient.
ALMANACS benchmarks explainability methods on simulatability.
problem Evaluating the effectiveness of explainability methods for language models.
method ALMANACS is a simulatability benchmark that evaluates explainability methods on twelve safety-relevant topics.
result No explainability method outperforms the explanation-free control across all topics.
RAW-Explainer generates interpretable subgraph explanations for link predictions in knowledge graphs.
problem Interpreting GNN predictions for link prediction in heterogeneous settings is challenging.
method RAW-Explainer uses random walk objective and neural network to generate connected, concise subgraph explanations.
result RAW-Explainer strikes a balance between explanation quality and computational efficiency.
Deep learning is increasingly used as a building block of security systems. Unfortunately, neural networks are hard to interpret and typically opaque to the practitioner. The machine learning community has started to address this problem by developing methods for explaining the predictions of neural networks. While sev…
Attack reveals model details from counterfactual explanations.
problem Extracting model details from counterfactual explanations.
method Adversary uses counterfactual explanations to build high-fidelity model.
result High-fidelity and high-accuracy model extraction possible.
We introduce a new method to explain Gaussian processes using Shapley values.
problem Explaining the uncertainty in Gaussian process models.
method Extending Shapley values to stochastic cooperative games for Gaussian processes.
result Our method generates explanations that are random variables and satisfy favorable axioms.
Improves local model explanations using GANs and Linear Model Trees.
problem Need for accurate and intuitive explanations of complex machine learning models.
method Generative Adversarial Network (GAN) for synthetic data generation and Linear Model Trees for surrogate model training.
result Significantly improved local model explanations with contextual information.
With the advent of GDPR, the domain of explainable AI and model interpretability has gained added impetus. Methods to extract and communicate visibility into decision-making models have become legal requirement. Two specific types of explanations, contrastive and counterfactual have been identified as suitable for huma…