Study robustness of global feature effect explanations in machine learning models.
problem Vulnerability of global feature effect explanations to data and model perturbations.
method Theoretical bounds and experimental evaluation of partial dependence plots and accumulated local effects.
result Quantifies the gap between best and worst-case scenarios of misinterpreting machine learning predictions globally.
GADGET framework decomposes global feature effects using recursive partitioning.
problem Misleading global feature effects when feature interactions are present.
method Generalized additive decomposition of global effects (GADGET) based on recursive partitioning.
result Minimizes interaction-related heterogeneity of local feature effects.
Study compares and contrasts various ML explanation methods, highlighting their disagreements and similarities.
problem Understanding and quantifying the differences between various machine learning explanation methods.
method Synthesized and visualized various explanation methods for global and local aspects of ML models.
result There is substantial agreement on the top features but less on specific rankings, and tree interpreter is comparable to SHAP for feature effects.
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Proposes φ-table for statistical SHAP explanations in regression models.
problem Lack of clear directional summaries, uncertainty, and fidelity in SHAP feature importance.
method SHAP importance selection, fitting a standardized linear surrogate, reporting coefficients, uncertainty, fidelity, and stability.
result Extends SHAP into a statistical global explanation with direction, uncertainty, fidelity, and stability.
A barrier to the wider adoption of neural networks is their lack of interpretability. While local explanation methods exist for one prediction, most global attributions still reduce neural network decisions to a single set of features. In response, we present an approach for generating global attributions called GAM, w…
Unified local and global explanations through functional decomposition of low dimensional structures.
problem Tackles the challenge of extracting meaningful local and global explanations from machine learning models.
method Proposes a new identification constraint to decompose the global representation into main and interaction components of arbitrary order.
result Unified local and global explanations by connecting partial dependence plots and interventional SHAP values.
Develops transparent global models consistent with local explanations.
problem Creating globally interpretable models that align with local explanations from black-box models.
method Custom boolean features from sparse local contrastive explanations are used to train a globally transparent model.
result Custom transparent models have higher local consistency compared to other strategies.
Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-bas…
GLOBE-CE offers efficient global counterfactual explanations.
problem Lack of reliable and scalable global counterfactual explanations.
method Translation-based approach for global counterfactual explanations.
result GLOBE-CE performs significantly better than current methods across multiple metrics.
New method R-LOCO improves local feature importance analysis.
problem Local attribution methods fail to accurately identify important features.
method R-LOCO segments input space into regions and applies global methods within.
result R-LOCO delivers more accurate local attributions.
Defines globalness measure for explainers using optimal transport.
problem Challenges in evaluating and comparing explainability methods.
method Axiomatic definition and proof of Wasserstein Globalness measure.
result Wasserstein Globalness measure facilitates meaningful comparison and selection of explainers.
Model interpretability is an increasingly important component of practical machine learning. Some of the most common forms of interpretability systems are example-based, local, and global explanations. One of the main challenges in interpretability is designing explanation systems that can capture aspects of each of th…
As machine learning becomes more pervasive, there is an urgent need for interpretable explanations of predictive models. Prior work has developed effective methods for visualizing global model behavior, as well as generating local (instance-specific) explanations. However, relatively little work has addressed regional …
BIF assesses feature importance using Dirichlet distribution and Bayesian inference.
problem Quantitative feature importance assessment in statistical models.
method Utilizes Dirichlet distribution for probabilistic feature importance assessment via approximate Bayesian inference.
result Learned importance provides relative significance and confidence quantification of features.
Introduces joint Shapley values to measure feature importance in models.
problem Measuring the importance of feature sets in machine learning models.
method Extends Shapley's axioms to measure a set of features' average contribution to a model's prediction.
result Joint Shapley values provide unique insights and are more consistent with local intuitions.
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based e…
Gradient-based methods improve understanding of deep learning survival models.
problem Limited interpretability of deep learning survival models hinders their adoption.
method Gradient-based explanation methods tailored to survival neural networks.
result Gradient-based methods capture feature effects and temporal dynamics.
Dual explanation method using convex hulls and example-based vectors.
problem Local and global explanation of complex models.
method Dual representation of instances as convex combinations, generating new dual dataset, training linear surrogate model, computing feature importance.
result Effective example-based and local/global explanation of complex models.
Unified framework for feature-based explanations using ANOVA and game theory.
problem Differences between feature-based explanations methods limit their applicability.
method Introduces a unified framework combining fANOVA and cooperative game theory.
result Uncovered similarities and differences between various explanation techniques.
New method quantifies intrinsic causal contributions in neural networks.
problem Measuring the causal influence of input features in deep neural networks.
method Proposes an identifiable generative post-hoc framework to quantify intrinsic causal contributions (ICC) as structural causal models.
result ICC generates more intuitive and reliable explanations compared to existing global explanation techniques.
MACQ method explains deep learning models by analyzing feature contributions across prediction levels.
problem Explaining deep learning model predictions.
method Global gradient-based, model-agnostic approach focusing on marginal attribution.
result MACQ separates feature contributions from interaction effects and visualizes 3-way relationships.
COMRECGC finds common recourse for global counterfactual explanations in GNNs.
problem Finding common recourse for global counterfactual explanations in GNNs.
method Formalized the common recourse explanation problem and designed COMRECGC algorithm.
result COMRECGC outperforms strong baselines on four real-world graph datasets.
New methods compare local and global feature importance scores for bioinformatics models.
problem Improving feature importance estimation in tree-based models.
method Comparison of SHAP values and Conditional Feature Contributions (CFCs) for 164 bioinformatics problems.
result SHAP values and CFCs yield similar rankings and interpretations for random forests.
MBExplainer provides explanations for models combining graph embeddings and tabular features.
problem Explaining models using a mix of graph embeddings and tabular features.
method Model-agnostic approach using Shapley values and Monte Carlo Tree Search.
result MBExplainer efficiently finds human-readable explanations for model predictions.
The paper develops statistical inference methods for SHAP values.
problem Lack of statistical inference for SHAP values in model-agnostic feature importance.
method Semi-parametric approach using U-statistics and Neyman orthogonal scores for functionals of nested regressions.
result Asymptotically normal estimates of the pth powers of SHAP values for various p.
Theoretical analysis of vision transformers' performance with MAE and CL objectives.
problem Understanding the distinct representations learned by vision transformers with MAE and CL objectives.
method Modeling visual data distribution and analyzing ViTs training dynamics with gradient descent.
result ViTs trained with MAE objectives learn both global and local features, while CL-trained ViTs favor global features.
Improves global counterfactual explanations for model recourse.
problem Inability to provide explanations beyond local instances.
method Investigates and improves Actionable Recourse Summaries (AReS) for global counterfactual explanations.
result Develops more efficient and interactive explainability tools.
Current approaches for explaining machine learning models fall into two distinct classes: antecedent event influence and value attribution. The former leverages training instances to describe how much influence a training point exerts on a test point, while the latter attempts to attribute value to the features most pe…
As the application of deep neural networks proliferates in numerous areas such as medical imaging, video surveillance, and self driving cars, the need for explaining the decisions of these models has become a hot research topic, both at the global and local level. Locally, most explanation methods have focused on ident…
Proposes a new method to explain complex machine learning models.
problem Lack of joint feature effects in current explanation algorithms.
method Axiomatization of the Banzhaf index for capturing feature subsets.
result The generalized Banzhaf index optimally approximates black-box models.
Proposes a method for multilevel explanations of black-box models.
problem Need for explanations at intermediate or group levels, especially for GDPR compliance.
method Meta-method that builds a multilevel explanation tree using local explainability methods.
result Effective multilevel explanations for groups of data points, including novel test points.
GraphLIME explains GNN models by selecting key features locally.
problem Explaining the effectiveness of GNN models is challenging due to complex nonlinear transformations.
method GraphLIME uses HSIC Lasso for nonlinear feature selection in GNN models.
result GraphLIME provides more descriptive explanations than existing methods.
Enhanced Extended Isolation Forest (EIF+) improves anomaly detection and provides interpretable explanations.
problem Detecting anomalies in complex datasets and explaining model predictions.
method Extended Isolation Forest (EIF) and Extended Isolation Forest Feature Importance (ExIFFI) methods.
result EIF+ outperforms EIF in detecting unseen anomalies and provides better generalization.
Proposes ICE-based metric for better understanding interactions in black-box models.
problem Misleading global sensitivity metrics in black-box models due to interaction effects.
method Individual Conditional Expectation (ICE) curves to compute feature importance and interactions.
result ICE-based metric provides richer insights into feature importance and interactions.
Proposes an efficient method for ordered counterfactual explanations.
problem Insufficient explanation of perturbation vectors for executing actions.
method Mixed-Integer Linear Optimization (MILP) approach for evaluating and extracting optimal pairs of actions and orders.
result Demonstrated effectiveness of the proposed method on real datasets.
XGL uses global explanations to guide human supervision in machine learning.
problem Improving model quality through human-machine interaction.
method XGL employs global explanations to guide human selection of informative examples.
result XGL avoids overselling the model's quality and performs comparably to other strategies.
Understanding the behavior of a trained network and finding explanations for its outputs is important for improving the network's performance and generalization ability, and for ensuring trust in automated systems. Several approaches have previously been proposed to identify and visualize the most important features by…
The paper explores how Shapley value for a feature can vary based on model outcomes and feature distribution.
problem The uniqueness of Shapley value in explaining model predictions.
method Analyzes the relationship between feature distribution and Shapley value, and compares Shapley values for different model outcomes.
result Shapley value for a feature depends on more than just its mean and can vary significantly based on model outcome.
We propose a possible solution to a public challenge posed by the Fair Isaac Corporation (FICO), which is to provide an explainable model for credit risk assessment. Rather than present a black box model and explain it afterwards, we provide a globally interpretable model that is as accurate as other neural networks. O…
The paper proposes criteria and methods for evaluating and aggregating feature-based model explanations.
problem Lack of quantitative evaluation criteria for feature-based model explanations.
method Developed quantitative evaluation criteria (low sensitivity, high faithfulness, low complexity), devised a framework for aggregation, and derived a new aggregate Shapley value explanation function.
result A new aggregate Shapley value explanation function that minimizes sensitivity.
New measure SEV shows non-sparse models can still have low decision sparsity.
problem Non-sparse models can still make accurate decisions based on a few features.
method Introduced Sparse Explanation Value (SEV) to measure decision sparsity, not overall model sparsity.
result Many non-sparse models have low decision sparsity, as measured by SEV.
Model explanations based on pure observational data cannot compute the effects of features reliably, due to their inability to estimate how each factor alteration could affect the rest. We argue that explanations should be based on the causal model of the data and the derived intervened causal models, that represent th…
Simple aggregation of multiple methods defends against adversarial attacks on neural networks.
problem Manipulation of machine learning models to hide discriminating features.
method Aggregation of multiple explanation methods to make the model robust against adversarial attacks.
result The model remains robust even when the attacker has knowledge of the model weights and explanation methods.
CTE improves explanation estimation with less data and faster computation.
problem Inefficient and inaccurate explanation estimation in machine learning models.
method Distribution compression through kernel thinning to reduce sample size.
result CTE significantly improves accuracy and stability of explanation estimation.
TREX explains tree ensembles by identifying key training examples.
problem Identifying which training examples most influence tree ensemble predictions.
method TREX builds a surrogate model using a kernel that captures tree ensemble structure, approximating the original model.
result TREX provides accurate and effective explanations for tree ensembles.
Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models. We propose Mahé, a novel approach to provide Model-agnostic hierarchical éxplanations of how powerful machine learning models, such as deep ne…