Model-agnostic interpretation methods can mislead if not used carefully.
problem Misinterpretation of machine learning models due to improper use of techniques.
method General pitfalls of model-agnostic interpretation methods.
result Many pitfalls exist when using global interpretation techniques for machine learning models.
Tutorial on interpreting SPECT images for PD recognition using AI.
problem Difficulty in interpreting complex DL models for clinical use.
method Evaluation of six interpretation methods on four DCNN architectures.
result Guided backpropagation and SHAP methods are suitable for PD recognition.
Certifiable defense method improves robustness of deep learning interpretation.
problem Vulnerability of gradient-based saliency maps to adversarial attacks.
method Sparsified SmoothGrad method, extending certifiably robust smooth classifier bounds.
result Sparsified SmoothGrad method is certifiably robust against adversarial perturbations.
New method learns to encode predictions within interpretations, improving evaluation.
problem Need for interpretable machine learning, but existing methods are slow or lack fidelity.
method Amortized explanation methods that learn a global selector model optimizing fidelity of interpretations.
result Predictions can be encoded within interpretations, detected by EVAL-X.
Framework evaluates post-hoc interpretability methods in time-series classification.
problem Lack of suitable post-hoc interpretability methods for time-series classification.
method Proposes a framework with quantitative metrics to assess interpretability methods.
result Addresses several drawbacks of existing methods, including dependence on human judgement and data distribution shift.
New framework improves interpretability of trainable prompts.
problem Improving task-specific LLM performance with soft prompts remains a black-box method.
method Developed a theoretical framework for evaluating interpretability of trainable prompts, inspired new objective functions.
result Found a fundamental trade-off between interpretability and task performance in trainable prompts.
RCAV quantifies model sensitivity to semantic concepts, improving interpretability methods.
problem Lack of semantic interpretability in image classification models.
method RCAV calculates concept gradients and ascent steps to assess model sensitivity to semantic concepts.
result RCAV yields more accurate and robust interpretations of model behavior.
We propose to use boosted regression trees as a way to compute human-interpretable solutions to reinforcement learning problems. Boosting combines several regression trees to improve their accuracy without significantly reducing their inherent interpretability. Prior work has focused independently on reinforcement lear…
Adversarial manipulation fools neural network interpretations.
problem Ensuring interpretation methods are robust to model manipulations.
method Integrating manipulation into fine-tuning objective function.
result State-of-the-art interpreters can be fooled with minimal accuracy loss.
New method improves neural network interpretability against adversarial attacks.
problem Adversarial attacks can hide from neural network interpretability methods.
method Develops an interpretability-aware defensive scheme promoting robust interpretation.
result Achieves both robust classification and robust interpretation.
Study finds machine learning interpretations are often unstable and unreliable.
problem Reliability of machine learning interpretations in high-stakes domains.
method Stability study on global interpretations using tabular data.
result Popular interpretation methods are frequently unstable, less stable than predictions, and not associated with prediction accuracy.
Paper proposes a new method to evaluate AI model interpretability in bond default prediction.
problem Lack of standardized method to assess inherent interpretability of AI models.
method Uses LIME and SHAP to assess feature contributions in bond default prediction.
result Consistent results with intuitive understanding of model interpretability.
A fast method finds interpretable counterfactual explanations using class prototypes.
problem Finding understandable counterfactual explanations for classifier predictions.
method Using class prototypes, the method speeds up and improves interpretability of counterfactual instances.
result The method significantly speeds up and improves the interpretability of counterfactual explanations.
LIBRE learns interpretable Boolean rules from data.
problem Creating interpretable classifiers in imbalanced settings.
method Ensemble of weak learners on random feature subsets, combined with a simple union.
result Efficiently balances prediction accuracy and interpretability.
The paper introduces metrics to objectively evaluate interpretability methods.
problem Lack of objective evaluation metrics for interpretability methods.
method Proposes a set of metrics to evaluate interpretability methods along simplicity and broadness.
result Validated metrics on different benchmark tasks and showed their utility in method selection.
New method makes CNN interpretations robust to adversarial attacks.
problem Adversarial attacks on CNN interpretation maps.
method Renyi Differential Privacy (RDP) for robust interpretation.
result Certifiable top-k robustness and improved experimental robustness. New method quantifies model complexity for better interpretation.
problem Complex models produce misleading interpretation results.
method Functional decomposition to quantify model complexity.
result Post-hoc interpretation of complex models is more reliable and compact.
Machine-learning models have demonstrated great success in learning complex patterns that enable them to make predictions about unobserved data. In addition to using models for prediction, the ability to interpret what a model has learned is receiving an increasing amount of attention. However, this increased focus has…
The paper investigates instance-based interpretation methods for VAEs.
problem Understanding how VAEs make predictions without supervision.
method Formally framed influence functions for VAEs and developed VAE-TracIn.
result Varying training samples significantly impacts VAE predictions.
WISCA generates consensus explanations from conflicting model-agnostic interpretability methods.
problem Conflicting explanations from diverse interpretability algorithms.
method WISCA integrates class probability and normalized attributions to generate consistent explanations.
result WISCA consistently aligns with the most reliable individual method, improving explanation reliability.
QUACKIE creates a new benchmark for NLP interpretability.
problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.
VIBI interprets black-box systems by selecting key features that are both brief and comprehensive.
problem Lack of concise and comprehensive explanations for black-box decision systems.
method VIBI uses the information bottleneck principle to select key features that are maximally compressed and informative.
result VIBI provides more concise and comprehensive explanations compared to existing methods.
New framework learns interpretable rule ensembles without sacrificing accuracy.
problem Trade-off between accuracy and interpretability in rule ensembles.
method Introduces local interpretability and a regularizer to promote it, using coordinate descent with local search.
result Learns rule ensembles with fewer rules to explain individual predictions, maintaining comparable accuracy.
Decisions by Machine Learning (ML) models have become ubiquitous. Trusting these decisions requires understanding how algorithms take them. Hence interpretability methods for ML are an active focus of research. A central problem in this context is that both the quality of interpretability methods as well as trust in ML…
Interpretation of a machine learning induced models is critical for feature engineering, debugging, and, arguably, compliance. Yet, best of breed machine learning models tend to be very complex. This paper presents a method for model interpretation which has the main benefit that the simple interpretations it provides …
Brief history and challenges of interpretable machine learning.
problem Challenges in interpreting machine learning models, especially in scientific applications.
method Overview of state-of-the-art methods and discussion of challenges.
result Interpretable machine learning has a rich history but faces significant challenges.
Interpretable ML models predict recidivism as well as non-interpretable methods and are more fair.
problem Improving recidivism prediction models for fairness and interpretability.
method Trained interpretable ML models on two recidivism datasets, compared to existing methods, and analyzed fairness.
result Interpretable ML models can predict recidivism as well as non-interpretable methods and are more fair.
I-MAD detects malware with high accuracy and interpretable results.
problem Detecting new malware samples and providing interpretable results.
method Galaxy Transformer network and interpretable feed-forward neural network.
result Significantly outperforms existing static malware detection models.
New methods interpret clustering outcomes without altering data structure.
problem Post-processing methods destroy data integrity and obscure interpretations.
method Algorithm-agnostic interpretation methods using permutation feature importance, individual conditional expectation, and partial dependence.
result Preserves original feature structure and explains clustering outcomes.
Study compares imputation methods' effects on IML confidence intervals.
problem Missing data impacts IML interpretation and confidence intervals.
method Compared single vs multiple imputation methods on IML confidence intervals.
result Multiple imputation provides closer coverage to nominal than single imputation.
Improved interpretability methods for ML models using local regressions and variable importance.
problem Inability of existing interpretability methods to provide reliable explanations for ML models, especially in high-dimensional problems with irrelevant features and non-linear relationships.
method Introduces VarImp and SupClus methods using local regressions with weighted distance considering variable importance.
result VarImp and SupClus methods yield better explanations than state-of-the-art approaches, especially in high-dimensional problems with irrelevant features and non-linear relationships.
New framework aims to make neural network explanations more reliable.
problem Current interpretability methods rely on intuition and lack falsifiability.
method Proposes a framework for strongly falsifiable interpretability research.
result Falsifiable interpretability methods can generate meaningful advances in understanding DNNs.
New methods improve feature extraction and representation quality in supervised and unsupervised DR.
problem Statistical dependence, data diversity, contrast, and interpretability in conventional DR methods.
method Combines linear and nonlinear formulations for three new independence criteria.
result Significant improvements in contrast, accuracy, and interpretability over baselines.
In recent years, machine learning researchers have focused on methods to construct flexible and interpretable prediction models. However, an interpretability evaluation, a relationship between generalization performance and an interpretability of the model and a method for improving the interpretability have to be cons…
NAMLSS models provide interpretable neural regression for location, scale, and shape.
problem Lack of interpretability in deep learning models for complex data distributions.
method Combines classical statistical methods with DNNs for distributional regression.
result Achieves visual interpretability and predictive power of deep learning models.
Adaptive spectral RL method enhances RL performance and interpretability.
problem Balancing interpretability and performance in reinforcement learning.
method Spectral based linear RL approach with adaptive regularization.
result Near-optimal bounds for parameter estimation and generalization error.
TSInsight improves interpretability of deep time-series models.
problem Lack of interpretability methods for time-series data.
method Attach auto-encoder to classifier with sparsity-inducing norm, fine-tune based on gradients and reconstruction penalty.
result TSInsight effectively boosts interpretability of deep time-series models.
New method compares feature importance and rule extraction for text data interpretability.
problem Unexpected differences in explanations from feature importance and rule extraction methods.
method Proposes a new approach to compare explanations from different methods.
result Different methods can lead to unexpected explanations, even for simple models.
Interpreting machine learning models helps understand adversarial attacks and defenses.
problem Understanding model vulnerability to adversarial attacks.
method Model interpretation techniques to explore adversarial attacks and defenses.
result Interpretation methods can be applied to adversarial attacks and defenses.
This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervi…
New method tests black box models for important features.
problem Control false discoveries in high-stakes model interpretations.
method Reframe interpretability as hypothesis testing, propose two testing methods.
result Tests control false discovery rate and select intuitive features.
Proposes a deep learning model for probabilistic forecasting that is also interpretable.
problem Inability to explain predictions of neural network-based time series forecasting methods.
method Deep Autoregressive Networks (DANLIP) for locally interpretable probabilistic forecasting.
result DANLIP provides interpretable predictions with comparable performance to state-of-the-art methods.
New rule-based method for classification with scalability, interpretability, and fairness.
problem Developing a scalable and fair classification method.
method Column generation for linear programming, decision tree-based heuristic, and rule-based optimization.
result The method returns interpretable rules with optimal weights and addresses fairness constraints.
New method visualizes tabular feature semantics for better model understanding.
problem Lack of feature interaction interpretation in tabular ML models.
method Feature Vectors method for global tabular dataset interpretability.
result Visualizes semantic relationships among tabular features.
The paper proposes a method for interpretable mixture density estimation using a tree structure.
problem Complex probability distributions in machine learning models.
method Interpretable tree structure for mixture density estimation with fast inference.
result The method achieves both high speed and interpretability for mixture density estimation.
LI-ITR combines flexible ML with interpretable approximations for personalized treatment rules.
problem Combining flexibility and interpretability in personalized treatment rules.
method Uses variational autoencoders and a mixture of interpretable experts.
result Accurately recovers true local coefficients and optimal treatment strategies.
Study evaluates local explanation methods for time series forecasting.
problem Lack of local interpretability methods for multivariate time series forecasting.
method Proposed two novel evaluation metrics: Area Over the Perturbation Curve for Regression and Ablation Percentage Threshold.
result Comprehensive comparison of local explanation models on two datasets.
We argue that robustness of explanations---i.e., that similar inputs should give rise to similar explanations---is a key desideratum for interpretability. We introduce metrics to quantify robustness and demonstrate that current methods do not perform well according to these metrics. Finally, we propose ways that robust…