Proposes a method for explaining tabular data using copulas.
problem Lack of ground truth for explainability in complex datasets.
method Uses copulas to specify statistical properties and build intuition.
result Demonstrates improved explainability on logistic regression and correlation use cases.
Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.
problem CNNs lack accurate data representation in their built-in prototypes.
method Introduces k-means-based post-hoc explanations for CNNs, leveraging spatial consistency of convolutional receptive fields.
result Using shallower, less compressed feature activations improves semantic fidelity at the cost of slight predictive performance.
LLMs can help explain credit risk models but not autonomously.
problem Leveraging LLMs for post-hoc explainability in credit risk models.
method Comparison of LLM outputs with SHAP and coefficient-based attributions on three LMs.
result LLMs reliably preserve feature-importance rankings but poorly align with autonomous explanations.
FCDD improves image anomaly detection without post-hoc explainers.
problem Image anomaly detection, especially pixel-wise.
method Fully Convolutional Data Description (FCDD) directly addresses anomaly detection without post-hoc methods.
result FCDD achieves state-of-the-art results on pixel-wise AD tasks.
XCM improves MTS classification with explainable deep learning.
problem Lack of explainable deep learning models for MTS classification.
method XCM is a compact CNN that extracts variable and timestamp information directly from input data.
result XCM outperforms state-of-the-art MTS classifiers on large and small datasets.
Framework evaluates post-hoc interpretability methods in time-series classification.
problem Lack of suitable post-hoc interpretability methods for time-series classification.
method Proposes a framework with quantitative metrics to assess interpretability methods.
result Addresses several drawbacks of existing methods, including dependence on human judgement and data distribution shift.
This review explores methods to explain deep neural networks and their applications.
problem Understanding the decision-making process of deep neural networks.
method Overview of interpretability methods, theoretical foundations, and comparative evaluations.
result Demonstrates the effectiveness of explainable AI in various applications.
PACE explains ViTs by modeling patch-level concept distributions, surpassing existing methods.
problem Lack of trustworthy post-hoc explanations for Vision Transformers (ViTs)
method Variational Bayesian explanation framework (PACE)
result PACE surpasses state-of-the-art methods in meeting desiderata for ViT explanations.
EAGLE improves reproducibility and stability of model explanations.
problem Creating reliable explanations for opaque machine learning models.
method Formulates perturbation selection as an information-theoretic active learning problem.
result EAGLE learns a linear surrogate model with feature importance scores and uncertainty estimates.
Study proposes explainable analytics for manufacturing process planning.
problem Improving data-driven decision-making in manufacturing.
method Combines process mining, machine learning, and XAI. Uses deep learning for prediction and Shapley values/ICE plots for explanations.
result Enhanced decision-making capabilities through local post-hoc explanations.
iKF method uncovers complex variable interactions for scientific discovery.
problem Limited interpretability of existing models in decision-making applications.
method Iterative Kings' Forests (iKF) method to uncover multi-order interactions.
result iKF provides strong interpretive power for explainable modeling.
Kolmogorov-Arnold Networks offer interpretable models for energy applications.
problem Lack of interpretability in modern machine learning methods for sensitive industries.
method Symbolic regression with Kolmogorov-Arnold Networks compared to traditional feedforward neural networks.
result Kolmogorov-Arnold Networks yield perfectly interpretable models and learn real, physical relations.
New ML algorithms improve model interpretability without sacrificing performance.
problem Lack of interpretability in complex machine learning models.
method Developed new algorithms based on fANOVA framework, including GAMI-Lin-T and GAMI-Net.
result GAMI-Lin-T and GAMI-Net perform comparably to EBM and better in interpretability.
Developed an explainable DRL model for financial portfolio management.
problem Inability of DRL agents to provide interpretable financial investment policies.
method Integrating PPO with feature importance techniques (SHAP, LIME) to enhance transparency.
result Ability to interpret DRL agent actions in prediction time.
Surrogate explainers of black-box machine learning predictions are of paramount importance in the field of eXplainable Artificial Intelligence since they can be applied to any type of data (images, text and tabular), are model-agnostic and are post-hoc (i.e., can be retrofitted). The Local Interpretable Model-agnostic …
The paper explores how model complexity affects OOD detection performance.
problem Ensuring reliability and safety of machine learning systems through OOD detection.
method Investigates the relationship between model capacity and OOD detection performance using empirical and theoretical analysis.
result The Double Descent phenomenon is observed in post-hoc OOD detection, indicating that overparameterization can enhance OOD detection.
This review explores XAI in finance, highlighting common techniques and areas needing improvement.
problem Balancing accuracy and transparency in financial AI models.
method Bibliometric and content analysis of XAI applications in finance.
result Post-hoc interpretability techniques are most used in financial XAI.
As machine learning black boxes are increasingly being deployed in domains such as healthcare and criminal justice, there is growing emphasis on building tools and techniques for explaining these black boxes in an interpretable manner. Such explanations are being leveraged by domain experts to diagnose systematic error…
Paper proposes hybrid approach for transparent credit scoring models.
problem Lack of transparency in machine learning models limits their use in regulated environments.
method Post-hoc interpretation of black-box models guides feature selection, followed by training glass-box models.
result Reduces feature usage from 106 to 10 while maintaining comparable performance.
Post-hoc interpretability approaches have been proven to be powerful tools to generate explanations for the predictions made by a trained black-box model. However, they create the risk of having explanations that are a result of some artifacts learned by the model instead of actual knowledge from the data. This paper f…
ID-ExpO fine-tunes neural networks for more faithful explanations.
problem Improving the faithfulness of explanations for complex machine learning models.
method Differentiable insertion/deletion metric-aware regularizers for optimization.
result Fine-tuned predictors produce more faithful explanations.
Meta-learning is a field that aims at discovering how different machine learning algorithms perform on a wide range of predictive tasks. Such knowledge speeds up the hyperparameter tuning or feature engineering. With the use of surrogate models various aspects of the predictive task such as meta-features, landmarker mo…
CUBE explains models by balanced experiments and contrasts.
problem Post-hoc explanation of trained predictive models.
method Design-based framework using balanced low-high probes.
result Reveals dominant learned effect structure and clarifies query efficiency.
Anchors explains text classifiers by highlighting key words.
problem Interpreting machine learning models, especially for text classifiers.
method Formalizes Anchors' algorithm and analyzes its behavior on linear text classifiers.
result Anchors produces meaningful results on linear text classifiers.
In this paper, we study two challenging problems in explainable AI (XAI) and data clustering. The first is how to directly design a neural network with inherent interpretability, rather than giving post-hoc explanations of a black-box model. The second is implementing discrete k-means with a differentiable neural net…
Systematic review of ML explainability in process mining.
problem Understanding the black-box nature of ML models in process mining.
method Systematic literature review using PRISMA framework.
result Identification of key trends and challenges in interpretability.
In the context of post-hoc interpretability, this paper addresses the task of explaining the prediction of a classifier, considering the case where no information is available, neither on the classifier itself, nor on the processed data (neither the training nor the test data). It proposes an instance-based approach wh…
Post-hoc transforms can reverse model performance trends, especially in noisy settings.
problem Post-hoc transforms can reverse model performance trends, especially in noisy settings.
method Empirical study and analysis of post-hoc transforms like temperature scaling, ensembling, and SWA.
result Post-hoc reversal can prevent double descent and mitigate mismatches between test loss and test error.
The paper compares ML models for credit scoring and investment decisions using explainable AI.
problem The opacity of machine learning models in financial services.
method Comparison of various machine learning models (single classifiers, ensembles, neural networks) and explainability techniques (LIME, SHAP).
result Ensemble classifiers and neural networks outperform in credit scoring models.
S-LIME stabilizes LIME for more reliable model explanations.
problem Instability of post hoc explanation methods like LIME.
method Uses hypothesis testing based on central limit theorem to stabilize explanations.
result Demonstrates effectiveness of S-LIME on simulated and real-world data.
Paper proposes a method to make image model explanations robust to distortions.
problem Ensuring robustness of explanations for images under distortions.
method Embedding perceptual distances in surrogate explainers to evaluate and improve robustness.
result Surrogate explanations become more coherent and robust to distortions.
Post-hoc calibration improves uncertainty under domain shift.
problem Improving uncertainty calibration under domain shift.
method Apply perturbations to validation set before post-hoc calibration.
result Perturbation step results in better calibration under domain shift.
Survey reviews explainability in AI for healthcare, emphasizing trust and transparency.
problem Lack of transparency hinders AI adoption in healthcare.
method Comprehensive literature review to guide explainable AI design.
result Quantitative evaluation metrics are needed for some explainability properties.
Survey interprets foundation models' inner workings.
problem Limited explainability of black-box FMs.
method Interpretable methods to reveal FMs' mechanisms.
result Identifies next research directions for FMs.
RAW-Explainer generates interpretable subgraph explanations for link predictions in knowledge graphs.
problem Interpreting GNN predictions for link prediction in heterogeneous settings is challenging.
method RAW-Explainer uses random walk objective and neural network to generate connected, concise subgraph explanations.
result RAW-Explainer strikes a balance between explanation quality and computational efficiency.
STOOD-X detects out-of-distribution samples without distributional assumptions and provides explainable visualizations.
problem Challenges in OOD detection, including restrictive assumptions, scalability issues, and lack of interpretability.
method Two-stage methodology combining statistical nonparametric test and explainability enhancements.
result Achieves competitive performance in high-dimensional and complex settings, with explainability framework enabling human oversight.
Proposes a new model for explainable recommendation systems.
problem Developing predictive models that provide explanations for item recommendations.
method Generalized Additive Models with Manifest and Latent Interactions (GAMMLI).
result Advantages in both predictive performance and explainability.
Explains agent behavior through intended outcomes in reinforcement learning.
problem Proving impossibility of general post-hoc explanations in reinforcement learning.
method Derives local explanations based on intention for Q-function approximations, proving consistency with learned Q-values.
result Demonstrates the necessity of collecting information during training for accurate explanations.
Anchors explain text model decisions by highlighting key words.
problem Explaining neural network decisions for text data.
method Formalized Anchors for text classification, analyzing different models.
result Neural network words with highest partial derivatives are selected by Anchors.
Mathematical study shows post-hoc explanations are better than attention weights alone.
problem Understanding the internal behavior of attention-based models.
method Mathematical analysis of a simple attention-based architecture.
result Post-hoc explanations provide more useful insights than attention weights alone.
Researchers show how to manipulate Partial Dependence plots to deceive explanations of predictive models.
problem The robustness and trustworthiness of Partial Dependence (PD) explanations are compromised.
method Data poisoning using genetic and gradient algorithms to manipulate PD plots.
result PD explanations can be fooled and manipulated to mislead understanding of predictive models.
Defines explainability as reasoning under background knowledge.
problem Lack of agreed definitions in explainable AI.
method Reviews philosophical and social foundations, translates to tech realm.
result Defines explainability as logical reasoning under background knowledge.
CalArena benchmarks post-hoc calibration methods across various tasks.
problem Inconsistent evaluations of post-hoc calibration methods.
method Large-scale benchmark with 2000 experiments, covering diverse models and settings.
result Smooth calibration functions outperform binning-based approaches.
Most existing interpretable methods explain a black-box model in a post-hoc manner, which uses simpler models or data analysis techniques to interpret the predictions after the model is learned. However, they (a) may derive contradictory explanations on the same predictions given different methods and data samples, and…
Meta-Cal improves post-hoc calibration of neural networks.
problem Improving the accuracy of uncalibrated neural network predictions.
method Meta-Cal uses a base calibrator and a ranking model with constraints to provide high-probability bounds.
result Meta-Cal significantly outperforms existing methods in post-hoc multi-class classification calibration.
The paper proposes new interpretability paradigms to improve model faithfulness.
problem Improving the accuracy of explanations for complex models.
method Examining and evolving existing paradigms, proposing new models.
result Three new paradigms for interpretability are presented.
Attack reveals model details from counterfactual explanations.
problem Extracting model details from counterfactual explanations.
method Adversary uses counterfactual explanations to build high-fidelity model.
result High-fidelity and high-accuracy model extraction possible.
This paper enhances credit risk management using explainable AI techniques.
problem Lack of transparency and explainability in AI models for credit risk management.
method Implement LIME and SHAP for explaining ML-based credit scoring models.
result Demonstrates practical challenges and solutions for XAI methods in finance.