ViTaX provides formal guarantees for targeted explanations in safety-critical systems.
problem Need trustworthy explanations for safety-critical deep neural networks.
method Formal reachability analysis for targeted, semifactual explanations.
result First method to provide formally guaranteed explanations of model resilience.
Paper introduces algorithms for explaining monotonic classifiers.
problem Need for explanations of monotonic classifiers.
method Polynomial algorithms for formal explanations of monotonic classifiers.
result Efficient model-agnostic algorithm for enumerating explanations.
Paper defines XAI concepts using category theory.
problem Lack of precise mathematical definitions for XAI.
method Uses Category theory to define XAI concepts rigorously.
result Establishes a theoretical foundation for XAI.
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
We regard explanations as a blending of the input sample and the model's output and offer a few definitions that capture various desired properties of the function that generates these explanations. We study the links between these properties and between explanation-generating functions and intermediate representations…
Interpretability is an elusive but highly sought-after characteristic of modern machine learning methods. Recent work has focused on interpretability via explanations, which justify individual model predictions. In this work, we take a step towards reconciling machine explanations with those that humans prod…
G-SHAP generates multiple types of explanations for machine learning models.
problem Understanding model predictions and their differences across groups.
method Generalization of SHAP method to produce additional types of explanations.
result G-SHAP produces explanations for classification, intergroup differences, and model failure.
XAI methods fail to explain ML models reliably.
problem Current XAI methods fail to provide reliable explanations for ML models.
method Formally define problems and design methods accordingly.
result Diverse notions of explanation correctness and metrics needed.
We define and compute plausible counterfactual explanations using density constraints.
problem Efficiently compute plausible counterfactual explanations for machine learning models.
method Propose and study a formal definition of plausible counterfactual explanations, use density estimators, and introduce convex density constraints.
result Convex density constraints ensure plausible and feasible counterfactual explanations.
The paper explores methods to explain decisions of deep learning models by faithfully reproducing their training data views.
problem Explaining decisions of complex deep learning models trained on large datasets.
method Data view extraction through hill-climbing and GAN-driven approaches, followed by creation of shadow models for explanation.
result Shadow models based on faithfully reproduced data views are effective for explaining decisions of blackbox deep learning models.
ID-ExpO fine-tunes neural networks for more faithful explanations.
problem Improving the faithfulness of explanations for complex machine learning models.
method Differentiable insertion/deletion metric-aware regularizers for optimization.
result Fine-tuned predictors produce more faithful explanations.
The paper introduces a method to learn models with built-in explanations.
problem Lack of interpretability in deep learning models.
method Formalizes learning with explanation constraints and provides a learning theoretic framework.
result Models that satisfy these constraints have reduced Rademacher complexities, improving their performance.
Paper bridges generative models and explainability.
problem Lack of connection between generative models and explainability.
method Proposes a probabilistic framework for example-based explanations.
result Formally defines example-based explanations for deep generative models.
LLMs' explanations are often insufficient and vary with input distribution.
problem Evaluating the sufficiency of LLM explanations without predefined biases.
method Generalizing sufficiency to arbitrary explanations, using LLM's input beliefs, and introducing SCSuff metric.
result Explanation sufficiency can vary with input distribution and is weakly correlated with model size, accuracy, or output entropy.
Recourse explanations can become invalid if collective actions change statistical data.
problem Recourse explanations may become invalid due to collective behavior changing data statistics.
method Formal characterization of conditions under which recourse explanations remain valid under performativity.
result Recourse actions may become invalid if they are influenced by or intervene on non-causal variables.
COMRECGC finds common recourse for global counterfactual explanations in GNNs.
problem Finding common recourse for global counterfactual explanations in GNNs.
method Formalized the common recourse explanation problem and designed COMRECGC algorithm.
result COMRECGC outperforms strong baselines on four real-world graph datasets.
Formulates approach for guiding explanation types based on user specifications.
problem Creating explainable AI components from user-defined specifications.
method Develops a method for generating explanations based on user-defined specifications.
result Demonstrates feasibility of user-defined explanations for complex models like Bayesian networks and graph neural networks.
New metric assesses reliability of AI explanations.
problem Unreliable AI explanations under realistic conditions.
method Explanation Reliability Index (ERI) metrics quantifying stability under four axioms.
result Widespread reliability failures in popular explanation methods.
Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.
problem CNNs lack accurate data representation in their built-in prototypes.
method Introduces k-means-based post-hoc explanations for CNNs, leveraging spatial consistency of convolutional receptive fields.
result Using shallower, less compressed feature activations improves semantic fidelity at the cost of slight predictive performance.
Interpretable Machine Learning (IML) has become increasingly important in many real-world applications, such as autonomous cars and medical diagnosis, where explanations are significantly preferred to help people better understand how machine learning systems work and further enhance their trust towards systems. Howeve…
LLMs can help explain credit risk models but not autonomously.
problem Leveraging LLMs for post-hoc explainability in credit risk models.
method Comparison of LLM outputs with SHAP and coefficient-based attributions on three LMs.
result LLMs reliably preserve feature-importance rankings but poorly align with autonomous explanations.
Interpretable ML methods for better decision-making with explanations.
problem Lack of transparency in black-box ML models.
method Use of Formal Concept Analysis and cooperative game theory to assess attribute importance and reduce attribute count.
result Developed methods to assess attribute importance and reduce attribute count in ML models.
Paper introduces a method to explain concept drift using counterfactual explanations.
problem Understanding the features where concept drift occurs for better model adjustment.
method Formal definition and algorithm based on counterfactual explanations.
result Demonstrates usefulness of the method in various examples.
IRDs provide local, model-agnostic explanations using hyperboxes.
problem Local model-agnostic explanations for machine learning predictions.
method Formalizes IRDs as hyperboxes, defines optimization problem, introduces unified framework.
result IRDs offer semi-factual explanations and highlight feature importance.
Proposes a new method to explain complex machine learning models.
problem Lack of joint feature effects in current explanation algorithms.
method Axiomatization of the Banzhaf index for capturing feature subsets.
result The generalized Banzhaf index optimally approximates black-box models.
Neural networks memorize exceptions, leading to poor generalization.
problem Memorization of exceptions hinders neural network generalization.
method Formalized memorization-generalization interplay, proposed MAT to shift logits.
result MAT improves generalization by learning robust patterns invariant across distributions.
A new method improves LIME for better model explanation.
problem Current LIME explanations are not faithful and weak in understanding.
method Proposes a novel Modified Perturbed Sampling (MPS) for LIME.
result MPS-LIME achieves better performance in understandability, fidelity, and efficiency.
As the application of deep neural networks proliferates in numerous areas such as medical imaging, video surveillance, and self driving cars, the need for explaining the decisions of these models has become a hot research topic, both at the global and local level. Locally, most explanation methods have focused on ident…
Much work aims to explain a model's prediction on a static input. We consider explanations in a temporal setting where a stateful dynamical model produces a sequence of risk estimates given an input at each time step. When the estimated risk increases, the goal of the explanation is to attribute the increase to a few r…
The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the model handles semantic compositions, we study hierarchical explanation of neural network predictions. We identify non-additivity and contex…
New methods identify concepts in trained embeddings reliably without human labels.
problem Identifying interpretable concepts in trained embedding spaces without human labels.
method Explicitly connecting concept discovery to PCA and ICA, proposing novel approaches for dependent concepts.
result Proven methods outperform competitors on a variety of experiments, achieving up to 29% better alignment with ground truth.
The paper shows averaging gradients leads to memorization, proposing an alternative algorithm to focus on invariances.
problem The principle that 'good explanations are hard to vary' in deep learning is investigated.
method Formalizing consistency for loss surface minima, proposing an alternative algorithm based on logical AND.
result The alternative algorithm prevents memorization and focuses on invariances.
Artificial Intelligence (AI) has become an integral part of domains such as security, finance, healthcare, medicine, and criminal justice. Explaining the decisions of AI systems in human terms is a key challenge--due to the high complexity of the model, as well as the potential implications on human interests, rights, …
Unified feature importance for machine learning models tackles sufficiency and necessity limitations.
problem Insufficient and incomplete explanations of machine learning models.
method Formalized sufficiency and necessity notions, proposing a unified importance measure.
result Unified importance measure detects features missed by sufficiency and necessity alone.
Automated decision making is used routinely throughout our everyday life. Recommender systems decide which jobs, movies, or other user profiles might be interesting to us. Spell checkers help us to make good use of language. Fraud detection systems decide if a credit card transactions should be verified more closely. M…
Predictive models are being increasingly used to support consequential decision making at the individual level in contexts such as pretrial bail and loan approval. As a result, there is increasing social and legal pressure to provide explanations that help the affected individuals not only to understand why a predictio…
New findings show local attributions can't be both robust and provide recourse.
problem Ensuring machine learning systems are accountable and provide actionable recourse options.
method Formal definition of recourse sensitivity and counterexamples for popular attribution methods.
result It is impossible for any single attribution method to be both robust and provide recourse.
Unified framework explains few-shot multimodal medical imaging performance.
problem Limited labeled data in rare diseases and low-resource settings.
method PAC learning, VC theory, PAC Bayesian analysis, information gain, Chain of Thought reasoning.
result Unified theoretical framework for few-shot multimodal medical imaging.
We identify action representations from video data, proving their statistical benefits.
problem Identifying latent action policies from video data.
method Entropy-regularized LAPO objective, formalizing desiderata for action representations.
result Entropy-regularized LAPO identifies action representations satisfying desiderata under suitable conditions.
Studies geometric mechanics for autonomous and nonautonomous systems.
problem Understanding the geometric basis of mechanics.
method Geometric descriptions, Lagrangian, Hamiltonian, unified formalisms, symmetries, variational principles.
result Characterization of dynamical systems' properties and characteristics.
In this paper, we present a new explainability formalism designed to shed light on how each input variable of a test set impacts the predictions of machine learning models. Hence, we propose a group explainability formalism for trained machine learning decision rules, based on their response to the variability of the i…
Survey reviews explainability in AI for healthcare, emphasizing trust and transparency.
problem Lack of transparency hinders AI adoption in healthcare.
method Comprehensive literature review to guide explainable AI design.
result Quantitative evaluation metrics are needed for some explainability properties.
A new framework for robot block-stacking tasks using causal probabilistic models.
problem Robots fail outside controlled environments due to uncertainty and lack of explicit design for all scenarios.
method Causal probabilistic framework combining causal models and probabilistic representations of noise.
result Robots can perceive, reason about, and explain their environment for block-stacking tasks.
I show that the solution of a standard clearing model commonly used in contagion analyses for financial systems can be expressed as a specific form of a generalized Katz centrality measure under conditions that correspond to a system-wide shock. This result provides a formal explanation for earlier empirical results wh…
We study stratified G-structures in N=2 compactifications of M-theory on eight-manifolds M using the uplift to the auxiliary nine-manifold M^=M×S1. We show that the cosmooth generalized distribution D^ on M^ which arises in this formalism may have pointwise transverse or…
Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based e…
Gradient Starvation causes neural networks to focus on a subset of features, leading to imbalance.
problem Neural networks focusing on only a subset of features despite having access to more.
method Using Dynamical Systems theory, identified properties of learning dynamics leading to feature imbalance.
result Gradient descent can lead to feature imbalance in over-parameterized networks.
New framework formalizes RLHF trilemma: improving safety, fairness, and robustness is computationally infeasible.
problem Aligning large language models with diverse human values while maintaining computational feasibility and robustness.
method Complexity-theoretic analysis integrating statistical learning theory and robust optimization.
result Achieving both representativeness (epsilon <= 0.01) and robustness (delta <= 0.001) for global-scale populations requires super-polynomial operations.