Model interpretability is an increasingly important component of practical machine learning. Some of the most common forms of interpretability systems are example-based, local, and global explanations. One of the main challenges in interpretability is designing explanation systems that can capture aspects of each of th…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “explanation systems”
The ubiquity of systems using artificial intelligence or "AI" has brought increasing attention to how those systems should be regulated. The choice of how to regulate AI systems will require care. AI systems have the potential to synthesize large amounts of data, allowing for greater levels of personalization and preci…
The study evaluates how well local explanations align with model predictions.
problem Capturing the faithfulness of local explanations to model predictions.
method Introducing consistency and sufficiency as properties, and developing quantitative measures and estimators.
result Quantitative measures of consistency and sufficiency depend on test-time data distribution.
System allows users to critique explanations of recommendations.
problem Improving trust and perceived quality in recommendation systems.
method Personalized explanations generated from review texts, with a novel critiquing method.
result Users prefer explanations with critiques over those without.
New metric assesses reliability of AI explanations.
problem Unreliable AI explanations under realistic conditions.
method Explanation Reliability Index (ERI) metrics quantifying stability under four axioms.
result Widespread reliability failures in popular explanation methods.
In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image for which a vision system predicts class , a counterfactual visual explanation identifies how could change such that the system would output a different specified class . To do this, we select a 'dis…
Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We propose an alternative to these approaches by directly r…
New method provides calibrated feature importance explanations for regression models.
problem Lack of uncertainty quantification in existing local explanation methods.
method Extension of Calibrated Explanations method to support regression and probabilistic regression.
result Calibrated Explanations for regression provides quantified uncertainty and robust explanations.
Paper introduces Native Guide for generating time series counterfactual explanations.
problem Lack of explainability for time series data in AI systems.
method Model-agnostic, instance-based counterfactual generation for time series classification.
result Native Guide produces better counterfactual explanations than benchmarks.
We examine counterfactual explanations for explaining the decisions made by model-based AI systems. The counterfactual approach we consider defines an explanation as a set of the system's data inputs that causally drives the decision (i.e., changing the inputs in the set changes the decision) and is irreducible (i.e., …
Interpretable Machine Learning (IML) has become increasingly important in many real-world applications, such as autonomous cars and medical diagnosis, where explanations are significantly preferred to help people better understand how machine learning systems work and further enhance their trust towards systems. Howeve…
Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly human-interpretable remains poorly understood. This work advances our understanding of what makes explanations …
The need for transparency of predictive systems based on Machine Learning algorithms arises as a consequence of their ever-increasing proliferation in the industry. Whenever black-box algorithmic predictions influence human affairs, the inner workings of these algorithms should be scrutinised and their decisions explai…
Proposes a method for explaining ranking decisions in learning systems.
problem Limited work on interpreting ranking decisions from learning systems.
method Model agnostic local explanation method using optimization to maximize validity.
result Approach outperforms other methods in validity across different LTR models.
Interpretable explanations for recommender systems and other machine learning models are crucial to gain user trust. Prior works that have focused on paths connecting users and items in a heterogeneous network have several limitations, such as discovering relationships rather than true explanations, or disregarding oth…
We propose a general model explanation system (MES) for "explaining" the output of black box classifiers. This paper describes extensions to Turner (2015), which is referred to frequently in the text. We use the motivating example of a classifier trained to detect fraud in a credit card transaction history. The key asp…
GRETEL unifies GCE evaluation across various settings.
problem Lack of standardized evaluation for Graph Counterfactual Explanations.
method Unified framework for testing GCE methods in diverse settings.
result GRETEL promotes reproducible evaluations of GCE techniques.
CLEAR learns causal graphs from attention in recommender systems to explain user behavior.
problem Understanding why specific recommendations are made in recommender systems.
method CLEAR learns session-specific causal graphs from attention in pre-trained neural recommenders, addressing latent confounders.
result CLEAR provides counterfactual explanations that are shorter and more effective than naive methods.
Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, whose explanation quality can be unpredictable. Our method, ExpO, is a hybridization of these approaches that regu…
System detects financial misinformation and generates clear explanations.
problem Identifying and explaining fraudulent financial content.
method Combined large language models, pre-processing, and sequential learning.
result Achieved F1-score of 0.8283 for classification and ROUGE-1 of 0.7253 for explanations.
Explearn learns to explain predictions using Gaussian Processes.
problem Learning to explain predictions effectively.
method Gaussian Processes-based contextual bandits.
result Guaranteed convergence with high probability.
Deep learning is increasingly used as a building block of security systems. Unfortunately, neural networks are hard to interpret and typically opaque to the practitioner. The machine learning community has started to address this problem by developing methods for explaining the predictions of neural networks. While sev…
Symmetric observations don't necessarily imply symmetric causal explanations.
problem Inferring causal models from observed correlations is challenging and computationally intensive.
method An explicit example using a tripartite probability distribution over binary events.
result Symmetries in observations cannot be used to reduce the hypothesis space of causal models.
This paper reviews counterfactual explanations for machine learning models.
problem Making machine learning models understandable to humans.
method Categorization and evaluation of counterfactual explanation algorithms.
result A rubric for evaluating counterfactual explanations.
In recent years, a number of artificial intelligent services have been developed such as defect detection system or diagnosis system for customer services. Unfortunately, the core in these services is a black-box in which human cannot understand the underlying decision making logic, even though the inspection of the lo…
With the advent of GDPR, the domain of explainable AI and model interpretability has gained added impetus. Methods to extract and communicate visibility into decision-making models have become legal requirement. Two specific types of explanations, contrastive and counterfactual have been identified as suitable for huma…
Local Interpretable Model-Agnostic Explanations (LIME) is a popular technique used to increase the interpretability and explainability of black box Machine Learning (ML) algorithms. LIME typically generates an explanation for a single prediction by any ML model by learning a simpler interpretable model (e.g. linear cla…
ViTaX provides formal guarantees for targeted explanations in safety-critical systems.
problem Need trustworthy explanations for safety-critical deep neural networks.
method Formal reachability analysis for targeted, semifactual explanations.
result First method to provide formally guaranteed explanations of model resilience.
Artificial intelligence (AI) comes with great opportunities but can also pose significant risks. Automatically generated explanations for decisions can increase transparency and foster trust, especially for systems based on automated predictions by AI models. However, given, e.g., economic incentives to create dishones…
Paper proposes Coalitional BAE to improve explainability of unsupervised deep learning models.
problem Improving explainability of Autoencoder's predictions.
method Introduces Coalitional BAE, inspired by agent-based system theory, to reduce correlation in explanations.
result Improved quality of explanations using Coalitional BAE on publicly available datasets.
DECE visualizes machine learning decisions with counterfactual explanations.
problem Making machine learning models transparent and explainable.
method Interactive visualization system supporting counterfactual explanations at instance- and subgroup-levels.
result DECE enables users to explore and understand machine learning model decisions.
TREX explains tree ensembles by identifying key training examples.
problem Identifying which training examples most influence tree ensemble predictions.
method TREX builds a surrogate model using a kernel that captures tree ensemble structure, approximating the original model.
result TREX provides accurate and effective explanations for tree ensembles.
There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, p…
Interpretable machine learning has gained much attention recently. Briefness and comprehensiveness are necessary in order to provide a large amount of information concisely when explaining a black-box decision system. However, existing interpretable machine learning methods fail to consider briefness and comprehensiven…
Financial decisions impact our lives, and thus everyone from the regulator to the consumer is interested in fair, sound, and explainable decisions. There is increasing competitive desire and regulatory incentive to deploy AI mindfully within financial services. An important mechanism towards that end is to explain AI d…
Proposes minimal interventions over counterfactual explanations for algorithmic recourse.
problem Lack of actionable recommendations for algorithmic recourse.
method Causal reasoning to shift focus from explanations to recommendations.
result Minimal interventions provide more actionable recommendations for recourse.
COMRECGC finds common recourse for global counterfactual explanations in GNNs.
problem Finding common recourse for global counterfactual explanations in GNNs.
method Formalized the common recourse explanation problem and designed COMRECGC algorithm.
result COMRECGC outperforms strong baselines on four real-world graph datasets.
Autonomous AI systems will be entering human society in the near future to provide services and work alongside humans. For those systems to be accepted and trusted, the users should be able to understand the reasoning process of the system, i.e. the system should be transparent. System transparency enables humans to fo…
Enhances counterfactual explanations with more valid and informative saliency maps.
problem Lack of valid counterfactual explanations in existing models.
method Introduces a modified approach to CELS model by removing mask normalization.
result Demonstrates higher validity and more informative counterfactual explanations.
Paper proposes counterfactual explanations for ML on multivariate time series data.
problem Lack of user trust and difficulty in debugging ML frameworks using multivariate time series data.
method Proposes a novel explainability technique for providing counterfactual explanations.
result Outperforms state-of-the-art explainability methods in metrics like faithfulness and robustness.
Automates investor/company matching with AI, explaining decisions.
problem Matching companies and investors is hard due to limited data and need for explanations.
method Representation learning for small datasets + parameterized explanation generation.
result System performs well on matching task and explains decisions.
Recourse explanations can become invalid if collective actions change statistical data.
problem Recourse explanations may become invalid due to collective behavior changing data statistics.
method Formal characterization of conditions under which recourse explanations remain valid under performativity.
result Recourse actions may become invalid if they are influenced by or intervene on non-causal variables.
Study proposes a novel local explanation method for deep learning classifiers in process mining.
problem Lack of interpretability in deep learning models for process mining.
method Defines local regions using latent space representations and visualizes explanations.
result Deep learning classifier achieves high performance and local explanations increase user trust.
New explanation of reservoir computing using random projections.
problem Understanding the randomness in reservoir computing.
method Constructing strongly universal reservoir systems as random projections of state-space systems.
result Approximation of any fading memory filters class by training a linear readout for each filter.
Automated decision making is used routinely throughout our everyday life. Recommender systems decide which jobs, movies, or other user profiles might be interesting to us. Spell checkers help us to make good use of language. Fraud detection systems decide if a credit card transactions should be verified more closely. M…
Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networkscs.CV
Interpretation and explanation of deep models is critical towards wide adoption of systems that rely on them. In this paper, we propose a novel scheme for both interpretation as well as explanation in which, given a pretrained model, we automatically identify internal features relevant for the set of classes considered…
As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpr…
New feature mapping approach improves recommendation accuracy and explainability.
problem Balancing recommendation accuracy and explainability using metadata.
method Maps uninterpretable features to interpretable aspect features, minimizing both prediction and interpretation losses.
result Strong performance in recommendation and explainability, eliminating metadata need.