Extract class-specific subnetworks from neural models for better understanding and improved explanations.
problem Understanding and explaining the complex behavior of deep neural networks.
method For each semantic class, extract a class-specific subnetwork with a compressed structure that maintains comparable performance.
result Extracted subnetworks improve explanation saliency and adversarial example detection.
Method extracts understandable explanations for black-box models using matrix factorization.
problem Lack of human-understandable explanations for black-box classification models.
method Introduces contribution matrix and explanation embedding using matrix factorization.
result Extracts rule-like model explanations from contribution matrix.
Attack reveals model details from counterfactual explanations.
problem Extracting model details from counterfactual explanations.
method Adversary uses counterfactual explanations to build high-fidelity model.
result High-fidelity and high-accuracy model extraction possible.
ExpBERT uses natural language explanations to improve text interpretation.
problem Improving text interpretation for relation extraction tasks.
method Fine-tuning BERT on MultiNLI to interpret natural language explanations.
result ExpBERT matches a BERT baseline but requires less labeled data and improves performance.
Advances text explanation method for social media attacks.
problem Identifying personal attacks in social media comments.
method Adversarial approach to extract high-recall explanations from neural text classifiers.
result Demonstrates the importance of manually setting a default behavior for the model.
CRITS improves time series classification with interpretable local explanations.
problem Lack of detailed explanations in time series classification models.
method CRITS uses convolutional kernels, max-pooling, and rectified linear units to extract feature weights.
result CRITS provides intrinsically interpretable local explanations without requiring gradients or random perturbations.
Evaluates explanations of LTR models using decision paths and compares their accuracy.
problem Challenges in evaluating local explanations of LTR models due to lack of ground truth feature importance scores.
method Focuses on tree-based LTR models, extracts ground truth feature importance scores using decision paths, and compares them with explanation techniques.
result Explanation accuracy varies depending on the model and data point.
Study shows more data improves model explanations, aiding reliable knowledge extraction.
problem Challenges in deriving reliable knowledge from machine learning models due to the Rashōmon effect.
method Examined the influence of sample size on explanations from models in a Rashōmon set using SHAP.
result Explanations from <128 samples are highly variable, but agreement improves with more data.
Local surrogate explainers vary in objectives, leading to incomparable explanations.
problem Variability in objectives among local surrogate explainers.
method Review of multiple local surrogate explainers, focusing on extracted information.
result Diverse explanations from similar methods due to differing objectives.
New method compares feature importance and rule extraction for text data interpretability.
problem Unexpected differences in explanations from feature importance and rule extraction methods.
method Proposes a new approach to compare explanations from different methods.
result Different methods can lead to unexpected explanations, even for simple models.
The paper explores methods to explain decisions of deep learning models by faithfully reproducing their training data views.
problem Explaining decisions of complex deep learning models trained on large datasets.
method Data view extraction through hill-climbing and GAN-driven approaches, followed by creation of shadow models for explanation.
result Shadow models based on faithfully reproduced data views are effective for explaining decisions of blackbox deep learning models.
Extracts salient concepts from CNNs for explaining deep neural networks.
problem Explaining the opaque behavior of deep neural networks in safety-critical domains.
method Uses autoencoders to extract salient concepts and builds a Bayesian causal model.
result Identifies and visualizes features influencing deep neural network classifications.
Proposes a method to generate counterfactual and contrastive explanations using SHAP.
problem Need for explainable AI and legal requirement for model interpretability.
method Model agnostic method using SHAP to generate contrastive and counterfactual explanations.
result Demonstrates effectiveness of the method on various datasets.
This paper introduces glocal explanations for expected goal models in soccer.
problem Limited interpretability of expected goal models trained with black-box methods.
method Proposes glocal explanations using aggregated SHAP values and partial dependence profiles.
result Extracts knowledge from expected goal models for teams and players, enhancing performance analysis.
Explains agent behavior through intended outcomes in reinforcement learning.
problem Proving impossibility of general post-hoc explanations in reinforcement learning.
method Derives local explanations based on intention for Q-function approximations, proving consistency with learned Q-values.
result Demonstrates the necessity of collecting information during training for accurate explanations.
Novel framework explains machine learning models using ontology-based sampling.
problem Generating precise and insightful explanations for machine learning models.
method Ontology-based sampling technique and learnable anchor algorithm.
result Our approach generates more precise and insightful explanations.
New method to evaluate visual explanations from neural networks.
problem Lack of consensus on measuring effectiveness of visual explanations.
method Proposed a new procedure for evaluating explanations using a range of sources.
result Demonstrated the benefit of combining different sources and the impact of bias parameters.
Paper introduces new evaluation criteria for feature-based model explanations.
problem Establishing reliable feature importance explanations for models.
method Robustness analysis using smaller adversarial perturbations.
result New explanations that are necessary and sufficient for predictions.
EDUCE model explains text predictions by identifying key concepts.
problem Providing clear explanations for text predictions.
method Uses unsupervised binary concept representations to explain model decisions.
result Model automatically defines relevant concepts for prediction tasks.
The paper explains deep learning models for recommendations using layer-wise relevance propagation.
problem Explainable recommendations in deep learning models.
method Layer-wise relevance propagation applied to a Deep Convolutional Neural Network.
result Demonstrates the effectiveness of the method on an Amazon products dataset.
Identifies influential neurons in deep networks for better explanations.
problem Explaining complex behaviors of deep neural networks.
method Identifies neurons with high influence using an influence measure and provides interpretations.
result Identifies influential concepts that generalize across instances and isolates individual features used by the network.
Method uses elastic black-boxes to create interpretable models from complex ones.
problem Lack of trust and stability in opaque models and time-consuming feature engineering in interpretable models.
method Surrogate assisted feature extraction for model learning.
result Trains interpretable and accurate models without time-consuming feature engineering.
Develops ACE to automatically identify meaningful concepts from neural network predictions.
problem Challenges in interpreting feature importance scores for machine learning models.
method Proposes concept-based explanation principles and develops ACE algorithm to extract visual concepts.
result Demonstrates ACE discovers human-meaningful, coherent concepts for neural network predictions.
DEMUD-VIS detects novel image content and explains it visually.
problem Detecting and explaining novel image content in large datasets.
method Uses CNN for feature extraction, reconstruction error for novelty detection, and up-convolutional networks for image reconstruction.
result Demonstrates visual explanations of novel image content on diverse datasets.
Proposes a Bayesian approach to explain, justify, and quantify uncertainty in DNNs.
problem Lack of transparency and confidence in DNNs for critical applications.
method Bayesian approach to extract explanations, justifications, and uncertainty estimates from black box DNNs.
result Improves interpretability and reliability of DNNs, validated on CIFAR-10.
New method clusters disease subtypes from model explanations.
problem Discovering disease subtypes in noisy, high-dimensional data.
method Train classifier, extract explanations, cluster in explanation space.
result Cluster analysis on model explanations outperforms classical methods.
New method for explaining dialogue response generation models.
problem Interpreting sequence generation models, especially dialogue response generation.
method Local Explanation of Response Generation (LERG) method.
result LERG improves dialogue response generation explanations compared to existing methods.
New method explains GNN predictions using walks.
problem GNNs are black-boxes and hard to explain.
method Nested attribution scheme using relevant walks.
result Extracts meaningful explanations from GNNs.
DTOR explains anomalies with rule-based explanations.
problem Need to explain anomalies in data effectively.
method Applies Decision Tree Regressor to estimate anomaly scores and generate rule-based explanations.
result DTOR produces robust and consistent rule-based explanations.
Hybrid Deep Embedding for aspect-level explanations in recommendations.
problem Challenges in personalization, dynamic explanations, and aspect-level granularity in recommendation systems.
method Proposes Hybrid Deep Embedding (HDE) to learn dynamic embeddings for user and item preferences, and aspect-level quality vectors.
result Demonstrates improved recommending performance and dynamic aspect-level explanations.
This paper improves neural network explanations by quantifying and visualizing semantic compositions.
problem Improving neural network explanations for natural language processing tasks.
method Proposes a formal way to quantify word and phrase importance, introduces SCD and SOC algorithms.
result Our algorithms outperform prior methods in explaining neural network predictions.
Proposes an efficient method for ordered counterfactual explanations.
problem Insufficient explanation of perturbation vectors for executing actions.
method Mixed-Integer Linear Optimization (MILP) approach for evaluating and extracting optimal pairs of actions and orders.
result Demonstrated effectiveness of the proposed method on real datasets.
Interactive explanations improve machine learning transparency.
problem Transparency of machine learning predictions for diverse stakeholders.
method Personalized counterfactual explanations and follow-up questions.
result Improved understanding of black-box systems through interactive explanations.
VINE visualizes statistical interactions in complex models.
problem Lack of utilities for regional explanations in black box models.
method VINE algorithm to extract and visualize statistical interaction effects.
result VINE provides a novel evaluation metric for visualizations.
Proposes new methods for interpreting document classification models.
problem Interpretation fragility of attention-based neural networks.
method Corpus-level and concept-based explanation methods using attention weights.
result Extracts semantically meaningful keywords and concepts for model predictions.
Archipelago provides interpretable explanations of feature interactions in machine learning models.
problem Interpreting the impact of feature interactions on predictions in machine learning models.
method Archipelago is a novel framework for extracting and attributing feature interactions in a scalable and interpretable manner.
result Archipelago provides significantly more interpretable explanations of feature interactions than comparable methods.
Unified local and global explanations through functional decomposition of low dimensional structures.
problem Tackles the challenge of extracting meaningful local and global explanations from machine learning models.
method Proposes a new identification constraint to decompose the global representation into main and interaction components of arbitrary order.
result Unified local and global explanations by connecting partial dependence plots and interventional SHAP values.
A new method simplifies feature explanation for complex models.
problem Explain predictions from complex models efficiently with many features.
method groupShapley: groups features for Shapley value computation.
result Equivalent to summing feature-wise Shapley values within groups.
New method explains cluster assignments in neural networks.
problem Lack of explainability in cluster models.
method Rewriting clustering models as neural networks.
result Ability to attribute cluster predictions to input features.
Combining global and local explanations improves user understanding of RL agents.
problem Challenges in explaining agent behavior due to large state spaces and delayed rewards.
method Integrating strategy summaries with saliency maps to provide both global and local explanations.
result Summaries including important states significantly improve user understanding of RL agents.
Self-explaining AI provides understandable explanations for AI decisions.
problem Difficulty in interpreting decisions made by deep neural networks, especially in critical applications.
method Introducing self-explaining AI that provides human-understandable explanations and confidence levels.
result Deep neural networks operate by interpolating between data points, making them hard to interpret.
Proposes a new feature-based evaluation method for explaining Deep Learning models in text classification.
problem Lack of consideration for linguistic dependencies in existing attribution-based explanations.
method Investigates perturbations based on embedded features removal from intermediate layers of Convolutional Neural Networks.
result Visualization tool assists analysts in understanding model predictions better.
WassersteinGrad improves weather forecasting explanations by addressing geometric misalignment issues.
problem Improving explainability of autoregressive neural predictions on dynamic physical fields.
method WassersteinGrad, a geometric consensus method for averaged perturbed attribution maps.
result WassersteinGrad provides more accurate explanations for weather forecasting models.
This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two ba…
xNN simplifies neural networks for interpretable feature extraction.
problem Complex neural networks are hard to interpret.
method Developed a structured neural network (xNN) for interpretable features.
result xNN provides a parsimonious explanation of feature relationships.
DeepUnHide uses deep learning to reveal hidden demographic features in recommender systems.
problem Extracting hidden demographic features from recommender systems factors.
method Gradient-based localization in deep learning for feature extraction.
result DeepUnHide outperforms state-of-the-art feature selection methods.
Extends SHAP to explain autoencoder anomalies, aiding experts.
problem Manual validation of autoencoder anomalies is challenging.
method Extends SHAP to explain autoencoder anomalies.
result Visualizes features contributing to and offsetting anomalies.
OpenTag extracts missing attribute values from product descriptions.
problem Extract missing attribute values from product descriptions.
method Developed a deep tagging model OpenTag using LSTM and CRF, with an attention mechanism and active learning.
result OpenTag discovers new attribute values with minimal human annotation, achieving high F-score.