ExpBERT uses natural language explanations to improve text interpretation.
problem Improving text interpretation for relation extraction tasks.
method Fine-tuning BERT on MultiNLI to interpret natural language explanations.
result ExpBERT matches a BERT baseline but requires less labeled data and improves performance.
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
iPrompt uses LLMs to generate natural-language explanations of data patterns.
problem Finding and explaining patterns in data using natural language.
method Interpretable autoprompting (iPrompt) that generates natural-language explanations based on LLMs.
result iPrompt can accurately find and explain data patterns, improving upon human-written prompts.
This research integrates attention into XAI frameworks for better model explanations.
problem Improving the interpretability of transformer models.
method Developed two novel explanation methods: Shapley value decomposition and token-level directional derivatives.
result Attention weights can be meaningfully incorporated into XAI frameworks, enhancing transformer explainability.
The task of Natural Language Inference (NLI) is widely modeled as supervised sentence pair classification. While there has been a lot of work recently on generating explanations of the predictions of classifiers on a single piece of text, there have been no attempts to generate explanations of classifiers operating on …
Language helps RL agents learn complex relational and causal structures.
problem Learning relational and causal structure in complex environments.
method Training RL agents to predict language descriptions and explanations.
result Language aids agents in learning challenging relational and causal tasks.
ALMANACS benchmarks explainability methods on simulatability.
problem Evaluating the effectiveness of explainability methods for language models.
method ALMANACS is a simulatability benchmark that evaluates explainability methods on twelve safety-relevant topics.
result No explainability method outperforms the explanation-free control across all topics.
This paper presents the beginnings of an automatic statistician, focusing on regression problems. Our system explores an open-ended space of statistical models to discover a good explanation of a data set, and then produces a detailed report with figures and natural-language text. Our approach treats unknown regression…
We focus on the problem of search in the multilingual setting. Examining the problems of next-sentence prediction and inverse cloze, we show that at large scale, instance-based transfer learning is surprisingly effective in the multilingual setting, leading to positive transfer on all of the 35 target languages and two…
Measures faithfulness of LLM explanations to reveal hidden biases and misleading claims.
problem LLM explanations can misrepresent the model's reasoning process, leading to over-trust and misuse.
method Defines faithfulness in terms of concept influence and uses counterfactuals and Bayesian models to estimate it.
result Can quantify and discover interpretable patterns of unfaithfulness in LLM explanations.
LLMs can help explain credit risk models but not autonomously.
problem Leveraging LLMs for post-hoc explainability in credit risk models.
method Comparison of LLM outputs with SHAP and coefficient-based attributions on three LMs.
result LLMs reliably preserve feature-importance rankings but poorly align with autonomous explanations.
The paper argues for prioritizing identifying structure over complex models for scientific discovery.
problem Underdetermination of mechanisms in high-dimensional data, leading to unreliable explanations.
method Proposes concrete standards for 'mechanistic ML' to avoid collapsing explanations.
result Large language models (LLMs) can collapse large equivalence classes of explanations, making it hard to distinguish between mechanisms.
Humans are able to explain their reasoning. On the contrary, deep neural networks are not. This paper attempts to bridge this gap by introducing a new way to design interpretable neural networks for classification, inspired by physiological evidence of the human visual system's inner-workings. This paper proposes a neu…
This work introduces RISE to explain LLMs more reliably by distinguishing essential context.
problem Identifying which context elements influence LLM outputs reliably.
method RISE (Redundancy-Insensitive Scoring of Explanation) method.
result RISE provides more robust explanations than traditional methods.
Study finds optimal vocabulary size for neural machine translation.
problem Imbalanced class distribution in language data affects NMT performance.
method Casts NMT as a classification task, analyzes vocabulary sizes, and tests multiple languages.
result Certain vocabulary sizes outperform others, explaining NMT performance.
System detects financial misinformation and generates clear explanations.
problem Identifying and explaining fraudulent financial content.
method Combined large language models, pre-processing, and sequential learning.
result Achieved F1-score of 0.8283 for classification and ROUGE-1 of 0.7253 for explanations.
The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the model handles semantic compositions, we study hierarchical explanation of neural network predictions. We identify non-additivity and contex…
Although neural networks can achieve very high predictive performance on various different tasks such as image recognition or natural language processing, they are often considered as opaque "black boxes". The difficulty of interpreting the predictions of a neural network often prevents its use in fields where explaina…
Study uses ML to predict currency and bond returns from news sentiment.
problem Predicting financial returns from news sentiment.
method Pretrained FinBERT model on finance-specific language, XGBoost classifier, SHAP for interpretability.
result XGBoost strategy outperforms benchmarks with Sharpe ratios > 5.
A new approach to rationalization identifies true rationales by considering causal relationships.
problem Existing rationalization methods struggle with spuriousness, where snippets with similar contributions are hard to distinguish.
method The method leverages causal inference to identify non-spurious rationales, defining probabilities of causation based on a structural causal model.
result The proposed causal rationalization outperforms existing methods on real-world datasets.
A new framework quantifies how model explanations influence each other.
problem Understanding how different model explanations interact and influence each other.
method Introducing the metagame, a conceptual framework for measuring second-order interaction effects of model explanations using Shapley values.
result Meta-attributions provide directional insights into how feature interactions influence model explanations.
A simple model explains phase transition in large language models.
problem Understanding the emergence of abilities in large language models.
method Modeling LLM as a sequence-to-sequence random function and using a list decoder.
result A critical threshold exists where the expected number of erroneous sequences grows exponentially.
LLMs' explanations are often insufficient and vary with input distribution.
problem Evaluating the sufficiency of LLM explanations without predefined biases.
method Generalizing sufficiency to arbitrary explanations, using LLM's input beliefs, and introducing SCSuff metric.
result Explanation sufficiency can vary with input distribution and is weakly correlated with model size, accuracy, or output entropy.
Triangulation filters spurious circuits in multilingual models.
problem Unreliable explanations of multilingual models across languages.
method Formalizes reference families and introduces triangulation as a causal acceptance rule.
result Triangulation provides a falsifiable standard for mechanistic claims.
SEP framework teaches LLMs to generate explainable stock predictions.
problem Challenging task of generating human-readable explanations for stock predictions.
method Self-reflective agent and Proximal Policy Optimization (PPO) for autonomous learning.
result Fine-tuned LLM outperforms traditional methods in prediction accuracy and Matthews correlation coefficient.
Formulates approach for guiding explanation types based on user specifications.
problem Creating explainable AI components from user-defined specifications.
method Develops a method for generating explanations based on user-defined specifications.
result Demonstrates feasibility of user-defined explanations for complex models like Bayesian networks and graph neural networks.
Structured prediction is used in areas such as computer vision and natural language processing to predict structured outputs such as segmentations or parse trees. In these settings, prediction is performed by MAP inference or, equivalently, by solving an integer linear program. Because of the complex scoring functions …
This paper reviews methods for interpreting deep learning models with sequential data.
problem Limited interpretability of deep learning models in sequential data domains.
method Reviews and compares techniques for sequential interpretability.
result Current techniques have limitations and future research is needed.
Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative Poisson topic model with long documents, dropout training improves the exponent…
TRUST improves tree models' accuracy while maintaining interpretability.
problem Piecewise-constant regression trees lack in predictive accuracy compared to black-box models.
method Combines Random Forest accuracy with interpretability of shallow trees and sparsity of linear models, using LLMs for explanations.
result TRUST outperforms other interpretable models in predictive accuracy and matches Random Forest's accuracy.
ClauseLens uses reinforcement learning to price reinsurance treaties transparently and auditably.
problem Opaque and difficult-to-audit reinsurance treaty pricing practices.
method ClauseLens models treaty pricing as a Risk-Aware Constrained Markov Decision Process (RA-CMDP), incorporating legal clauses and generating interpretable explanations.
result ClauseLens reduces solvency violations and improves tail-risk performance, achieving 88.2% accuracy in clause-grounded explanations.
Research aims to make fact-checking models more transparent.
problem Making fact-checking models explainable in a complex field.
method Combines fact-checking methods with explainable AI techniques.
result Developed initial solutions for explainable fact-checking.
This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
PACE explains ViTs by modeling patch-level concept distributions, surpassing existing methods.
problem Lack of trustworthy post-hoc explanations for Vision Transformers (ViTs)
method Variational Bayesian explanation framework (PACE)
result PACE surpasses state-of-the-art methods in meeting desiderata for ViT explanations.
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
problem Understanding brain activity during language processing.
method Brain imaging studies and natural language representations.
result Development of brain-aware natural language representations.
Explearn learns to explain predictions using Gaussian Processes.
problem Learning to explain predictions effectively.
method Gaussian Processes-based contextual bandits.
result Guaranteed convergence with high probability.
We introduce a method to learn a hierarchy of successively more abstract representations of complex data based on optimizing an information-theoretic objective. Intuitively, the optimization searches for a set of latent factors that best explain the correlations in the data as measured by multivariate mutual informatio…
KG-A2C agent learns natural language IF games by reasoning and constraining action spaces.
problem Challenges of natural language understanding, partial observability, and combinatorially large action spaces in IF games.
method Builds a dynamic knowledge graph while exploring, constraining actions using templates.
result Outperforms current IF agents across various games with larger action spaces.
New framework explains ML credit scoring models using counterfactual examples.
problem Explaining complex ML models in finance for credit scoring.
method Adversarial counterfactual examples for tabular data.
result Proposes a method to generate realistic counterfactual examples for tabular data.
PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.
problem Sparse reward settings in RL limit applicability to complex problems.
method Directly maps natural language descriptions to pixel-based rewards for guiding RL.
result Language-based rewards significantly improve sample efficiency in policy learning.
Improved robot navigation using multi-head attention for natural language instructions.
problem Improving robot navigation in unfamiliar environments.
method Proposes a multi-head attention mechanism blending layer in a neural network model.
result Significant performance gains in translating instructions for unseen environments.
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
Explains agent behavior through intended outcomes in reinforcement learning.
problem Proving impossibility of general post-hoc explanations in reinforcement learning.
method Derives local explanations based on intention for Q-function approximations, proving consistency with learned Q-values.
result Demonstrates the necessity of collecting information during training for accurate explanations.
Automates translating natural language to Verilog for digital design.
problem Manual translation of natural language specifications to Verilog is time-consuming and error-prone.
method Fine-tuned GPT-2 to derive Verilog from English, using a dataset of design tasks.
result GPT-2 achieved 94.8% correct translation across simple and abstract design tasks.
Paper proposes Coalitional BAE to improve explainability of unsupervised deep learning models.
problem Improving explainability of Autoencoder's predictions.
method Introduces Coalitional BAE, inspired by agent-based system theory, to reduce correlation in explanations.
result Improved quality of explanations using Coalitional BAE on publicly available datasets.
The study enhances financial rule matching using NLP without datasets.
problem Performing semantic matching between financial rules and policies.
method Outperforming pre-trained models with NLP techniques using free resources.
result Improved semantic matching between financial rules and policies.
This study applies neural models to automatically recognize medical entities from natural language.
problem Automated recognition of medical entities from natural language is complex and time-consuming.
method Utilizes deep neural sequence models trained on a large dataset of death certificates.
result Deep neural models can efficiently recognize medical entities from natural language.
New algorithm explains DNN predictions using adversarial attacks.
problem Incomprehensible complexity of deep neural networks.
method Adversarial machine learning to identify feature importance.
result Consistent and efficient explanations of DNN predictions.