BL learns interpretable optimization structures from data.
problem Learning interpretable optimization structures from data.
method BL parameterizes a compositional utility function from intrinsically interpretable modular blocks.
result BL supports architectures from single to hierarchical compositions, modeling hierarchical optimization structures.
Paper develops a framework for learning interpretable representations of sequential decision behavior.
problem Obtaining a transparent description of existing behavior.
method Inverse decision modeling framework, formalizing both forward and inverse problems.
result Learning interpretable representations of behavior, including suboptimal actions, biased beliefs, and imperfect knowledge.
FLARKO uses LLMs, KGs, and KTO to generate profitable, behaviorally aligned financial recommendations.
problem Financial recommendation systems often fail to account for behavioral and regulatory factors.
method FLARKO integrates LLMs, KGs, and KTO to generate profitable and behaviorally aligned recommendations.
result FLARKO consistently outperforms state-of-the-art recommendation baselines on behavioral alignment and joint profitability.
Interprets how intrinsic motivation shapes behavior in RL agents.
problem Understanding how intrinsic motivation influences behavior in reinforcement learning agents.
method Analyzed five RL agents in procedurally generated environments using various interpretability techniques.
result Curiosity-driven agents exhibit broader and more dynamic attention than extrinsically motivated agents.
Improved scalability and interpretability in training data attribution.
problem Identifying which training data drives specific behaviors, especially unintended ones.
method Leveraging interpretable structures within the model to attribute model behavior to semantic directions, not individual test examples.
result Simple probe-based attribution methods are first-order approximations of Concept Influence that achieve comparable performance while being over an order-of-magnitude faster.
New framework learns interaction rules from animal trajectories.
problem Challenges in extracting interaction rules from animal movement data.
method Augmented behavioral models with neural networks and theory-guided regularization.
result Improved performance over baselines and novel biological insights.
New framework evaluates model explanations based on decision task improvement.
problem Evaluation of model explanations often misses practical value.
method Decision-theoretic framework quantifying three key values.
result Provides benchmarks and interprets human-AI decision support.
To study how mental object representations are related to behavior, we estimated sparse, non-negative representations of objects using human behavioral judgments on images representative of 1,854 object categories. These representations predicted a latent similarity structure between objects, which captured most of the…
Continuous collection of physiological data from wearable sensors enables temporal characterization of individual behaviors. Understanding the relation between an individual's behavioral patterns and psychological states can help identify strategies to improve quality of life. One challenge in analyzing physiological d…
Machine learning has proved to be very successful for making predictions in travel behavior modeling. However, most machine-learning models have complex model structures and offer little or no explanation as to how they arrive at these predictions. Interpretations about travel behavior models are essential for decision…
The roundworm C. elegans exhibits robust escape behavior in response to rapidly rising temperature. The behavior lasts for a few seconds, shows history dependence, involves both sensory and motor systems, and is too complicated to model mechanistically using currently available knowledge. Instead we model the process p…
Paper exposes vulnerabilities in interpreting machine learning models using adversarial attacks on PD plots.
problem Vulnerability of permutation-based interpretation methods, particularly PD plots, to adversarial attacks.
method Adversarial framework to manipulate black-box models and produce deceptive PD plots.
result It is possible to hide discriminatory behaviors in machine learning models through interpretation tools like PD plots.
At the core of interpretable machine learning is the question of whether humans are able to make accurate predictions about a model's behavior. Assumed in this question are three properties of the interpretable output: coverage, precision, and effort. Coverage refers to how often humans think they can predict the model…
Logit models are usually applied when studying individual travel behavior, i.e., to predict travel mode choice and to gain behavioral insights on traveler preferences. Recently, some studies have applied machine learning to model travel mode choice and reported higher out-of-sample predictive accuracy than traditional …
Recent years have witnessed an increased focus on interpretability and the use of machine learning to inform policy analysis and decision making. This paper applies machine learning to examine travel behavior and, in particular, on modeling changes in travel modes when individuals are presented with a novel (on-demand)…
The paper proposes a method to calibrate healthcare AI models for reliability and interpretability.
problem Characterizing model reliability and enabling introspection of model behavior in clinical decision making.
method A calibration-driven learning method combined with interpretability techniques based on counterfactual reasoning.
result Demonstrates the effectiveness of the proposed approach using a lesion classification problem with dermoscopy images.
Study of urban lifestyles from mobility data of 1.2M people in 11 U.S. cities.
problem Lack of interpretability in digital mobility data for understanding urban lifestyles.
method Privacy-enhanced dataset of mobility visitation patterns, latent activity behavior decomposition.
result Detected 12 latent activity behaviors that describe urban lifestyles, not single lifestyles.
The current work characterizes the users of a VoD streaming space through user-personas based on a tenure timeline and temporal behavioral features in the absence of explicit user profiles. A combination of tenure timeline and temporal characteristics caters to business needs of understanding the evolution and phases o…
VICE embeds concepts in a vector space using human data.
problem Developing numerical models for mental representations of object concepts.
method Variational Interpretable Concept Embeddings (VICE) using variational inference and triplet odd-one-out task data.
result VICE outperforms SPoSE at predicting human behavior and provides more reproducible object representations.
Our everyday interactions with pervasive systems generate traces that capture various aspects of human behavior and enable machine learning algorithms to extract latent information about users. In this paper, we propose a machine learning interpretability framework that enables users to understand how these generated t…
Predicts academic risk in college students using interpretable machine learning.
problem Predicting academic risk from high-dimensional, unbalanced student data.
method Binary classification task using LightGBM model and Shapley value.
result 8 predictors for academic risk identified, including quality of academic partners and dormitory study atmosphere.
Method learns evolving policies in healthcare contexts.
problem Understanding non-stationary behavior in evolving decision-making processes.
method Inverse Contextual Bandits (ICB) approach for learning interpretable representations of non-stationary behavior.
result Demonstrated applicability and accuracy of ICB method in liver transplantation policies.
Paper models market dynamics using bull and bear forces.
problem Complex market dynamics influenced by biases and narratives.
method Bias to Behavior from Bull-Bear Dynamics (B4) model.
result Model predicts market trends with superior performance and interpretable insights.
This paper tackles interpretability of LLMs in finance.
problem Complexity and lack of transparency of LLMs in finance.
method Mechanistic interpretability to understand LLM behavior.
result Demonstrates practical relevance of mechanistic interpretability in financial use cases.
Interpretable machine learning tackles the important problem that humans cannot understand the behaviors of complex machine learning models and how these models arrive at a particular decision. Although many approaches have been proposed, a comprehensive understanding of the achievements and challenges is still lacking…
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
Traffic actors' future motion predicted using a hybrid graph model.
problem Predicting long-term behaviors of traffic actors in complex scenes.
method A hybrid graph model with nodes for actors and traffic elements, and edges for interaction types.
result TrafficGraphNet achieves state-of-the-art trajectory prediction accuracy.
XAI methods often misattribute importance to suppressor variables, limiting their reliability.
problem XAI methods lack theoretical and empirical evidence for the correctness of their explanations.
method Analytical expressions for XAI methods on a simple binary classification problem.
result XAI methods will often attribute non-zero importance to non-class-related suppressor features.
New model combines neural networks and embeddings for better choice modeling interpretability.
problem Limited behavioral insights in embedding representations for categorical variables.
method Combines discrete choice models and neural networks using embeddings for interpretability.
result Proposed models deliver state-of-the-art predictive performance while preserving interpretability.
New analysis shows interpretability doesn't guarantee steering utility in LLMs.
problem Does higher interpretability lead to better steering utility in large language models?
method Trained 90 SAEs across three LLMs, evaluated interpretability and steering utility, used Kendall's rank coefficients for analysis.
result Interpretability is only weakly associated with steering utility, and features selected by Delta Token Confidence improve steering performance.
State of the art machine learning algorithms are highly optimized to provide the optimal prediction possible, naturally resulting in complex models. While these models often outperform simpler more interpretable models by order of magnitudes, in terms of understanding the way the model functions, we are often facing a …
Model-agnostic interpretation methods can mislead if not used carefully.
problem Misinterpretation of machine learning models due to improper use of techniques.
method General pitfalls of model-agnostic interpretation methods.
result Many pitfalls exist when using global interpretation techniques for machine learning models.
New methods predict language model out-of-distribution behaviors using causal mechanisms.
problem Predicting how language models behave on unseen data.
method Two methods: counterfactual simulation and value probing.
result Both methods achieve high AUC-ROC and outperform causal-agnostic approaches in out-of-distribution settings.
This paper interprets neural network ECG models by breaking them into understandable components.
problem Lack of model interpretability in deep learning for medical applications.
method Factorizes neural network models into interpretable black box components using hierarchical equations.
result Demonstrates interpretable component models for ECG waveforms, improving model understanding and predictability.
AI-Interpret transforms opaque policies into simple, interpretable decision rules.
problem Designing effective decision aids for professionals to mitigate decision-making biases.
method Combining imitation learning, program induction, and clustering to transform learned policies into interpretable descriptions.
result Providing interpretable decision rules as flowcharts significantly improves people's planning strategies and decisions.
In this paper, we propose a new feature extraction technique for program execution logs. First, we automatically extract complex patterns from a program's behavior graph. Then, we embed these patterns into a continuous space by training an autoencoder. We evaluate the proposed features on a real-world malicious softwar…
The problem of explaining deep learning models, and model predictions generally, has attracted intensive interest recently. Many successful approaches forgo global approximations in order to provide more faithful local interpretations of the model's behavior. LIME develops multiple interpretable models, each approximat…
We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet. TabNet uses sequential attention to choose which features to reason from at each decision step, enabling interpretability and more efficient learning as the learning capacity is used for the most salient fea…
Unified approach to verify NN properties using ReLU's unique polytope structure.
problem Lack of robustness and interpretability in ReLU NNs for risk-sensitive applications.
method Identifying and traversing the local polytopes of ReLU NNs, developing an algorithm to verify properties.
result Unified approach to examine network behavior in risk-sensitive settings.
New framework improves interpretability of trainable prompts.
problem Improving task-specific LLM performance with soft prompts remains a black-box method.
method Developed a theoretical framework for evaluating interpretability of trainable prompts, inspired new objective functions.
result Found a fundamental trade-off between interpretability and task performance in trainable prompts.
The paper uses model-based trees to create interpretable surrogate models for complex machine learning models.
problem Interpreting complex machine learning models.
method Using model-based trees to partition feature space and create interpretable models.
result Model-based trees generate optimal surrogate models that balance interpretability and performance.
Combines neural networks and STL for multi-class time-series classification.
problem Lack of interpretability in neural networks for time-series data.
method Proposes a method that uses neural networks to classify time-series data using STL specifications, introducing margin for multi-class classification and STL-based attributes for interpretability.
result Evaluations show improved interpretability and performance compared to state-of-the-art baselines.
Study develops a new model for predicting individual mobility based on activity patterns.
problem Existing mobility prediction methods overlook underlying activity patterns.
method Activity-based hidden Markov model (IOHMM) using transit smart card data.
result Proposed model achieves similar prediction performance to LSTM but also provides behavioral insights.
We calculate the asymptotic behavior of the Kashaev invariant of a twice-itarated torus knot and obtain topological interpretation of the formula in terms of the Chern--Simons invariant and the twisted Reidemeister torsion.
Mathematical study shows post-hoc explanations are better than attention weights alone.
problem Understanding the internal behavior of attention-based models.
method Mathematical analysis of a simple attention-based architecture.
result Post-hoc explanations provide more useful insights than attention weights alone.
Interpole learns transparent decision-making policies from data.
problem Understanding human decision-making in opaque environments.
method Interpole combines belief-update and belief-action mapping estimation.
result Interpole provides interpretable models of decision-making behavior.
Study evaluates five LLMs for financial report analysis, revealing performance differences and variability.
problem Lack of understanding in reliability, consistency, and transparency of LLMs in financial analysis.
method Human evaluation, automated similarity metrics, and behavioral diagnostics applied to five transformer-based LLMs over U.S. 10-K filings.
result No single LLM consistently dominates across all evaluation perspectives, highlighting variability and need for interpretability.
We model human decision-making behaviors in a risk-taking task using inverse reinforcement learning (IRL) for the purposes of understanding real human decision making under risk. To the best of our knowledge, this is the first work applying IRL to reveal the implicit reward function in human risk-taking decision making…