The paper proposes an interpretable off-policy learning algorithm for medical treatments.
problem Lack of interpretable methods for personalized treatment decisions from observational data.
method Hyperbox search approach for interpretable policies in disjunctive normal form.
result The proposed algorithm outperforms state-of-the-art methods in terms of regret and is rated highly interpretable by clinical experts.
CPR models complex decision processes by breaking them into context-specific policies, improving interpretability and accuracy.
problem Interpreting dynamic human decision-making processes in medical contexts.
method Develops Contextualized Policy Recovery (CPR) framework for multi-task learning, modeling each context-specific policy as a linear map.
result Achieves state-of-the-art performance in predicting medical decisions, closing the gap between interpretable and black-box methods.
Signalized intersections are managed by controllers that assign right of way (green, yellow, and red lights) to non-conflicting directions. Optimizing the actuation policy of such controllers is expected to alleviate traffic congestion and its adverse impact. Given such a safety-critical domain, the affiliated actuatio…
Paper proposes interpretable RL policies from a mixture of experts.
problem Making RL policies transparent and understandable in real-world applications.
method Policy iteration scheme with interpretable experts and prototypical states.
result Proposed algorithm learns policies comparable to neural networks but more interpretable.
The paper interprets policy-gradient algorithms using continuation theory.
problem Optimizing nonconvex functions in reinforcement learning.
method Formulates policy optimization as optimization by continuation, interprets policy-gradient algorithms as implicitly optimizing deterministic policies.
result Exploration in policy-gradient algorithms is seen as computing a continuation of the return of the policy.
We present a reinforcement learning framework, called Programmatically Interpretable Reinforcement Learning (PIRL), that is designed to generate interpretable and verifiable agent policies. Unlike the popular Deep Reinforcement Learning (DRL) paradigm, which represents policies by neural networks, PIRL represents polic…
The paper studies how neural policies can be interpreted using decision trees.
problem Understanding how machine learning controllers make decisions in complex environments.
method The approach involves disentangled representation using decision trees to interpret neural policies.
result The paper shows that disentanglement of learned neural dynamics improves interpretability.
The paper proposes a policy learning framework for interpretable personalization.
problem Effective personalization of goods and services to improve revenues and maintain competitive edge.
method Policy learning with linear decision boundaries using causal inference and Bayesian optimization.
result The learned policy improves net sales revenue by 88.2% and provides insights into important features.
System interprets complex treatment effects for personalized policies.
problem Complex, hard-to-understand treatment effect models.
method Scalable, interpretable personalized experimentation system.
result Learned explanations and generated interpretable policies.
Simplifies complex pricing models for better interpretability and revenue.
problem Complex pricing models are hard to interpret and not widely adopted.
method Model distillation to create interpretable pricing policies.
result Maximizes revenue while maintaining interpretability.
AI-Interpret transforms opaque policies into simple, interpretable decision rules.
problem Designing effective decision aids for professionals to mitigate decision-making biases.
method Combining imitation learning, program induction, and clustering to transform learned policies into interpretable descriptions.
result Providing interpretable decision rules as flowcharts significantly improves people's planning strategies and decisions.
This paper uses NLDT to find interpretable control rules from complex DRL policies.
problem Complex, non-interpretable policies from black-box AI methods.
method Evolutionary optimization of NLDT for hierarchical control rules.
result Interpretable control rules with similar performance to black-box DRL.
This research improves interpretability in sequential explanations using mental models.
problem Improving interpretability in sequential explanations between two parties.
method A reinforcement learning framework that selects explanations based on the explainee's mental model.
result Mental model-based policies increase interpretability over random selection in multiple sequential explanations.
ESRL uses uncertainty quantification to learn safe, optimal policies in offline RL.
problem Challenges in interpreting and measuring uncertainty of learned policies in offline RL.
method Expert-Supervised Reinforcement Learning (ESRL) framework that uses hypothesis testing and posterior distributions.
result The framework can learn safe and optimal policies with theoretical guarantees and independent sample efficiency.
Concept-driven OPE reduces variance in off-policy decision evaluation.
problem High variance in off-policy decision evaluation due to limited sample sizes.
method Integrating human-explainable concepts into OPE to reduce variance.
result Concept-based OPE estimators remain unbiased and reduce variance when concepts are known and predefined.
Develops a method to evaluate OPE robustness to hyperparameters and policies.
problem Difficulty in selecting and tuning OPE estimators due to limited experimental evaluations.
method Introduces IEOE (Interpretable Evaluation for Offline Evaluation) to assess robustness.
result Demonstrates improved evaluation of OPE estimators' reliability.
Simplifies complex RL policies by ranking important decisions.
problem Complexity in RL policies makes them hard to analyze and interpret.
method Statistical fault localisation to rank states and prune unimportant decisions.
result Pruned policies can perform similarly to original policies, improving interpretability.
Developed an explainable DRL model for financial portfolio management.
problem Inability of DRL agents to provide interpretable financial investment policies.
method Integrating PPO with feature importance techniques (SHAP, LIME) to enhance transparency.
result Ability to interpret DRL agent actions in prediction time.
Study improves U.S. monetary policy forecasting by integrating text and data.
problem Forecasting central bank policy decisions, especially the Fed's rate changes.
method Multi-modal approach combining structured data and unstructured text from Fed communications.
result Hybrid models outperform unimodal baselines, achieving a test AUC of 0.83.
Interpole learns transparent decision-making policies from data.
problem Understanding human decision-making in opaque environments.
method Interpole combines belief-update and belief-action mapping estimation.
result Interpole provides interpretable models of decision-making behavior.
Framework evaluates post-hoc interpretability methods in time-series classification.
problem Lack of suitable post-hoc interpretability methods for time-series classification.
method Proposes a framework with quantitative metrics to assess interpretability methods.
result Addresses several drawbacks of existing methods, including dependence on human judgement and data distribution shift.
This paper develops explainable treatment policies for RPM using clinical knowledge.
problem Barriers to adoption of DHIs and lack of interpretability in purely black-box algorithms.
method Developed a pipeline for learning explainable treatment policies using clinician-informed representations.
result Policies learned from clinician-informed representations are more efficacious and efficient than black-box policies.
Bayesian framework for policy learning in decision problems.
problem Maximizing expected welfare in decision-making problems.
method Loss-based Bayesian updating and squared-loss surrogate for welfare maximization.
result General Bayes posterior over decision rules with Gaussian pseudo-likelihood interpretation.
Decision trees are ubiquitous in machine learning for their ease of use and interpretability. Yet, these models are not typically employed in reinforcement learning as they cannot be updated online via stochastic gradient descent. We overcome this limitation by allowing for a gradient update over the entire tree that i…
Efficiently computes optimal policies for Entropic Risk Measures.
problem Optimizing risk-sensitive metrics in MDPs is computationally expensive.
method Uses Entropic Risk Measures and novel structural analysis for efficient computation.
result Achieves strong performance in various decision-making scenarios.
Method learns optimal treatment policies from observational data.
problem Learning interpretable treatment assignment policies from observational data.
method Mixed-integer optimization (MIO) technology.
result Asymptotically exact in converging to optimal treatment policies.
Policy optimization is a core component of reinforcement learning (RL), and most existing RL methods directly optimize parameters of a policy based on maximizing the expected total reward, or its surrogate. Though often achieving encouraging empirical success, its underlying mathematical principle on {\em policy-distri…
ScoreMatchingRiesz improves debiased machine learning and policy effects estimation.
problem Improving debiased machine learning and policy effects estimation.
method Score matching and Riesz representer estimation.
result Estimates policy path for continuous treatments, improving interpretability.
The paper proposes a method to explain expert decisions by modeling preferences with 'what if' outcomes.
problem Interpreting and auditing decision-making policies in institutions.
method Integrating counterfactual reasoning into batch inverse reinforcement learning.
result The method effectively recovers accurate and interpretable descriptions of expert behavior.
This work improves policy-based training by proposing an evaluation balance objective for GFlowNets.
problem Reliable estimation of policy divergence under directed acyclic graphs remains challenging.
method Proposes an evaluation balance objective over partial episodes to measure policy divergence and improve policy-based training reliability.
result Evaluation balance strengthens policy-based training reliability and broadens its flexibility.
Imitation learning can reproduce policies by observing experts, which poses a problem regarding policy privacy. Policies, such as human, or policies on deployed robots, can all be cloned without consent from the owners. How can we protect against external observers cloning our proprietary policies? To answer this quest…
Method learns evolving policies in healthcare contexts.
problem Understanding non-stationary behavior in evolving decision-making processes.
method Inverse Contextual Bandits (ICB) approach for learning interpretable representations of non-stationary behavior.
result Demonstrated applicability and accuracy of ICB method in liver transplantation policies.
The ability to interpret machine learning models has become increasingly important now that machine learning is used to inform consequential decisions. We propose an approach called model extraction for interpreting complex, blackbox models. Our approach approximates the complex model using a much more interpretable mo…
Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting. However, vanilla online variants are on-policy only and not able to take advantage of off-policy data. In this paper we describe a new technique that combines policy gradient with off-policy Q-learning, drawing experie…
Paper proposes a new DRL algorithm optimizing Spectral Risk Measures for better risk management.
problem Inconsistencies and conservatism in existing risk measures in DRL.
method Optimizes a broader class of static Spectral Risk Measures (SRM) in DRL.
result Demonstrates improved performance over existing risk-neutral and risk-sensitive DRL models.
In recent years, state-of-the-art game-playing agents often involve policies that are trained in self-playing processes where Monte Carlo tree search (MCTS) algorithms and trained policies iteratively improve each other. The strongest results have been obtained when policies are trained to mimic the search behaviour of…
Deep reinforcement learning has led to several recent breakthroughs, though the learned policies are often based on black-box neural networks. This makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned polici…
New approach for off-policy learning in contextual bandits with performance guarantees.
problem Improving performance of logging policies in contextual bandits.
method PAC-Bayesian analysis of policy mixtures, providing tighter generalization bounds and tractable optimization algorithms.
result Proved tighter generalization bounds and demonstrated effectiveness in practical scenarios.
New model combines neural networks and embeddings for better choice modeling interpretability.
problem Limited behavioral insights in embedding representations for categorical variables.
method Combines discrete choice models and neural networks using embeddings for interpretability.
result Proposed models deliver state-of-the-art predictive performance while preserving interpretability.
Interactive tool helps RL researchers debug and understand their models.
problem Challenges in debugging reinforcement learning algorithms.
method Design and implementation of an interactive visualization tool.
result Addresses features missing from previous tools for RL.
We study the problem of programmatic reinforcement learning, in which policies are represented as short programs in a symbolic language. Programmatic policies can be more interpretable, generalizable, and amenable to formal verification than neural policies; however, designing rigorous learning approaches for such poli…
New approach to understand recurrent policies as FSMs without minimization.
problem Minimization of FSMs obscures the semantics of policy decisions.
method Start with unminimized FSM, apply interpretable reductions, use attention tool.
result Reveals insights into policy decisions not previously noticed.
Unified approach to Merton's portfolio problem using Pontryagin's principles.
problem Optimizing consumption and investment strategies in financial portfolios.
method PG-DPO framework combining neural networks with Pontryagin's maximum principle.
result Locally optimal policies closely tied to classical stochastic control.
Real-world tasks are often highly structured. Hierarchical reinforcement learning (HRL) has attracted research interest as an approach for leveraging the hierarchical structure of a given task in reinforcement learning (RL). However, identifying the hierarchical policy structure that enhances the performance of RL is n…
Optimizes insurance pricing by accounting for policyholders' price sensitivity.
problem Traditional insurance pricing does not consider policyholders' price sensitivity.
method Formulates insurance pricing as a decision-making problem and uses off-policy evaluation and stochastic control.
result Neural networks outperform existing techniques for policy optimization.
Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems. RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features. In …
New interface explains contextual bandits to non-experts.
problem Interpreting and managing contextual bandits for non-expert operators.
method Developed a metric 'value gain' for off-policy evaluation and designed an interface to explain bandit behavior.
result Empowered non-experts to manage complex machine learning systems through accessible presentation.
New framework optimizes decisions under uncertainty considering causal and continuous data.
problem Optimizing decisions under uncertain distributions with causal and continuous data structures.
method Developed a framework using Causal Sinkhorn DRO with Soft Regression Forest decision rules.
result Framework provides interpretable and tractable decision rules for optimizing under uncertainty.