Paper proposes a method to optimize policies for diverse individuals using heterogeneous data.
problem Learning optimal policies for a heterogeneous population from pre-collected data.
method Individualized offline policy optimization framework for heterogeneous MDPs.
result The proposed P4L algorithm achieves a fast rate of average regret.
New method estimates optimal dose intervals for personalized treatment.
problem Learning optimal dose intervals from observational data.
method Probability dose interval (PDI) method using DC algorithm.
result Consistent policy with risk converging to best-in-class at root-n rate.
Aims to create safe reinforcement learning policies by considering individual harm.
problem Optimal policies for a population may harm certain individuals.
method Formalizes individual harm, proposes a two-stage procedure, and establishes finite-sample properties.
result Learned policies maximize expected return while minimizing harm.
We are witnessing an increasing use of data-driven predictive models to inform decisions. As decisions have implications for individuals and society, there is increasing pressure on decision makers to be transparent about their decision policies. At the same time, individuals may use knowledge, gained by transparency, …
We develop an off-policy actor-critic algorithm for learning an optimal policy from a training set composed of data from multiple individuals. This algorithm is developed with a view towards its use in mobile health.
Method learns optimal treatment sequences from observational data.
problem Optimal dynamic treatment regimes for public policies and medical interventions.
method Doubly robust classification-based approach via backward induction.
result Achieves optimal convergence rate of n^(-1/2) for welfare regret.
The paper proposes a new policy for optimal treatment allocation based on quantile treatment effects.
problem Optimal treatment allocation policies that target distributional welfare, especially when individuals are heterogeneous.
method The approach involves allocating treatments based on the conditional quantile of individual treatment effects (QoTE), considering both prudent and negligent policymakers.
result The proposed minimax policies are robust to model uncertainty and can be generalized to various settings.
Study uses machine learning to estimate effective policies in settings with hidden individual actions.
problem Estimating effective policies in settings with hidden individual actions.
method Instrumental Regression and Generalized Method of Moments (GMM) estimator.
result Demonstrates how to estimate a good contract in principal-agent problems.
The paper develops methods to estimate optimal treatment sequences under policy constraints.
problem Estimating the best sequence of treatments over multiple stages for individuals.
method Empirical welfare maximization approach, solving treatment assignment sequentially or simultaneously.
result Established convergence rates and upper bounds for estimation methods.
Optimizes COVID-19 testing policy using a Multi-Armed Bandit approach.
problem Balancing discovery of positive cases with population surveillance.
method Risk scoring and random sampling based on Multi-Armed Bandit theory.
result Effective prioritization captures 65-92% of positive cases with varying testing capacity.
Proposes a framework to create fair IDRs by enforcing demographic parity constraints.
problem Discrimination in IDRs trained on biased data.
method Incorporates DP and CDP constraints into IDR estimation.
result Theoretically optimal IDRs can be efficiently obtained through perturbations.
The paper tackles fair policy targeting by optimizing allocation rules to minimize unfairness.
problem Discrimination in individualized treatments of social welfare programs.
method Formulated as a mixed-integer linear program, solved using off-the-shelf algorithms, derived regret bounds and small sample guarantees.
result Designs fair and efficient treatment allocation rules within the Pareto frontier.
New algorithm learns policies without uniform overlap assumption.
problem Learning optimal policies from non-uniformly collected data.
method Pessimistic Policy Learning (PPL) using lower confidence bounds.
result Efficient policy learning for adaptively collected data.
Study optimal retirement time and consumption with habitual persistence.
problem Understanding retirement consumption patterns with habitual persistence.
method Established concise habitual evolution, used martingale and duality methods.
result Optimal consumption declines sharply at retirement but excess consumption increases.
Discuss new policy learning objectives and methods.
problem Improving policy learning efficiency and robustness.
method Introducing curvature considerations and calibration data methods.
result Efficient retargeting and distributionally robust policies.
New method optimizes treatment policies to avoid winner's curse.
problem Winner's curse in treatment policy optimization.
method Inference-aware policy optimization.
result Optimizes for both estimated performance and downstream evaluation.
FRONT optimizes decisions with interference, reducing regret over time.
problem Short-sighted policies in online decision-making due to ignoring interference.
method FRONT considers long-term impacts of decisions, using exploratory and exploitative strategies.
result FRONT achieves sublinear regret in both immediate and consequential impacts.
Modern vehicle fleets, e.g., for ridesharing platforms and taxi companies, can reduce passengers' waiting times by proactively dispatching vehicles to locations where pickup requests are anticipated in the future. Yet it is unclear how to best do this: optimal dispatching requires optimizing over several sources of unc…
New algorithm reduces individual regret and communication costs in cooperative bandits.
problem Optimal individual and group regret in cooperative multi-agent bandits.
method Integrates a new communication policy into a learning algorithm.
result Achieves optimal individual regret and constant communication costs.
The paper learns personalized treatment rules from observational data.
problem Developing effective treatment policies for individual patients.
method Contextual bandit approach to minimize expected risk of treatment policies.
result The proposed method outperforms physicians and baseline approaches in IV and VP administration.
Policy learning can be used to extract individualized treatment regimes from observational data in healthcare, civics, e-commerce, and beyond. One big hurdle to policy learning is a commonplace lack of overlap in the data for different actions, which can lead to unwieldy policy evaluation and poorly performing learned …
As data-driven predictive models are increasingly used to inform decisions, it has been argued that decision makers should provide explanations that help individuals understand what would have to change for these decisions to be beneficial ones. However, there has been little discussion on the possibility that individu…
ESOP uses Bayesian optimization to find optimal lock-down schedules.
problem Finding optimal lock-down schedules balancing health and economy.
method Bayesian optimization interacting with epidemiological models.
result ESOP schedules balance public health and economic impacts.
This paper focuses on stochastic orders and its applications : policy limits and deductibles. Further, many applications and some examples are given : comparison of two families of copulas, individual and collective risk model, reinsurance contracts and dependent portfolios increase risk. More precisely, we propose a n…
This paper proposes a definition of system health in the context of multiple agents optimizing a joint reward function. We use this definition as a credit assignment term in a policy gradient algorithm to distinguish the contributions of individual agents to the global reward. The health-informed credit assignment is t…
We provide a comparative study of several widely used off-policy estimators (Empirical Average, Basic Importance Sampling and Normalized Importance Sampling), detailing the different regimes where they are individually suboptimal. We then exhibit properties optimal estimators should possess. In the case where examples …
Introduces LTQL for factored policies in cooperative MARL.
problem Learning optimal joint policies in collaborative MARL scenarios.
method Logical Team Q-learning (LTQL) as a stochastic approximation to dynamic programming.
result LTQL provides factored policies for optimal joint behavior in cooperative MARL.
Proposes a new reinforcement learning algorithm using Q-function.
problem Optimal control in Markov Decision Processes (MDPs).
method Regularized linear-programming formulation, Q-function, saddle-point optimization.
result Demonstrates effectiveness on various benchmark problems.
Algorithm improves learning by integrating diverse agents' behaviors.
problem Lack of social learning in reinforcement learning algorithms.
method Free energy approach for social bandit learning.
result Algorithm converges to optimal policy and enhances learning.
Study optimal and equitable encouragement policies for treatment adherence.
problem Optimal treatment adherence policies in the presence of human non-adherence.
method Covariate-conditional no-direct-effect model of encouragement; tractable policy characterizations under constraints.
result Induced treatment take-up is the fairness target, not recommendation rates.
A new policy switching technique improves offline RL performance.
problem Challenges in adapting off-policy algorithms to different datasets and tasks.
method Combines off-policy RL and BC, using epistemic uncertainty for policy switching.
result Outperforms individual algorithms and state-of-the-art methods on benchmarks.
In treatment allocation problems the individuals to be treated often arrive sequentially. We study a problem in which the policy maker is not only interested in the expected cumulative welfare but is also concerned about the uncertainty/risk of the treatment outcomes. At the outset, the total number of treatment assign…
Paper introduces privacy-preserving inventory policy learning for feature-based newsvendor with unknown demand.
problem Privacy-preserving inventory policy learning for feature-based newsvendor with unknown demand distribution and nonsmooth loss function.
method Developed a clipped noisy gradient descent algorithm based on convolution smoothing for optimal inventory estimation within f-differential privacy framework.
result Achieved privacy-preserving optimal inventory policy with provable privacy guarantees and desirable statistical precision.
The paper develops deep learning models for personalized treatment rules in survival analysis.
problem Deriving optimal treatment rules for bivariate survival outcomes in randomized trials.
method Adaptive prediction-powered learning using deep neural networks and stochastic policies.
result Maximizes joint survival probability beyond fixed time points (t1,t2). Deep reinforcement learning finds optimal learning policies for adaptive systems.
problem Finding individualized learning plans for learners with unknown latent traits.
method Formulated as a Markov decision process, applied deep Q-learning with a transition model estimator.
result The algorithm efficiently discovers optimal learning policies with small data sets.
Adapting policy learning for data collected from evolving systems.
problem Challenges in learning optimal policies from adaptively collected data.
method Proposes an algorithm based on generalized augmented inverse propensity weighted (AIPW) estimators to control worst-case estimation variance.
result Achieves minimax rate optimal regret guarantees even with diminishing exploration.
The paper develops a method to learn cost-optimal sequential testing policies from retrospective data.
problem Learning cost-optimal sequential decision policies from retrospective data with missing test results.
method Doubly robust Q-learning framework with path-specific inverse probability weights.
result The method reduces testing cost without compromising predictive accuracy.
Framework integrates mental disorder measurements for personalized treatment.
problem Optimizing treatment for mental disorders with latent mental states and heterogeneity.
method Measurement theory and multi-layer neural network for complex treatment effects.
result Learned treatment policies outperform alternatives on heterogeneous treatment effects.
The paper introduces SuccessProbaMax to optimize policy success probability in online advertising.
problem Optimizing policy success probability in online advertising systems.
method SuccessProbaMax algorithm that optimizes for the probability of success rather than expected value.
result SuccessProbaMax outperforms conventional algorithms in terms of success rate.
Reduces variance in noisy social outcomes to improve policy evaluation and optimization.
problem Improving access to opportunity through personalized treatment decisions.
method Data-driven dimensionality-reduction using reduced rank regression to denoise multiple outcomes.
result Improves estimation error in policy evaluation and optimization, including on real-world data.
A new algorithm learns policies from batch data in hierarchical RL.
problem Learning policies from fixed batches of data without full exploration.
method Modeling RL as a two-player game with a leader-follower structure, proposing StackelbergLearner.
result StackelbergLearner achieves competitive performance in batch RL and real-world datasets.
Optimal investment and consumption model with habit formation constraint.
problem Formulating an optimal investment and consumption model with habit formation constraint.
method Formulated an infinite-horizon optimal investment and consumption problem with habit formation model, derived explicit policies, and analyzed the system of differential equations.
result Optimal investment and consumption policies derived explicitly, showing different consumption and investment strategies based on habit formation level.
Framework optimizes targeting high-need individuals while estimating treatment effects.
problem Balancing resource allocation to high-need individuals with evaluating treatment effects.
method Proposes a framework to design randomized allocation rules that balance targeting high-need recipients with learning treatment effects.
result Optimized policies can significantly mitigate the tradeoff between targeting high-need individuals and estimating treatment effects.
Proposes robust ITRs integrating multiple datasets to handle posterior shift.
problem Posterior shift in conditional outcome distributions between source and target populations.
method Distributionally robust approach with closed-form solution and adaptive uncertainty tuning.
result Achieves superior performance compared to existing methods in simulations and real-data applications.
PROWL uses robust reward estimates to improve ITR selection.
problem Reward uncertainty in ITR estimation leads to inflated performance.
method PAC-Bayesian framework with reward uncertainty certificates.
result PROWL achieves better robust treatment regime estimation.
New algorithm optimizes interventions under network interference, scaling to large networks.
problem Optimal policy learning under network interference where one individual's treatment affects others.
method Developed a scalable Thompson sampling algorithm for dynamic networks.
result Proved a Bayesian regret bound that is sublinear in network size and rounds.
Due to the recent advancements in wearables and sensing technology, health scientists are increasingly developing mobile health (mHealth) interventions. In mHealth interventions, mobile devices are used to deliver treatment to individuals as they go about their daily lives. These treatments are generally designed to im…
Paper proposes a new framework for individualized treatment rules that generalize better across different distributions.
problem Existing individualized treatment rules may not generalize well when training and testing distributions differ.
method Distributionally robust individualized treatment rules (DR-ITR) framework that maximizes worst-case value function across close distributions.
result Calibrated DR-ITR outperforms standard ITR in generalizability.