The paper develops methods to estimate optimal treatment sequences under policy constraints.
problem Estimating the best sequence of treatments over multiple stages for individuals.
method Empirical welfare maximization approach, solving treatment assignment sequentially or simultaneously.
result Established convergence rates and upper bounds for estimation methods.
Method learns optimal treatment sequences from observational data.
problem Optimal dynamic treatment regimes for public policies and medical interventions.
method Doubly robust classification-based approach via backward induction.
result Achieves optimal convergence rate of n^(-1/2) for welfare regret.
Dynamic treatment strategies on networks amplify policy impact through spillovers.
problem Effective dynamic treatment allocation in network settings.
method Q-Ising, a three-stage pipeline integrating Bayesian dynamic Ising model, treatment adoption histories, and offline reinforcement learning.
result Adaptive targeting outperforms static centrality benchmarks in Indian village microfinance networks and synthetic data.
Study optimal treatment assignment policies under strategic agent responses.
problem Learning optimal treatment policies with strategic agents complicates estimation.
method Dynamic model with threshold convergence to mean-field equilibrium, consistent estimator for policy gradient.
result Threshold for treatment assignment converges to mean-field equilibrium threshold under large but finite number of agents.
Q-learning adapted to find near-equivalent treatment strategies.
problem Finding optimal treatment sequences in dynamic treatment regimes.
method Introducing a worst-value tolerance criterion to find sets of near-equivalent policies.
result Constructs families of near-equivalent treatment strategies.
Estimates and infers multi-stage stationary treatment policies with variable selection.
problem Valid inference for multi-stage stationary treatment policies with high-dimensional feature variables.
method Estimate the value function using augmented inverse probability weighted estimator, apply penalty for variable selection, construct one-step improvements for valid inference.
result Improved estimators are asymptotically normal, valid inference for policy parameters demonstrated.
POLAR optimizes treatment strategies in dynamic settings with statistical guarantees.
problem Optimizing sequential decisions in dynamic treatment regimes with robustness and statistical guarantees.
method Pessimistic model-based approach estimating transition dynamics and incorporating uncertainty penalties.
result Offers statistical and computational guarantees, including finite-sample bounds on policy suboptimality.
The paper introduces a new method for estimating optimal policies in dynamic treatment regimes using information geometry.
problem Estimating optimal policies in dynamic treatment regimes.
method Minimum information divergence method based on γ-power divergence. result The γ-power divergence method effectively seeks the optimal policy by vanishing the divergence between policy-equivalent Q-functions. Estimates RL data for dynamic treatment effects using GMM.
problem Estimating dynamic treatment effects from RL data with nonstationary behavior policies.
method Weighted GMM approach to stabilize variance in adaptive RL settings.
result Valid hypothesis testing and confidence regions for dynamic treatment effects.
EHR-MPC optimizes sepsis treatment using digital twins and inference-time control.
problem Optimal sepsis treatment policies are contested and difficult to adapt during inference.
method EHR-MPC decouples learning patient dynamics from treatment optimization, enabling inference-time control over learned digital twins.
result EHR-MPC achieves comparable off-policy performance and improved simulation performance compared to RL baselines.
Proposes pT-Learning for optimal dynamic treatment regimes in mHealth.
problem Challenges in learning optimal dynamic treatment regimes with large intervention options and infinite time horizon.
method Proximal Temporal consistency Learning (pT-Learning) framework for adaptively adjusting between deterministic and stochastic policies.
result Minimax estimator avoids double sampling issue and can incorporate off-policy data.
Many applied decision-making problems have a dynamic component: The policymaker needs not only to choose whom to treat, but also when to start which treatment. For example, a medical doctor may choose between postponing treatment (watchful waiting) and prescribing one of several available treatments during the many vis…
Method estimates dynamic treatment effects using machine learning and g-estimation.
problem Estimating treatment effects over time with multiple treatments and potential future outcomes.
method Double/debiased machine learning framework for dynamic treatment effects, extending Neyman orthogonal cross-fitted g-estimation. result Provides finite sample guarantees and allows for non-linear effect heterogeneity and high-dimensional parameterizations.
Study optimal adjustment sets for causal policies with hidden variables.
problem Estimating dynamic treatment regimes with hidden variables.
method Developed criteria for graphs without hidden variables to compare estimators, extended to dynamic policies and hidden variables.
result Existence and computation of optimal minimal and globally optimal adjustment sets.
Estimates long-term effects of new treatments using historical and short-term data.
problem Estimating long-term effects of novel treatments with limited historical data.
method Surrogate indices, dynamic treatment effect estimation, and double machine learning combined in a unified pipeline.
result Consistent and asymptotically normal estimates of long-term effects under Markovian assumption.
Deep Bayesian models estimate causal effects for dynamic treatment regimes over long follow-up times.
problem Challenges in causal effect estimation for dynamic treatment regimes with long follow-up times.
method Combining outcome regression models with deep Bayesian models for high-dimensional features.
result Stable and accurate dynamic causal effect estimation from observational data, especially with long-term follow-up.
The paper proposes a new policy for optimal treatment allocation based on quantile treatment effects.
problem Optimal treatment allocation policies that target distributional welfare, especially when individuals are heterogeneous.
method The approach involves allocating treatments based on the conditional quantile of individual treatment effects (QoTE), considering both prudent and negligent policymakers.
result The proposed minimax policies are robust to model uncertainty and can be generalized to various settings.
Three approaches learn personalized treatment policies for UTI patients.
problem Learning optimal treatment policies in multiobjective settings with fully observed outcomes.
method Indirect and direct approaches using predictive models and without intermediate models.
result All approaches outperform clinicians in achieving better performance on all outcomes and trade-offs.
Efficiently find near-optimal medical treatments with less trial and error.
problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.
Comment on entropy learning for dynamic treatment regimes.
problem Evaluating dynamic treatment regimes using entropy loss.
method Optimization-based alternative to IPW estimate.
result Suggests optimization-based approach for evaluation.
Proposes Deep LTMLE for estimating dynamic treatment effects in longitudinal studies.
problem Estimating counterfactual mean outcomes under dynamic treatment policies in longitudinal settings.
method Uses a transformer architecture with temporal-difference learning for initial estimation, followed by TMLE correction and statistical inference.
result Demonstrates superior performance in complex, long-term scenarios compared to existing methods.
Develops framework for estimating and improving DTRs with time-varying IV in the presence of unmeasured confounding.
problem Estimating DTRs from observational data with unmeasured confounding.
method Time-varying instrumental variable (IV) framework for estimating and improving DTRs.
result IV-optimal and IV-improved DTRs perform better than DTRs assuming no unmeasured confounding.
Proposes a new method to estimate continuous treatment policies and match treatments effectively.
problem Current methods struggle with continuous treatment policies and complex matching.
method Formulates treatment effectiveness as a parametrizable model, using deep learning for optimization.
result Significant improvement in treatment effectiveness and matching efficiency.
Improves A/B testing for long-term outcomes in dynamic systems.
problem Estimating long-term effects from short-term A/B testing data.
method Develops optimal inference techniques and localized information sharing methods.
result New estimator reduces variance linearly with test arms and matches lower bounds.
New method for finding optimal treatment regimes in medical settings with time-varying unobserved factors.
problem Finding optimal treatment regimes in medical settings with time-varying unobserved factors.
method Extend Dynamic Treatment Regimes (DTRs) to Ambiguous Dynamic Treatment Regimes (ADTRs), connect to Ambiguous Partially Observable Mark Decision Processes (APOMDPs), and develop Reinforcement Learning methods.
result Established theoretical results for learning methods, including consistency and asymptotic normality.
Proposes a new Bayesian learning method for optimal treatment regimes.
problem Sub-optimal policies in offline data due to lack of exploration.
method Integrates pessimism principle with Thompson sampling and Bayesian machine learning.
result Derives a credible set that uniformly lower bounds the optimal Q-function.
AACE learns treatment policies from EHRs using annotations to improve accuracy.
problem Learning treatment policies from multimodal EHRs with bias and inefficiency.
method Annotation-assisted coarsened effects (AACE) method.
result AACE outperforms existing methods in predicting treatment benefit from multimodal EHRs.
Study optimal and equitable encouragement policies for treatment adherence.
problem Optimal treatment adherence policies in the presence of human non-adherence.
method Covariate-conditional no-direct-effect model of encouragement; tractable policy characterizations under constraints.
result Induced treatment take-up is the fairness target, not recommendation rates.
We study the problem of policy evaluation and learning from batched contextual bandit data when treatments are continuous, going beyond previous work on discrete treatments. Previous work for discrete treatment/action spaces focuses on inverse probability weighting (IPW) and doubly robust (DR) methods that use a reject…
The goal of personalized decision making is to map a unit's characteristics to an action tailored to maximize the expected outcome for that unit. Obtaining high-quality mappings of this type is the goal of the dynamic regime literature. In healthcare settings, optimizing policies with respect to a particular causal pat…
In both the fields of computer science and medicine there is very strong interest in developing personalized treatment policies for patients who have variable responses to treatments. In particular, I aim to find an optimal personalized treatment policy which is a non-deterministic function of the patient specific cova…
The paper learns personalized treatment rules from observational data.
problem Developing effective treatment policies for individual patients.
method Contextual bandit approach to minimize expected risk of treatment policies.
result The proposed method outperforms physicians and baseline approaches in IV and VP administration.
Paper derives policy rules from observational data for hepatitis C treatment.
problem Improving treatment guidelines for HIV/HCV co-infected patients.
method Weighted K-means algorithm for estimating CATEs, decision tree implementation.
result Identifies a subgroup with high spontaneous HCV clearance rate.
Develops deep jump learning for continuous treatment OPE.
problem Estimating mean outcomes under new treatment rules using historical data from different rules.
method Adaptive deep discretization of continuous treatment space using deep learning and multi-scale change point detection.
result Validated method through theoretical results, simulations, and real application to Warfarin Dosing.
Proposes a fusion method for many treatment groups in ITRs.
problem Challenges in handling many treatment groups with data sparsity and covariate imbalance.
method Calibration-weighted treatment fusion procedure that balances covariates and fuses similar treatments.
result Ensures robust treatment group recovery and policy value compared to existing methods.
Algorithm learns interference network and optimizes treatment allocation for unknown network effects.
problem Adaptive experimentation under unknown network interference.
method Thompson sampling algorithm with Gibbs sampler for joint learning of interference network and treatment allocation.
result Proves a Bayesian regret bound and achieves sublinear regret in real-world applications.
Proposes fair and robust methods for estimating treatment effects.
problem Estimating treatment effects while maintaining fairness.
method Simple, nonparametric framework with fairness constraints.
result Estimators are double robust and characterize welfare trade-offs.
In treatment allocation problems the individuals to be treated often arrive sequentially. We study a problem in which the policy maker is not only interested in the expected cumulative welfare but is also concerned about the uncertainty/risk of the treatment outcomes. At the outset, the total number of treatment assign…
Paper evaluates dynamic QTE for ridesharing data.
problem Assessing QTE in ridesharing with skewed outcomes.
method Developed VCDP models to estimate dynamic CQTE.
result Dynamic CQTE equals sum of individual CQTEs.
Study optimizes experimental design for best treatment arm identification.
problem Identifying the best treatment arm given contextual information.
method Adaptive Sampling-Policy Learning (PLAS) strategy for minimax rate optimality.
result PLAS strategy achieves minimax rate optimality in expected simple regret.
The paper proposes an interpretable off-policy learning algorithm for medical treatments.
problem Lack of interpretable methods for personalized treatment decisions from observational data.
method Hyperbox search approach for interpretable policies in disjunctive normal form.
result The proposed algorithm outperforms state-of-the-art methods in terms of regret and is rated highly interpretable by clinical experts.
This work addresses causal inference challenges in networked interference and proposes GNN-based estimators for individual treatment effects.
problem Estimating individual treatment effects in randomized experiments with networked interference.
method Uses Graph Neural Networks (GNNs) to capture network dependencies and derive causal effect estimators.
result Provides policy regret bounds and heuristic error bounds for GNN-based causal estimators under network interference and treatment capacity constraints.
Proposes a new decision rule for continuous treatments.
problem Developing personalized treatment recommendations for continuous treatments.
method Jump interval-learning method to estimate conditional mean of outcomes.
result Optimal interval-valued decision rule (I2DR) for continuous treatments.
End-to-end policy learning method improves CATE estimation.
problem Learning optimal treatment policies from partially observed data.
method Modified causal forest for policy learning.
result Maximizing policy value is equivalent to minimizing CATE.
New framework estimates treatment effects based on preferences.
problem Estimating treatment effects with flexible outcomes.
method Preference-based Conditional Treatment Effect (CPTE) framework.
result CPTE provides interpretable targets and new identifiability conditions.
System interprets complex treatment effects for personalized policies.
problem Complex, hard-to-understand treatment effect models.
method Scalable, interpretable personalized experimentation system.
result Learned explanations and generated interpretable policies.
Estimates personalized policies robust to shifts in target populations.
problem Estimating policies that perform well in diverse target populations.
method Develops methods for estimating robust policies considering shifts in outcomes and characteristics.
result Welfare-maximizing policies are robust to certain shifts in potential outcomes.
The paper addresses treatment recommendation problems by optimizing distributional characteristics.
problem Optimizing treatment recommendations based on distributional targets.
method Characterizes the problem's difficulty and proposes near-optimal policies.
result Characterizes the difficulty of the problem and proposes near-regret optimal policies.