arXiv research

Study designs logging policies to minimize off-policy evaluation error.

Extends OPE to evaluate policies using diverse logging data.

A new estimator for evaluating policies in unknown environments.

Develops an anytime-valid framework for optimal policy identification from logged contextual bandit data.

Proposes log density gradient to improve reinforcement learning sample complexity.

Paper tackles policy selection with logged data and limited online interactions.

A new method combines online and offline learning to tackle contextual bandits with missing action support.

Paper finds efficient OPE estimator for multiple logging policies with minimum variance.

Study on policy gradient for stochastic bandits using diffusion approximation.

When learning from a batch of logged bandit feedback, the discrepancy between the policy to be learned and the off-policy training data imposes statistical and computational challenges. Unlike classical supervised learning and online learning settings, in batch contextual bandit learning, one only has access to a colle…

Novel LSE estimator improves off-policy learning and evaluation.

DOLCE improves off-policy evaluation and learning by decomposing effects.

Paper proposes a new DR estimator for adaptive experiments with improved performance.

We consider active learning with logged data, where labeled examples are drawn conditioned on a predetermined logging policy, and the goal is to learn a classifier on the entire population, not just conditioned on the logging policy. Prior work addresses this problem either when only logged data is available, or purely…

New method selects best offline RL policies from logged data.

Study improves unbiased recommender learning by addressing missing-reward bias.

Develops a support-aware framework for reserve-policy selection in advertising markets.

Backward exploration reduces sample complexity in policy evaluation.

New RL algorithm reduces policy switching cost to loglog(T) with similar regret.

New estimator reduces risk in slate bandits by leveraging Bayes risk criterion.

New estimator uses clustering to improve off-policy evaluation accuracy.

Study nonparametric estimator for Markov chain transition matrices in offline setting.

Study evaluates new models using human feedback from another model.

Develops CLTs for Markov chain transition probabilities and policies.

This work highlights problems with off-policy estimation in recommender systems due to unobserved confounders.

Industrial recommender systems deal with extremely large action spaces -- many millions of items to recommend. Moreover, they need to serve billions of users, who are unique at any point in time, making a complex user state space. Luckily, huge quantities of logged implicit feedback (e.g., user clicks, dwell time) are …

Paper proposes a framework for reliable off-policy evaluation in reinforcement learning.

We study the problem of multi-agent reinforcement learning (MARL) with adaptivity constraints -- a new problem motivated by real-world applications where deployments of new policies are costly and the number of policy updates must be minimized. For two-player zero-sum Markov Games, we design a (policy) elimination base…

wd1 improves reasoning in dLLMs by optimizing policies without policy ratios.

Enhances reinforcement learning from sparse data.

Learning policies on data synthesized by models can in principle quench the thirst of reinforcement learning algorithms for large amounts of real experience, which is often costly to acquire. However, simulating plausible experience de novo is a hard problem for many complex environments, often resulting in biases for …

Efficiently identifies best policies in tabular MDPs with reduced computational cost.

Adaptive sequential decision making is one of the central challenges in machine learning and artificial intelligence. In such problems, the goal is to design an interactive policy that plans for an action to take, from a finite set of $n$ actions, given some partial observations. It has been shown that in many applicat…

We study the sample complexity of approximate policy iteration (PI) for the Linear Quadratic Regulator (LQR), building on a recent line of work using LQR as a testbed to understand the limits of reinforcement learning (RL) algorithms on continuous control tasks. Our analysis quantifies the tension between policy improv…

Study on adaptivity constraints in linear contextual bandits with optimal design.

Improves policy optimization with polylog(T) regret bounds for stochastic losses.

We study the problem of offline learning in automated decision systems under the contextual bandits model. We are given logged historical data consisting of contexts, (randomized) actions, and (nonnegative) rewards. A common goal is to evaluate what would happen if different actions were taken in the same contexts, so …

Two algorithms address limited adaptivity in generalized linear contextual bandits.

RWR converges to global optimum in certain settings.

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

A scalable framework optimizes multi-asset portfolios with constraints.

Contextual bandit methods fail with deficient support data.

The ability to perform offline A/B-testing and off-policy learning using logged contextual bandit feedback is highly desirable in a broad range of applications, including recommender systems, search engines, ad placement, and personalized health care. Both offline A/B-testing and off-policy learning require a counterfa…

We study the pricing problem faced by a firm that sells a large number of products, described via a wide range of features, to customers that arrive over time. Customers independently make purchasing decisions according to a general choice model that includes products features and customers' characteristics, encoded as…

The paper tackles counterfactual learning for stochastic policies with continuous actions.

Optimizes LLM prompts using logged user feedback.

Counterfactual learning is a natural scenario to improve web-based machine translation services by offline learning from feedback logged during user interactions. In order to avoid the risk of showing inferior translations to users, in such scenarios mostly exploration-free deterministic logging policies are in place. …

New actor-critic algorithm achieves optimal sample efficiency in RL.