arXiv research

This work explains why online imitation learning improves faster than theory predicts.

New method learns decisions from collective preferences without individual covariates.

Develops a method to estimate optimal policy value in online learning.

Monte Carlo Tree Search (MCTS) algorithms perform simulation-based search to improve policies online. During search, the simulation policy is adapted to explore the most promising lines of play. MCTS has been used by state-of-the-art programs for many problems, however a disadvantage to MCTS is that it estimates the va…

KL-regularized RL from expert demos can lead to slow, unstable learning.

Study online learning with off-policy feedback in adversarial bandit problems.

Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…

In real-world machine learning applications, there is a cost associated with sampling of different features. Budgeted learning can be used to select which feature-values to acquire from each instance in a dataset, such that the best model is induced under a given constraint. However, this approach is not possible in th…

New policy for non-stationary multi-armed bandits with recovering rewards.

RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.

This paper explores how imperfect reward models can improve online RLHF.

Greedy policy maximizes information in unknown linear systems.

New algorithm reduces decision switching in dynamic environments.

New RL method learns from state transitions without actions.

New RL algorithm tackles online robust MDPs with uncertainty.

In this paper, we develop a multi-agent reinforcement learning (MARL) framework to obtain online power control policies for a large energy harvesting (EH) multiple access channel, when only causal information about the EH process and wireless channel is available. In the proposed framework, we model the online power co…

A new policy switching technique improves offline RL performance.

New algorithm for sequential off-policy learning improves performance over batch methods.

Offline RL tackles resource-constrained online deployment with improved policy transfer.

New algorithms reduce dynamic regret in online MDPs with changing losses.

DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.

New model-based methods adapt pre-trained policies to unseen environments efficiently.

New control methods for systems with adversarial perturbations.

The paper introduces SuccessProbaMax to optimize policy success probability in online advertising.

Paper tackles policy selection with logged data and limited online interactions.

Sample efficiency is critical in solving real-world reinforcement learning problems, where agent-environment interactions can be costly. Imitation learning from expert advice has proved to be an effective strategy for reducing the number of interactions required to train a policy. Online imitation learning, which inter…

We consider a context-based dynamic pricing problem of online products, which have low sales. Sales data from Alibaba, a major global online retailer, illustrate the prevalence of low-sale products. For these products, existing single-product dynamic pricing algorithms do not work well due to insufficient data samples.…

Online Apprenticeship Learning aims to match expert performance without access to cost functions.

Model-based reinforcement learning (MBRL) with model-predictive control or online planning has shown great potential for locomotion control tasks in terms of both sample efficiency and asymptotic performance. Despite their initial successes, the existing planning methods search from candidate sequences randomly generat…

The Knowledge Gradient (KG) policy was originally proposed for online ranking and selection problems but has recently been adapted for use in online decision making in general and multi-armed bandit problems (MABs) in particular. We study its use in a class of exponential family MABs and identify weaknesses, including …

This paper bridges offline and online RL by studying policy finetuning with a reference policy.

In this paper, we propose a novel framework for approximating the explicit MPC law for linear parameter-varying systems using supervised learning. In contrast to most existing approaches, we not only learn the control policy, but also a "certificate policy", that allows us to estimate the sub-optimality of the learned …

A3RL combines online and offline RL with active sampling to improve policy learning.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

New method optimizes policies in non-stationary environments.

New algorithm reduces policy regret in tallying bandits.

This work proposes robust reinforcement learning methods using both offline and online data.

A new method combines online and offline learning to tackle contextual bandits with missing action support.

Unified Bayesian framework for efficient off-policy evaluation and learning in large action spaces.

Reinforcement learning improves online matching by combining expert policies.

Benchmarking off-policy evaluation methods for complex policies.

Value aggregation is a general framework for solving imitation learning problems. Based on the idea of data aggregation, it generates a policy sequence by iteratively interleaving policy optimization and evaluation in an online learning setting. While the existence of a good policy in the policy sequence can be guarant…

New technique reduces imitation learning performance gap in finite samples.

It is well known that the historical logs are used for evaluating and learning policies in interactive systems, e.g. recommendation, search, and online advertising. Since direct online policy learning usually harms user experiences, it is more crucial to apply off-policy learning in real-world applications instead. Tho…

Paper optimizes GAIL for online and offline learning with linear approximations.

Extends OPE to evaluate policies using diverse logging data.

The paper tackles robust reinforcement learning with performance guarantees.

This paper investigates the problem of online prediction learning, where learning proceeds continuously as the agent interacts with an environment. The predictions made by the agent are contingent on a particular way of behaving, represented as a value function. However, the behavior used to select actions and generate…