arXiv research

Designs a single policy for collecting data to train near-optimal policies.

A single policy suffices for near-optimal parallel exploration in RL.

AE-LSVI identifies near-optimal policies in complex systems with minimal data.

Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.

We decode latent states in Block MDPs and learn near-optimal policies.

Standard reinforcement learning methods aim to master one way of solving a task whereas there may exist multiple near-optimal policies. Being able to identify this collection of near-optimal policies can allow a domain expert to efficiently explore the space of reasonable solutions. Unfortunately, existing approaches t…

We consider a problem of learning the reward and policy from expert examples under unknown dynamics. Our proposed method builds on the framework of generative adversarial networks and introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies. Empowe…

New algorithm finds near-optimal policies efficiently in zero-sum games.

New method learns near-optimal policies with polynomial samples in A and H.

Proposes method to discover diverse near-optimal policies in reinforcement learning.

Temporal difference learning and Residual Gradient methods are the most widely used temporal difference based learning algorithms; however, it has been shown that none of their objective functions is optimal w.r.t approximating the true value function $V$. Two novel algorithms are proposed to approximate the true value…

Learning near-optimal behaviour from an expert's demonstrations typically relies on the assumption that the learner knows the features that the true reward function depends on. In this paper, we study the problem of learning from demonstrations in the setting where this is not the case, i.e., where there is a mismatch …

Paper proposes a sequential statistical test for comparing imitation learning policies with near-optimal stopping.

Paper addresses offline policy evaluation in RL, achieving near-optimal bounds for various policy classes.

This paper introduces a new metric, ULI, for RL that ensures both cumulative and instantaneous performance.

StepMix algorithm ensures safe exploration in reinforcement learning with near-optimal performance.

New bounds for learning near-optimal policies in CMDPs with constraints.

New algorithm identifies near-optimal policies in adversarial distributed RL settings.

Deep RL learns 2-opt heuristics to improve TSP solutions.

GN algorithm solves batched bandit for nondegenerate functions near-optimally.

Paper tackles offline CMDP problems with near-optimal algorithm and sample complexity bound.

New algorithm learns optimal policies without explicit rewards.

Exploration is widely regarded as one of the most challenging aspects of reinforcement learning (RL), with many naive approaches succumbing to exponential sample complexity. To isolate the challenges of exploration, we propose a new "reward-free RL" framework. In the exploration phase, the agent first collects trajecto…

Adaptive reduction scheme approximates optimal policy in regularized MDPs.

This paper bridges offline and online RL by studying policy finetuning with a reference policy.

New algorithm reduces switching costs in RL beyond linear MDPs.

We study Imitation Learning (IL) from Observations alone (ILFO) in large-scale MDPs. While most IL algorithms rely on an expert to directly provide actions to the learner, in this setting the expert only supplies sequences of observations. We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Lea…

New RL algorithms reduce costs for single-agent and federated learning.

We present the first PAC optimal algorithm for Bayes-Adaptive Markov Decision Processes (BAMDPs) in continuous state and action spaces, to the best of our knowledge. The BAMDP framework elegantly addresses model uncertainty by incorporating Bayesian belief updates into long-term expected return. However, computing an e…

Efficiently find near-optimal medical treatments with less trial and error.

One of the key approaches to save samples in reinforcement learning (RL) is to use knowledge from an approximate model such as its simulator. However, how much does an approximate model help to learn a near-optimal policy of the true unknown model? Despite numerous empirical studies of transfer reinforcement learning, …

A collaborative algorithm reduces regret in federated linear contextual bandits.

We study the problem of estimating the expected reward of the optimal policy in the stochastic disjoint linear bandit setting. We prove that for certain settings it is possible to obtain an accurate estimate of the optimal policy value even with a number of samples that is sublinear in the number that would be required…

Paper establishes first instance-dependent lower bound for PAC reinforcement learning.

Restless bandit problems are instances of non-stationary multi-armed bandits. These problems have been studied well from the optimization perspective, where the goal is to efficiently find a near-optimal policy when system parameters are known. However, very few papers adopt a learning perspective, where the parameters…

A method to learn robust policies for environments with model mismatches.

We study the problem of multi-agent reinforcement learning (MARL) with adaptivity constraints -- a new problem motivated by real-world applications where deployments of new policies are costly and the number of policy updates must be minimized. For two-player zero-sum Markov Games, we design a (policy) elimination base…

In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle. We study the effectiveness of the near-optimal cost-to-go oracle on the planning horizon and demonstrate that the cost-to-go oracle shortens the learner's planning horizon as function of its accurac…

New approach improves stock policies for paper companies, reducing waste and costs.

Study optimizes portfolio allocation policies using off-policy data and constraints.

Paper tackles RLHF with DCPPO method, proving near-optimal suboptimality.

Any reinforcement learning algorithm that applies to all Markov decision processes (MDPs) will suffer $Ω(\sqrt{SAT})$ regret on some MDP, where $T$ is the elapsed time and $S$ and $A$ are the cardinalities of the state and action spaces. This implies $T = Ω(SA)$ time to guarantee a near-optimal policy. In many settings…

In the Bayesian approach to sequential decision making, exact calculation of the (subjective) utility is intractable. This extends to most special cases of interest, such as reinforcement learning problems. While utility bounds are known to exist for this problem, so far none of them were particularly tight. In this pa…

New method optimizes policies in non-stationary environments.

New algorithms reduce dynamic regret in non-stationary RL environments.

In this paper, we present a new approach to Transfer Learning (TL) in Reinforcement Learning (RL) for cross-domain tasks. Many of the available techniques approach the transfer architecture as a method of speeding up the target task learning. We propose to adapt and reuse the mapped source task optimal-policy directly …

Post-training optimizes model performance beyond base model limits.

Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.