arXiv research

Accelerates policy optimization in RL with optimistic and adaptive updates.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

New method minimizes regret in AMDP with high probability.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

New algorithm reduces regret in sequential decision-making problems.

Develops ODRPO to improve RL algorithms with better performance and stability.

In this paper, we prove some convergence results of a special case of optimistic policy iteration algorithm for stochastic shortest path problem. We consider both Monte Carlo and $TD(λ)$ methods for the policy evaluation step under the condition that the termination state will eventually be reached almost surely.

Optimistic actor-critic tackles linear MDPs with parametric policies.

Optimistic search speeds up change point detection in large datasets.

Optimistic PPO variant solves linear MDPs with improved regret bound.

Bayesian classifier improves robustness with optimistic score ratio.

Optimistic method adapted for faster convex-concave min-max problems.

PROPO tackles non-stationary MDPs with efficient policy optimization.

New RL algorithm optimizes policies with bandit feedback, matching previous bounds.

OPPO optimizes policy learning with provable efficiency.

New RL algorithm tackles online robust MDPs with uncertainty.

A new policy for contextual bandits adapts to reward vector shifts.

New batch RL method avoids overly optimistic policies.

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

A framework for navigating environments with spatially correlated obstacles and uncertain blockage status.

New method for optimistic planning in MDPs using regularization.

Study optimal policy regret in partially observable Markov games with adaptive opponents.

Algorithm converges to Nash equilibria in competitive games.

A generalized optimistic method for saddle point problems with improved complexity.

Monte Carlo Tree Search (MCTS) algorithms perform simulation-based search to improve policies online. During search, the simulation policy is adapted to explore the most promising lines of play. MCTS has been used by state-of-the-art programs for many problems, however a disadvantage to MCTS is that it estimates the va…

We consider a covariate shift problem where one has access to several different training datasets for the same learning problem and a small validation set which possibly differs from all the individual training distributions. This covariate shift is caused, in part, due to unobserved features in the datasets. The objec…

New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.

Bayesian approach for policy search in stochastic domains.

Off-policy reinforcement learning (RL) using a fixed offline dataset of logged interactions is an important consideration in real world applications. This paper studies offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games. We demonstrate that recent off-p…

Improved POMDP regret to sqrt(T) with known observation model.

We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i.e. sequences of actions - and under budget constraint. In this setting, the Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical guarantees but is …

New algorithms reduce regret in online MDPs by adapting to data and variance.

Enhances RL performance with a population-guided parallel learning scheme.

Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…

PCTS optimizes noisy, delayed, multi-fidelity feedbacks in black-box optimization.

Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…

Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference. This is unsatisfactory in many applications where the reference policy is suboptimal and the goal of learning is to improve upon it. Ca…

SIREN protocol corrects optimistic winner's scores in LLM evaluation.

Learning policies on data synthesized by models can in principle quench the thirst of reinforcement learning algorithms for large amounts of real experience, which is often costly to acquire. However, simulating plausible experience de novo is a hard problem for many complex environments, often resulting in biases for …

Paper proposes an RL algorithm to ensure policy performance guarantees.

Paper shows MCTS approximates policy optimization, proposing an improved variant.

Simple policy search outperforms advanced learnable test-time augmentation techniques.

Paper proposes MCTSPO for better reinforcement learning policy optimization.

We propose a novel neural architecture search algorithm via reinforcement learning by decoupling structure and operation search processes. Our approach samples candidate models from the multinomial distribution on the policy vectors defined on the two search spaces independently. The proposed technique improves the eff…

New methods use vector search and nearest-neighbor matching for policy learning in causal inference.

In many cases an intelligent agent may want to learn how to mimic a single observed demonstrated trajectory. In this work we consider how to perform such procedural learning from observation, which could help to enable agents to better use the enormous set of video data on observation sequences. Our approach exploits t…

Scarce data is a major challenge to scaling robot learning to truly complex tasks, as we need to generalize locally learned policies over different task contexts. Contextual policy search offers data-efficient learning and generalization by explicitly conditioning the policy on a parametric context space. In this paper…