arXiv research

Paper improves policy updates in reinforcement learning to speed up learning.

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…

Paper addresses OPE for dependent bandit samples using MDS and batch updates.

AMUSE uses reinforcement learning to predict optimal model updates.

Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updat…

P3O merges on-policy and off-policy updates without extra hyper-parameters.

A new resampling strategy, Importance Resampling, improves sample efficiency and reduces variance in off-policy prediction.

New algorithm learns deterministic policies in continuous domains using sign of advantage function.

Simplifies RL training with fewer techniques, reducing bias and instability.

Accelerates policy optimization in RL with optimistic and adaptive updates.

New algorithm for sequential off-policy learning improves performance over batch methods.

Rarely switch policies to optimize treatment effects, reducing harmful changes.

MAME models a separate exploration policy for faster adaptation.

Paper introduces a new policy optimization method using importance sampling.

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

CRPO solves challenging SRL problems with convergence guarantee.

To ensure stability of learning, state-of-the-art generalized policy iteration algorithms augment the policy improvement step with a trust region constraint bounding the information loss. The size of the trust region is commonly determined by the Kullback-Leibler (KL) divergence, which not only captures the notion of d…

A novel framework optimizes experience replay for reinforcement learning.

Single-timescale actor-critic finds globally optimal policy.

SFPO optimizes LLM reasoning by repositioning before updating, improving stability and efficiency.

Optimizes antenna tilt for better QoS in cellular networks.

We make policy optimization algorithms batch size-invariant by decoupling proximal and behavior policies.

Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this pa…

Step-DAD improves BED by periodically updating a design policy during experiments.

Logit dynamics formula reveals self-regulation in softmax policy gradient methods.

Enhances SAC for better sample efficiency in continuous-action tasks.

This paper improves reinforcement learning policies in a scalable way.

Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…

Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control. In a recent work \cite{efroni2018beyond}, multiple-step greedy policies and their use in vanilla Policy Iteration algorithms were proposed and analy…

Enhanced tracking control for AUVs with improved policy gradient method.

We propose a new sample-efficient methodology, called Supervised Policy Update (SPU), for deep reinforcement learning. Starting with data generated by the current policy, SPU formulates and solves a constrained optimization problem in the non-parameterized proximal policy space. Using supervised regression, it then con…

Develops a new method for optimizing policies in hierarchical models.

Enhances deep RL learning with stable policy updates.

We study the optimal trading policies for a wind energy producer who aims to sell the future production in the open forward, spot, intraday and adjustment markets, and who has access to imperfect dynamically updated forecasts of the future production. We construct a stochastic model for the forecast evolution and deter…

Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e.g., ExIt from [2], AlphaGo-Zero from [27]). This new family of algorithms maintains, and alternately optimizes, two policies: a fast, reactive policy (e.g., a deep neural network) deployed at t…

Paper proposes Cycle-of-Learning framework for better reinforcement learning performance.

Revel tackles safe exploration in RL with verified symbolic policies.

Study nonparametric contextual bandits with batched updates, achieving optimal regret.

Algorithm finds novel policies for tasks using novelty reward.

DG separates successes and failures by gating updates with advantage and surprisal.

New actor-critic method reduces sample complexity for reinforcement learning.

A framework for navigating environments with spatially correlated obstacles and uncertain blockage status.

Enhances RL performance with a population-guided parallel learning scheme.

Two algorithms address limited adaptivity in generalized linear contextual bandits.

Paper achieves $ε^{-2}$ sample complexity for actor-critic methods with minimal assumptions.

Interpolates between SPG and NeuRD with Capped Implicit Exploration.

Selective state-adaptive regularization improves offline RL performance.

FOCOPS optimizes agent's behavior while adhering to constraints.