arXiv research

The paper interprets policy-gradient algorithms using continuation theory.

New algorithm finds near-optimal policies efficiently in zero-sum games.

EPIC quantifies reward differences without policy optimization.

Proposes a Riemannian optimization for policy improvement in MDPs.

Paper introduces a new value function for state transitions and optimal policy learning.

Adapts GRPO for off-policy RL, improving reward.

Entropy regularization improves policy optimization in reinforcement learning.

Temporal difference learning and Residual Gradient methods are the most widely used temporal difference based learning algorithms; however, it has been shown that none of their objective functions is optimal w.r.t approximating the true value function $V$. Two novel algorithms are proposed to approximate the true value…

PPOS improves PPO by smoothing the surrogate objective function.

This paper optimizes sampling policies for Bayesian optimization to improve exploration and exploitation.

This paper optimizes off-policy evaluation in reinforcement learning with function approximation.

N-discount optimality was introduced as a hierarchical form of policy- and value-function optimality, with Blackwell optimality lying at the top level of the hierarchy Veinott (1969); Blackwell (1962). We formalize notions of myopic discount factors, value functions and policies in terms of Blackwell optimality in MDPs…

Entropy regularization is commonly used to improve policy optimization in reinforcement learning. It is believed to help with \emph{exploration} by encouraging the selection of more stochastic policies. In this work, we analyze this claim using new visualizations of the optimization landscape based on randomly perturbi…

Advocates focusing on utility functions to avoid unfair outcomes.

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

Managers of US National Forests must decide what policy to apply for dealing with lightning-caused wildfires. Conflicts among stakeholders (e.g., timber companies, home owners, and wildlife biologists) have often led to spirited political debates and even violent eco-terrorism. One way to transform these conflicts into…

Paper optimizes traffic signal control for better traffic flow.

ENIAC method optimizes and explores complex RL problems with non-linear policies.

New method optimizes policies without assuming known link functions between preferences and rewards.

PROPO tackles non-stationary MDPs with efficient policy optimization.

We consider off-policy evaluation and optimization with continuous action spaces. We focus on observational data where the data collection policy is unknown and needs to be estimated. We take a semi-parametric approach where the value function takes a known parametric form in the treatment, but we are agnostic on how i…

We accelerate PMD algorithms for reinforcement learning using functional methods.

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-func…

PPO algorithm converges to global optimality in multi-agent reinforcement learning.

Single-timescale actor-critic finds globally optimal policy.

New method estimates and optimizes policy differences using orthogonal learning.

We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions. For such a problem, we construct a sequence of surrogate convex constrained optimiza…

Discuss new policy learning objectives and methods.

PPG separates policy and value function training phases for better reinforcement learning efficiency.

ZSPO optimizes RL from unknown link functions using human feedback.

A key problem in reinforcement learning for control with general function approximators (such as deep neural networks and other nonlinear functions) is that, for many algorithms employed in practice, updates to the policy or $Q$-function may fail to improve performance---or worse, actually cause the policy performance …

New method learns optimal policies in presence of unmeasured confounders.

Bayesian Robust Optimization for Imitation Learning (BROIL) balances risk and reward.

Policy optimization is an effective reinforcement learning approach to solve continuous control tasks. Recent achievements have shown that alternating online and offline optimization is a successful choice for efficient trajectory reuse. However, deciding when to stop optimizing and collect new trajectories is non-triv…

Stabilizes policy optimization with off-policy data using divergence augmentation.

In this paper, we propose a distributed off-policy actor critic method to solve multi-agent reinforcement learning problems. Specifically, we assume that all agents keep local estimates of the global optimal policy parameter and update their local value function estimates independently. Then, we introduce an additional…

The paper analyzes optimal liquidation strategies for cryptocurrencies considering both temporary and permanent price impacts.

We consider a version of the stochastic inventory control problem for a spectrally positive Lévy demand process, in which the inventory can only be replenished at independent exponential times. We show the optimality of a periodic barrier replenishment policy that restocks any shortage below a certain threshold at each…

Optimizes Thompson sampling policies using policy gradient methods.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

A new method improves policy evaluation in RL by tracking value uncertainties.

Proposes a new reinforcement learning algorithm using Q-function.

This paper optimizes MDP policies for efficient state aggregation.

New method efficiently evaluates policies using trajectory data.

DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.

This paper optimizes slate decision systems for large action spaces.

Unified minimax value interval for off-policy evaluation and optimization.

OPPO optimizes policy learning with provable efficiency.