arXiv research

Bayesian approach uses Gaussian process for reinforcement learning.

Proposes a Riemannian optimization for policy improvement in MDPs.

We propose expected policy gradients (EPG), which unify stochastic policy gradients (SPG) and deterministic policy gradients (DPG) for reinforcement learning. Inspired by expected sarsa, EPG integrates (or sums) across actions when estimating the gradient, instead of relying only on the action in the sampled trajectory…

RL solves discrete LQ control with Gaussian optimal policy.

PFPN uses particle filtering to improve character control in physics-based simulations.

Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting. However, explicitly parameterized policies are limited by the scope of the chosen parametric probability distribution. We show t…

Proposes DGCN with trajectory sampling for data-efficient policy search in MBRL.

We propose expected policy gradients (EPG), which unify stochastic policy gradients (SPG) and deterministic policy gradients (DPG) for reinforcement learning. Inspired by expected sarsa, EPG integrates across the action when estimating the gradient, instead of relying only on the action in the sampled trajectory. We es…

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…

We propose a method to optimise the parameters of a policy which will be used to safely perform a given task in a data-efficient manner. We train a Gaussian process model to capture the system dynamics, based on the PILCO framework. Our model has useful analytic properties, which allow closed form computation of error …

A new method optimizes in nonstationary environments with many arms efficiently.

Improved exploration in cooperative multi-agent reinforcement learning.

Improved statistical efficiency of Thompson Sampling for combinatorial semi-bandits.

Paper formulates mutual information optimal control for discrete-time systems.

Reinforcement learning algorithms rely on exploration to discover new behaviors, which is typically achieved by following a stochastic policy. In continuous control tasks, policies with a Gaussian distribution have been widely adopted. Gaussian exploration however does not result in smooth trajectories that generally c…

The paper analyzes the reward improvement of aligned policies in large language models.

The paper proves the convergence of Q-value for Gaussian rewards.

Paper optimizes Bayesian optimization for complex functions with macro-actions.

Learning from human feedback is a viable alternative to control design that does not require modelling or control expertise. Particularly, learning from corrective advice garners advantages over evaluative feedback as it is a more intuitive and scalable format. The current state-of-the-art in this field, COACH, has pro…

Develops a new RL algorithm for medical treatment regimes.

Framework for sensitivity analysis in biomanufacturing processes.

Bayesian framework for policy learning in decision problems.

Bayesian sOED uses PG reinforcement learning for efficient experiment design.

The paper interprets policy-gradient algorithms using continuation theory.

Data-driven RL solves Merton's expected utility problem via policy randomization.

New approach to portfolio optimization shows entropy regularization is ineffective.

Revisits PPO design choices, exposing failure modes and proposing alternatives.

Entropy regularization is an important idea in reinforcement learning, with great success in recent algorithms like Soft Q Network (SQN) and Soft Actor-Critic (SAC1). In this work, we extend this idea into the on-policy realm. We propose the soft policy gradient theorem (SPGT) for on-policy maximum entropy reinforcemen…

The trade-off between the cost of acquiring and processing data, and uncertainty due to a lack of data is fundamental in machine learning. A basic instance of this trade-off is the problem of deciding when to make noisy and costly observations of a discrete-time Gaussian random walk, so as to minimise the posterior var…

The paper tackles estimating optimal policy value in linear bandits with general context distributions.

Bayesian Markowitz portfolio problem shows entropy regularization is ineffective.

UK's rapid vaccine rollout linked to reduced COVID-19 mortality.

This work optimizes RL algorithms using entropy regularisation for continuous-time LQ problems.

We study the variance of the REINFORCE policy gradient estimator in environments with continuous state and action spaces, linear dynamics, quadratic cost, and Gaussian noise. These simple environments allow us to derive bounds on the estimator variance in terms of the environment and noise parameters. We compare the pr…

The study analyzes how neural reward models learn features for policy optimization in a Gaussian single-index model.

New algorithm reduces worst-case regret for heavy-tailed bandits.

A method for accurate pricing of multidimensional derivatives under uncertain volatility.

A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.

In quadruped gait learning, policy search methods that scale high dimensional continuous action spaces are commonly used. In most approaches, it is necessary to introduce prior knowledge on the gaits to limit the highly non-convex search space of the policies. In this work, we propose a new approach to encode the symme…

This paper studies the potential of the return distribution for exploration in deterministic reinforcement learning (RL) environments. We study network losses and propagation mechanisms for Gaussian, Categorical and Gaussian mixture distributions. Combined with exploration policies that leverage this return distributio…

This paper presents a novel nonmyopic adaptive Gaussian process planning (GPP) framework endowed with a general class of Lipschitz continuous reward functions that can unify some active learning/sensing and Bayesian optimization criteria and offer practitioners some flexibility to specify their desired choices for defi…

Survey explores geometric aspects of policy optimization in control systems.

Policy Gradient (PG) algorithms are among the best candidates for the much-anticipated applications of reinforcement learning to real-world control tasks, such as robotics. However, the trial-and-error nature of these methods poses safety issues whenever the learning process itself must be performed on a physical syste…

The Knowledge Gradient (KG) policy was originally proposed for online ranking and selection problems but has recently been adapted for use in online decision making in general and multi-armed bandit problems (MABs) in particular. We study its use in a class of exponential family MABs and identify weaknesses, including …

We propose a policy improvement algorithm for Reinforcement Learning (RL) which is called Rerouted Behavior Improvement (RBI). RBI is designed to take into account the evaluation errors of the Q-function. Such errors are common in RL when learning the $Q$-value from finite past experience data. Greedy policies or even …

Policy evaluation is a key process in Reinforcement Learning (RL). It assesses a given policy by estimating the corresponding value function. When using parameterized value functions, common approaches minimize the sum of squared Bellman temporal-difference errors and receive a point-estimate for the parameters. Kalman…

A new method for steering large agent populations efficiently.

New framework learns policies for partially observable systems.