arXiv research

The paper proposes an iterative approach to batch reinforcement learning for safer and more informative data collection.

Many practical applications of reinforcement learning constrain agents to learn from a fixed batch of data which has already been gathered, without offering further possibility for data collection. In this paper, we demonstrate that due to errors introduced by extrapolation, standard off-policy deep reinforcement learn…

This paper unifies three regularization methods in batch reinforcement learning.

Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment. Following this result, there have been several papers showing reasonable performances under a variety of environments and batch settings. In this pape…

We study a security threat to batch reinforcement learning and control where the attacker aims to poison the learned policy. The victim is a reinforcement learner / controller which first estimates the dynamics and the rewards from a batch data set, and then solves for the optimal policy with respect to the estimates. …

ABS dynamically adjusts batch size based on policy stability, improving RL performance.

Two new algorithms improve Q* approximation in batch RL with linear error propagation.

New RL algorithm learns good actions from offline data, reducing uncertainty and divergence.

A new framework reduces traffic congestion by 36%.

There has recently been a surge in research in batch Deep Reinforcement Learning (DRL), which aims for learning a high-performing policy from a given dataset without additional interactions with the environment. We propose a new algorithm, Best-Action Imitation Learning (BAIL), which strives for both simplicity and per…

New algorithm learns optimal policy with multi-step lookahead information.

Sparse feature selection improves batch RL efficiency.

Study batch reinforcement learning methods for personalized medical treatments.

New batch RL method avoids overly optimistic policies.

The paper analyzes risk bounds and Rademacher complexity in batch RL.

Efficient RL algorithms for linear function approximation with limited adaptivity constraints.

Improved TD learning reduces batch sampling error.

STEEL tackles batch RL with singularity, improving policy optimization.

Breaks the hardness conjecture for batch RL with a novel tournament-based approach.

New RL difficulty shown for discounted settings.

Meta-algorithm for efficient reinforcement learning from human preferences.

A novel approach learns goal-conditioned policies for locomotion using batch RL.

New batched Langevin Thompson Sampling reduces communication costs for sequential decision making.

Value-function approximation methods that operate in batch mode have foundational importance to reinforcement learning (RL). Finite sample guarantees for these methods often crucially rely on two types of assumptions: (1) mild distribution shift, and (2) representation conditions that are stronger than realizability. H…

Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being dep…

A new algorithm learns policies from batch data in hierarchical RL.

The paper improves model-based reinforcement learning by using multi-timestep objectives.

Critic-regularized regression improves offline RL performance.

The paper proposes a method to explain expert decisions by modeling preferences with 'what if' outcomes.

LF-IBIS learns optimal policies online without explicit likelihood.

Novel online algorithm for hierarchical imitation learning.

This paper introduces sample-averaged Q-learning for better RL performance.

A new method for risk-averse decision-making in Markov processes with improved regret bounds.

We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain. For instance, a chatbot used in automated customer service support can be scored by quality assurance agents, but this process can be expensive, time consuming and noisy. Previous …

Off-policy reinforcement learning algorithms promise to be applicable in settings where only a fixed data-set (batch) of environment interactions is available and no new experience can be acquired. This property makes these algorithms appealing for real world problems such as robot control. In practice, however, standa…

Machine learning, especially deep neural networks, has been rapidly developed in fields including computer vision, speech recognition and reinforcement learning. Although Mini-batch SGD is one of the most popular stochastic optimization methods in training deep networks, it shows a slow convergence rate due to the larg…

Improved theoretical guarantees for SBEED algorithm.

Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN) have achieved state-of-the-art results in a variety of challenging, high-dimensional domains. This success is mainly attributed to the power of deep neural networks to learn rich domain representations for approximating the value function or pol…

BYOL learns useful representations without batch statistics.

A new batch ensemble method reduces regret in stochastic bandits.

Bayesian sOED uses PG reinforcement learning for efficient experiment design.

New algorithm identifies near-optimal policies in adversarial distributed RL settings.

Batch Reinforcement Learning (RL) algorithms attempt to choose a policy from a designer-provided class of policies given a fixed set of training data. Choosing the policy which maximizes an estimate of return often leads to over-fitting when only limited data is available, due to the size of the policy class in relatio…

Paper proposes IRL methods for limited interaction scenarios.

In an increasing number of domains it has been demonstrated that deep learning models can be trained using relatively large batch sizes without sacrificing data efficiency. However the limits of this massive data parallelism seem to differ from domain to domain, ranging from batches of tens of thousands in ImageNet to …

Despite the increasing interest in multi-agent reinforcement learning (MARL) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch MARL with networked agents…

Variance-reduced algorithms, although achieve great theoretical performance, can run slowly in practice due to the periodic gradient estimation with a large batch of data. Batch-size adaptation thus arises as a promising approach to accelerate such algorithms. However, existing schemes either apply prescribed batch-siz…

Reinforcement learning has exceeded human-level performance in game playing AI with deep learning methods according to the experiments from DeepMind on Go and Atari games. Deep learning solves high dimension input problems which stop the development of reinforcement for many years. This study uses both two techniques t…