arXiv research

A single policy suffices for near-optimal parallel exploration in RL.

Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.

KalMamba improves RL efficiency with probabilistic SSMs.

New method ensures consistent inference across different tensor parallel sizes for large language models.

This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.

Generative flow networks use RL to learn probabilistic models efficiently.

We introduce SLM Lab, a software framework for reproducible reinforcement learning (RL) research. SLM Lab implements a number of popular RL algorithms, provides synchronous and asynchronous parallel experiment execution, hyperparameter search, and result analysis. RL algorithms in SLM Lab are implemented in a modular w…

Surrogate models speed up RL training in dynamic systems.

The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time. To accelerate training, practitioners often turn to distributed reinforcement learning architectures to parallelize and accelerate the training process. However, modern methods for scalable reinforcement learnin…

Recent years have witnessed significant progresses in deep Reinforcement Learning (RL). Empowered with large scale neural networks, carefully designed architectures, novel training algorithms and massively parallel computing devices, researchers are able to attack many challenging RL problems. However, in machine learn…

Paper tackles robust knowledge transfer in parallel RL tasks.

Paper connects RL and non-equilibrium statistical mechanics for entropy-regularized RL.

TorchBeast is a platform for reinforcement learning (RL) research in PyTorch. It implements a version of the popular IMPALA algorithm for fast, asynchronous, parallel training of RL agents. Additionally, TorchBeast has simplicity as an explicit design goal: We provide both a pure-Python implementation ("MonoBeast") as …

There are two halves to RL systems: experience collection time and policy learning time. For a large number of samples in rollouts, experience collection time is the major bottleneck. Thus, it is necessary to speed up the rollout generation time with multi-process architecture support. Our work, dubbed WALL-E, utilizes…

Fiber simplifies RL and population-based methods for distributed training.

D2D-SPL uses discrete states and a classifier to train RL faster.

New RL algorithms reduce costs for single-agent and federated learning.

In this paper, a new population-guided parallel learning scheme is proposed to enhance the performance of off-policy reinforcement learning (RL). In the proposed scheme, multiple identical learners with their own value-functions and policies share a common experience replay buffer, and search a good policy in collabora…

Most kernel-based methods, such as kernel or Gaussian process regression, kernel PCA, ICA, or $k$-means clustering, do not scale to large datasets, because constructing and storing the kernel matrix $\mathbf{K}_n$ requires at least $\mathcal{O}(n^2)$ time and space for $n$ samples. Recent works show that sampling point…

New algorithm uses density ratios for efficient online reinforcement learning.

Unified framework for policy improvement in RL with benefits in data efficiency and computation.

JAX-LOB simulates thousands of LOBs for RL training.

Rather than proposing a new method, this paper investigates an issue present in existing learning algorithms. We study the learning dynamics of reinforcement learning (RL), specifically a characteristic coupling between learning and data generation that arises because RL agents control their future data distribution. I…

We present RL-VAE, a graph-to-graph variational autoencoder that uses reinforcement learning to decode molecular graphs from latent embeddings. Methods have been described previously for graph-to-graph autoencoding, but these approaches require sophisticated decoders that increase the complexity of training and evaluat…

This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners. In particular, two typical settings encountered in several applications are considered: multi-agent reinforcement learning (RL) and parallel RL, where frequent information exchan…

Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functio…

ESAC combines genetic methods with RL to improve scalability and efficiency.

New method accelerates MBRL by 10x for complex quadrotor tasks.

The distributional perspective on reinforcement learning (RL) has given rise to a series of successful Q-learning algorithms, resulting in state-of-the-art performance in arcade game environments. However, it has not yet been analyzed how these findings from a discrete setting translate to complex practical application…

In this paper, we present a technique that improves the process of training an agent (using RL) for instruction following. We develop a training curriculum that uses a nominal number of expert demonstrations and trains the agent in a manner that draws parallels from one of the ways in which humans learn to perform comp…

Reinforcement learning (RL) has had many successes in both "deep" and "shallow" settings. In both cases, significant hyperparameter tuning is often required to achieve good performance. Furthermore, when nonlinear function approximation is used, non-stationarity in the state representation can lead to learning instabil…

Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episo…

New RL algorithm GDPO improves DLM reasoning efficiency.

We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural A…

Unified framework for randomized exploration in cooperative MARL.

ALMAB-DC optimizes expensive black-box experiments using active learning and distributed computing.

Hybrid RL algorithms improve offline and online RL in linear MDPs.

From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Re…

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

GPG improves RL from feedback with less memory and compute.

New method defends RL agents from poisoning attacks without MDP knowledge.

Reincarnating RL reuses prior work to accelerate RL progress.

Paper shows RLHF can be solved similarly to standard RL.

RL tackles decision making in unknown environments, focusing on efficiency and efficacy.

Catalyst.RL accelerates RL research with efficient training.

Study shows effectiveness of offline RL in online RL tasks.

MOReL learns offline RL policies using pessimistic MDPs.

RL research overhypes potential but lacks deployable solutions.