arXiv research

The paper addresses Dyna-style RL's value hallucination issue by proposing a new algorithm.

Dyna is an architecture for model-based reinforcement learning (RL), where simulated experience from a model is used to update policies or value functions. A key component of Dyna is search-control, the mechanism to generate the state and action from which the agent queries the model, which remains largely unexplored. …

Self-consistent models improve reinforcement learning by aligning predictions with future values.

New hybrid RL algorithm outperforms model-free and model-based methods.

A new search-control strategy improves Dyna's efficiency.

New algorithms accelerate value function convergence in MDPs.

Dyna-style reinforcement learning is a powerful approach for problems where not much real data is available. The main idea is to supplement real trajectories, or sequences of sampled states over time, with simulated ones sampled from a learned model of the environment. However, in large state spaces, the problem of lea…

Adversarial methods for imitation learning have been shown to perform well on various control tasks. However, they require a large number of environment interactions for convergence. In this paper, we propose an end-to-end differentiable adversarial imitation learning algorithm in a Dyna-like framework for switching be…

New model learning objective improves continuous control tasks.

We propose a hybrid approach aimed at improving the sample efficiency in goal-directed reinforcement learning. We do this via a two-step mechanism where firstly, we approximate a model from Model-Free reinforcement learning. Then, we leverage this approximate model along with a notion of reachability using Mean First P…

A framework schedules hyperparameters for model-based reinforcement learning, improving performance.

In unsupervised learning, there is no apparent straightforward cost function that can capture the significant factors of variations and similarities. Since natural systems have smooth dynamics, an opportunity is lost if an unsupervised objective function remains static during the training process. The absence of concre…

This paper motivates and develops source traces for temporal difference (TD) learning in the tabular setting. Source traces are like eligibility traces, but model potential histories rather than immediate ones. This allows TD errors to be propagated to potential causal states and leads to faster generalization. Source …

An algorithm is presented for momentum gradient descent optimization based on the first-order differential equation of the Newtonian dynamics. The fictitious mass is introduced to the dynamics of momentum for regularizing the adaptive stepsize of each individual parameter. The dynamic relaxation is adapted for stochast…

Hybrid RL algorithms improve offline and online RL in linear MDPs.

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

New method defends RL agents from poisoning attacks without MDP knowledge.

Reincarnating RL reuses prior work to accelerate RL progress.

Paper shows RLHF can be solved similarly to standard RL.

RL tackles decision making in unknown environments, focusing on efficiency and efficacy.

Catalyst.RL accelerates RL research with efficient training.

Study shows effectiveness of offline RL in online RL tasks.

MOReL learns offline RL policies using pessimistic MDPs.

RL research overhypes potential but lacks deployable solutions.

RL applied to finance tasks, highlighting challenges and future directions.

Reinforcement learning (RL) algorithms allow agents to learn skills and strategies to perform complex tasks without detailed instructions or expensive labelled training examples. That is, RL agents can learn, as we learn. Given the importance of learning in our intelligence, RL has been thought to be one of key compone…

A simple approach to offline RL without additional complexity.

MiniHack simplifies creation of complex RL environments.

Generative flow networks use RL to learn probabilistic models efficiently.

Experimentally, it has been observed that humans and animals often make decisions that do not maximize their expected utility, but rather choose outcomes randomly, with probability proportional to expected utility. Probability matching, as this strategy is called, is equivalent to maximum entropy reinforcement learning…

RL agents outperform baselines in asset allocation.

This paper shows RL with KL penalties is equivalent to Bayesian inference for fine-tuning LMs.

Despite the recent progress in deep reinforcement learning field (RL), and, arguably because of it, a large body of work remains to be done in reproducing and carefully comparing different RL algorithms. We present catalyst.RL, an open source framework for RL research with a focus on reproducibility and flexibility. Ma…

Since their introduction a year ago, distributional approaches to reinforcement learning (distributional RL) have produced strong results relative to the standard approach which models expected values (expected RL). However, aside from convergence guarantees, there have been few theoretical results investigating the re…

HTMRL uses HTM for RL, adapting faster to changing environments.

We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. We discuss six core elements, six important mechanisms, and twelve applications, focusing on contemporary work, and in historical contexts. We start with background of artificial intelligence, machine learning, deep…

New benchmarks for offline RL from diverse datasets.

M3PO improves model-based meta-RL with theoretical guarantees.

evo-RL combines evolutionary computation with reinforcement learning for better adaptability.

This work bridges offline RL and DRL to address distributional shift.

Effective network congestion control strategies are key to keeping the Internet (or any large computer network) operational. Network congestion control has been dominated by hand-crafted heuristics for decades. Recently, ReinforcementLearning (RL) has emerged as an alternative to automatically optimize such control str…

The study questions the necessity of accurate models in MBRL.

RL Unplugged benchmarks offline RL methods across diverse domains.

Optimistic RL algorithms are simplified for deep RL with competitive performance.

Paper proposes a hybrid RL algorithm that combines offline and online data without needing reward info.

Critic-regularized regression improves offline RL performance.

This paper surveys RL for combinatorial optimization, focusing on TSP.

Mixed RL improves RL efficiency with dual representations.