arXiv research

Paper proposes new $γ$-regret measure for non-episodic RL.

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then…

New algorithms reduce reinforcement learning regret in factored MDPs.

This paper sets communication complexity bounds for distributed RL.

UCRL-CMDP algorithm optimizes RL with constraints on average costs.

Study designs incentives for adapting multi-agent systems without knowing their learning dynamics.

We give a simple optimistic algorithm for which it is easy to derive regret bounds of $\tilde{O}(\sqrt{t_{\rm mix} SAT})$ after $T$ steps in uniformly ergodic Markov decision processes with $S$ states, $A$ actions, and mixing time parameter $t_{\rm mix}$. These bounds are the first regret bounds in the general, non-epi…

Restless bandit problems assume time-varying reward distributions of the arms, which adds flexibility to the model but makes the analysis more challenging. We study learning algorithms over the unknown reward distributions and prove a sub-linear, $O(\sqrt{T}\log T)$, regret bound for a variant of Thompson sampling. Our…

New algorithm learns FMDP structure while minimizing regret.

Hybrid RL algorithms improve offline and online RL in linear MDPs.

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

New method defends RL agents from poisoning attacks without MDP knowledge.

Reincarnating RL reuses prior work to accelerate RL progress.

Paper shows RLHF can be solved similarly to standard RL.

RL tackles decision making in unknown environments, focusing on efficiency and efficacy.

Catalyst.RL accelerates RL research with efficient training.

Study shows effectiveness of offline RL in online RL tasks.

MOReL learns offline RL policies using pessimistic MDPs.

RL research overhypes potential but lacks deployable solutions.

RL applied to finance tasks, highlighting challenges and future directions.

Reinforcement learning (RL) algorithms allow agents to learn skills and strategies to perform complex tasks without detailed instructions or expensive labelled training examples. That is, RL agents can learn, as we learn. Given the importance of learning in our intelligence, RL has been thought to be one of key compone…

A simple approach to offline RL without additional complexity.

MiniHack simplifies creation of complex RL environments.

Generative flow networks use RL to learn probabilistic models efficiently.

Experimentally, it has been observed that humans and animals often make decisions that do not maximize their expected utility, but rather choose outcomes randomly, with probability proportional to expected utility. Probability matching, as this strategy is called, is equivalent to maximum entropy reinforcement learning…

RL agents outperform baselines in asset allocation.

This paper shows RL with KL penalties is equivalent to Bayesian inference for fine-tuning LMs.

Despite the recent progress in deep reinforcement learning field (RL), and, arguably because of it, a large body of work remains to be done in reproducing and carefully comparing different RL algorithms. We present catalyst.RL, an open source framework for RL research with a focus on reproducibility and flexibility. Ma…

Since their introduction a year ago, distributional approaches to reinforcement learning (distributional RL) have produced strong results relative to the standard approach which models expected values (expected RL). However, aside from convergence guarantees, there have been few theoretical results investigating the re…

HTMRL uses HTM for RL, adapting faster to changing environments.

We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. We discuss six core elements, six important mechanisms, and twelve applications, focusing on contemporary work, and in historical contexts. We start with background of artificial intelligence, machine learning, deep…

New benchmarks for offline RL from diverse datasets.

M3PO improves model-based meta-RL with theoretical guarantees.

evo-RL combines evolutionary computation with reinforcement learning for better adaptability.

This work bridges offline RL and DRL to address distributional shift.

Effective network congestion control strategies are key to keeping the Internet (or any large computer network) operational. Network congestion control has been dominated by hand-crafted heuristics for decades. Recently, ReinforcementLearning (RL) has emerged as an alternative to automatically optimize such control str…

RL Unplugged benchmarks offline RL methods across diverse domains.

Optimistic RL algorithms are simplified for deep RL with competitive performance.

Paper proposes a hybrid RL algorithm that combines offline and online data without needing reward info.

Critic-regularized regression improves offline RL performance.

This paper surveys RL for combinatorial optimization, focusing on TSP.

Mixed RL improves RL efficiency with dual representations.

The standard reinforcement learning (RL) formulation considers the expectation of the (discounted) cumulative reward. This is limiting in applications where we are concerned with not only the expected performance, but also the distribution of the performance. In this paper, we introduce micro-objective reinforcement le…

Survey of RL algorithms for dynamic environments.

A new reinforcement learning method improves Max-Cut solutions without needing training data.

This chapter reviews statistical tools for reinforcement learning.