arXiv research

New model-free DR-RL algorithm with finite sample complexity.

A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.

New hybrid RL algorithm outperforms model-free and model-based methods.

Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…

Unified reinforcement learning methods using hybrid inference.

FORK improves model-free reinforcement learning performance.

Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15]. Recently, equipped with the idea of optimism in the face of uncertainty, Q-learning algorithms [Jin, Al…

New model-free algorithm achieves similar LQR regret guarantees.

Improved model-free RL algorithm with reduced sample complexity.

Optimizes control of noisy discrete systems without system matrix knowledge.

New model-free algorithms learn representations for low-rank MDPs efficiently.

New algorithm borrows future randomness to stabilize model-free control.

We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy. We design new algorithms for RL with a generic model class and analyze their statistical properties. Our algorithms have sample …

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

The OGY method is one of control methods for a chaotic system. In the method, we have to calculate a stabilizing periodic orbit embedded in its chaotic attractor. Thus, we cannot use this method in the case where a precise mathematical model of the chaotic system cannot be identified. In this case, the delayed feedback…

DREAM learns optimal strategies in imperfect games without needing a simulator.

Paper develops efficient algorithms for zero-sum Markov games with general function classes.

Improves sample efficiency in RL by matching model-based gradients.

Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. This is especially true with high-capacity parametric function approximators, such as deep networks. In this paper, we study how to bridge th…

Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…

New algorithms for model selection in off-policy evaluation of reinforcement learning.

Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of c…

The paper presents a model-free method for stabilizing unknown control systems.

New algorithm improves sample efficiency for zero-sum Markov games.

Improved model-free reinforcement learning with decision-estimation coefficient.

We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonu…

Simple image augmentation boosts deep RL from pixels.

New algorithm learns optimal policies with minimal memory and time.

Integrating model-free and model-based approaches in reinforcement learning has the potential to achieve the high performance of model-free algorithms with low sample complexity. However, this is difficult because an imperfect dynamics model can degrade the performance of the learning algorithm, and in sufficiently com…

Model-free reinforcement learning (RL) algorithms, such as Q-learning, directly parameterize and update value functions or policies without explicitly modeling the environment. They are typically simpler, more flexible to use, and thus more prevalent in modern deep RL than model-based approaches. However, empirical wor…

MOOSE improves offline RL robustness by using dynamics models.

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

Algorithm learns NE in imperfect information games with imperfect feedback.

The field of reinforcement learning (RL) is facing increasingly challenging domains with combinatorial complexity. For an RL agent to address these challenges, it is essential that it can plan effectively. Prior work has typically utilized an explicit model of the environment, combined with a specific planning algorith…

New model-free RL algorithm tackles robust average-reward problems with finite sample complexity analysis.

New algorithm improves RL performance across different environments.

Study compares model-free valuation to actual financial outcomes, finds it slightly conservative.

Unified framework for policy improvement in RL with benefits in data efficiency and computation.

Model-free reinforcement learning methods such as the Proximal Policy Optimization algorithm (PPO) have successfully applied in complex decision-making problems such as Atari games. However, these methods suffer from high variances and high sample complexity. On the other hand, model-based reinforcement learning method…

In many optimization problems in wireless communications, the expressions of objective function or constraints are hard or even impossible to derive, which makes the solutions difficult to find. In this paper, we propose a model-free learning framework to solve constrained optimization problems without the supervision …

This work proposes robust reinforcement learning methods using both offline and online data.

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-func…

We provide the first solution for model-free reinforcement learning of ω-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of ω-regular objectives to an almost- sure reachability problem and extend this technique to learning how to control an …

Robotic table tennis learns efficient policies to return balls at 100Hz.

Efficient Reinforcement Learning usually takes advantage of demonstration or good exploration strategy. By applying posterior sampling in model-free RL under the hypothesis of GP, we propose Gaussian Process Posterior Sampling Reinforcement Learning(GPPSTD) algorithm in continuous state space, giving theoretical justif…

Training an agent to solve control tasks directly from high-dimensional images with model-free reinforcement learning (RL) has proven difficult. A promising approach is to learn a latent representation together with the control policy. However, fitting a high-capacity encoder using a scarce reward signal is sample inef…

Paper proves efficiency of MARL with transformers, addressing agent complexity.

New algorithms cluster nodes in SBM graphs faster and more accurately.