arXiv research

Study batch reinforcement learning methods for personalized medical treatments.

Introduces LoCA regret to evaluate model-based RL methods.

Model-based machine learning improves communication systems.

Simple model-based reinforcement learning outperforms model-free methods in complex tasks.

Survey of integrating planning and learning in model-based reinforcement learning.

A framework schedules hyperparameters for model-based reinforcement learning, improving performance.

Estimates of predictive uncertainty are important for accurate model-based planning and reinforcement learning. However, predictive uncertainties---especially ones derived from modern deep learning systems---can be inaccurate and impose a bottleneck on performance. This paper explores which uncertainties are needed for…

A new method reduces compounding errors in model-based reinforcement learning.

Model-based reinforcement learning algorithms tend to achieve higher sample efficiency than model-free methods. However, due to the inevitable errors of learned models, model-based methods struggle to achieve the same asymptotic performance as model-free methods. In this paper, We propose a Policy Optimization method w…

We give an explicit algorithm and source code for constructing risk models based on machine learning techniques. The resultant covariance matrices are not factor models. Based on empirical backtests, we compare the performance of these machine learning risk models to other constructions, including statistical risk mode…

Unified reinforcement learning methods using hybrid inference.

Clarifies model-based RL's theoretical issues and counterexamples for popular losses.

FOCUS improves offline RL by incorporating causal structure into world-models.

Model-based reinforcement learning (RL) has proven to be a data efficient approach for learning control tasks but is difficult to utilize in domains with complex observations such as images. In this paper, we present a method for learning representations that are suitable for iterative model-based policy improvement, e…

Significant progress has been made in the area of model-based reinforcement learning. State-of-the-art algorithms are now able to match the asymptotic performance of model-free methods while being significantly more data efficient. However, this success has come at a price: state-of-the-art model-based methods require …

VMBPO optimizes model and policy jointly using variational lower-bound.

New hybrid RL algorithm outperforms model-free and model-based methods.

M3PO improves model-based meta-RL with theoretical guarantees.

Unified framework for policy improvement in RL with benefits in data efficiency and computation.

MAGE optimizes policies using action gradients from model-based learning.

Paper tackles action delays in reinforcement learning, proposing a delay-aware framework.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

The paper uses model-based trees to create interpretable surrogate models for complex machine learning models.

Traditional model-based reinforcement learning approaches learn a model of the environment dynamics without explicitly considering how it will be used by the agent. In the presence of misspecified model classes, this can lead to poor estimates, as some relevant available information is ignored. In this paper, we introd…

This study tackles adversarial corruption in model-based reinforcement learning.

MoMA improves model-based RL by using unrestricted policy classes.

We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…

A contraction analysis improves model-based RL's error recovery.

Unified Latent Dynamics unifies model-free and model-based reinforcement learning.

Improved model-based reinforcement learning for interactive dialogue tasks reduces sample needs and improves performance.

When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn a single-step transition model. In this paper, we propose a multi-step model that predicts the outcome of an action sequence with variable …

New RL framework models continuous-time dynamics using neural ODEs.

Curious Replay improves model-based reinforcement learning agents' adaptability.

Adaptive discretization improves model-based RL in large spaces.

A method for learning a context latent vector to improve generalization in model-based RL.

Model-based reinforcement learning methods typically learn models for high-dimensional state spaces by aiming to reconstruct and predict the original observations. However, drawing inspiration from model-free reinforcement learning, we propose learning a latent dynamics model directly from rewards. In this work, we int…

Model-free reinforcement learning methods such as the Proximal Policy Optimization algorithm (PPO) have successfully applied in complex decision-making problems such as Atari games. However, these methods suffer from high variances and high sample complexity. On the other hand, model-based reinforcement learning method…

Designing effective model-based reinforcement learning algorithms is difficult because the ease of data generation must be weighed against the bias of model-generated data. In this paper, we study the role of model usage in policy optimization both theoretically and empirically. We first formulate and analyze a model-b…

This work learns visual representations for deformable objects using contrastive estimation.

Survive method improves model-based RL by avoiding terminal states, reducing sample complexity.

Model-based reinforcement learning approaches carry the promise of being data efficient. However, due to challenges in learning dynamics models that sufficiently match the real-world dynamics, they struggle to achieve the same asymptotic performance as model-free methods. We propose Model-Based Meta-Policy-Optimization…

Recent success in deep reinforcement learning for continuous control has been dominated by model-free approaches which, unlike model-based approaches, do not suffer from representational limitations in making assumptions about the world dynamics and model errors inevitable in complex domains. However, they require a lo…

We introduce a model based off-the-grid image reconstruction algorithm using deep learned priors. The main difference of the proposed scheme with current deep learning strategies is the learning of non-linear annihilation relations in Fourier space. We rely on a model based framework, which allows us to use a significa…

Model-based learning algorithms have been shown to use experience efficiently when learning to solve Markov Decision Processes (MDPs) with finite state and action spaces. However, their high computational cost due to repeatedly solving an internal model inhibits their use in large-scale problems. We propose a method ba…

New model learning objective improves continuous control tasks.

Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the "Imagination-based Planner", the first m…

We propose a model-based machine-learning approach for polarization-multiplexed systems by parameterizing the split-step method for the Manakov-PMD equation. This approach performs hardware-friendly DBP and distributed PMD compensation with performance close to the PMD-free case.

A method to reduce bias in model-based policy evaluation by shifting operators.