arXiv research

Survey of integrating planning and learning in model-based reinforcement learning.

Greedy policies in model-based RL achieve tight regret bounds without full planning.

DPN combines model-based and model-free reinforcement learning for efficient planning.

Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the "Imagination-based Planner", the first m…

The paper tackles model-based RL's inaccuracy issue by dynamically adjusting planning horizons.

EBMs improve sample efficiency and generalization in RL.

Proposes learning latent reward model for planning from rewards.

AOP combines model-based planning with model-free learning to handle lifelong learning challenges.

Regularizes model-based planning using energy-based models for efficient learning.

POPLIN improves model-based planning in complex environments.

End-to-end Dyna-AIL combines planning and learning for faster convergence.

Improves model-based control and exploration by estimating model uncertainty.

Proposes a method for model-based RL in complex environments without perfect simulators.

This paper calibrates deep reinforcement learning models to improve planning and performance.

PALM learns abstract models for efficient planning and task transfer.

Model-Based Offline Planning (MBOP) learns models from offline data to control systems directly.

Paper proposes using expectation models for planning in stochastic environments.

Selective planning with imperfect models reduces harmful effects of model inadequacy.

Study shows plug-in model-based reinforcement learning is minimax optimal.

Residual algorithms improve reinforcement learning performance.

DADS discovers skills with predictable outcomes from unlabeled data.

Unified reinforcement learning methods using hybrid inference.

We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to inter…

This paper builds a model to predict the long-term future in reinforcement learning.

Study non-stationary MDPs using worst-case RL, proposing RATS algorithm.

Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI. This paper describes progresses on this challenge in the context of man-made environments, which are visually diverse but contain intrinsic semantic regularities. We propose a hybrid …

The study compares reinforcement learning models and finds model-based approaches superior for complex MDPs.

We demonstrate the use of conditional autoregressive generative models (van den Oord et al., 2016a) over a discrete latent space (van den Oord et al., 2017b) for forward planning with MCTS. In order to test this method, we introduce a new environment featuring varying difficulty levels, along with moving goals and obst…

New approach handles stochastic and partially-observable environments using discrete autoencoders and Monte Carlo tree search.

This work learns visual representations for deformable objects using contrastive estimation.

PlanGAN uses GANs to plan efficient trajectories for multi-goal tasks in sparse reward environments.

Improved CEM for fast real-time planning in high-dimensional control tasks.

The computational costs of inference and planning have confined Bayesian model-based reinforcement learning to one of two dismal fates: powerful Bayes-adaptive planning but only for simplistic models, or powerful, Bayesian non-parametric models but using simple, myopic planning strategies such as Thompson sampling. We …

A multi-step model reduces compounding errors in reinforcement learning.

New algorithm minimizes worst-case regret in uncertain, time-varying dynamics.

This paper presents a unifying framework for reinforcement learning and planning.

Unified Latent Dynamics unifies model-free and model-based reinforcement learning.

Novel approach to learning models based on subjective timescales for better exploration and decision-making.

Paper benchmarks MBRL algorithms with new environments.

We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…

We designed a grid world task to study human planning and re-planning behavior in an unknown stochastic environment. In our grid world, participants were asked to travel from a random starting point to a random goal position while maximizing their reward. Because they were not familiar with the environment, they needed…

This paper presents preliminary work on learning the search heuristic for the optimal motion planning for automated driving in urban traffic. Previous work considered search-based optimal motion planning framework (SBOMP) that utilized numerical or model-based heuristics that did not consider dynamic obstacles. Optimal…

When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn a single-step transition model. In this paper, we propose a multi-step model that predicts the outcome of an action sequence with variable …

Model learns and plans in real-time under constraints for robotic systems.

Enhances PlaNet for better planning in uncertain environments.

Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning dynamics models that are accurate enough for planning has been a long-standing challenge, especiall…

Self-consistent models improve reinforcement learning by aligning predictions with future values.

This paper introduces the QMDP-net, a neural network architecture for planning under partial observability. The QMDP-net combines the strengths of model-free learning and model-based planning. It is a recurrent policy network, but it represents a policy for a parameterized set of tasks by connecting a model with a plan…