arXiv research

Introduces LoCA regret to evaluate model-based RL methods.

Simple model-based reinforcement learning outperforms model-free methods in complex tasks.

M3PO improves model-based meta-RL with theoretical guarantees.

Model-based neural networks generalize better than ReLU networks for sparse recovery.

Study batch reinforcement learning methods for personalized medical treatments.

Model-based machine learning improves communication systems.

Model-based reinforcement learning algorithms tend to achieve higher sample efficiency than model-free methods. However, due to the inevitable errors of learned models, model-based methods struggle to achieve the same asymptotic performance as model-free methods. In this paper, We propose a Policy Optimization method w…

A new method reduces compounding errors in model-based reinforcement learning.

A framework schedules hyperparameters for model-based reinforcement learning, improving performance.

Clarifies model-based RL's theoretical issues and counterexamples for popular losses.

The paper uses model-based trees to create interpretable surrogate models for complex machine learning models.

Unified framework for model-based RL with sample complexity guarantees.

Unified reinforcement learning methods using hybrid inference.

Survey of integrating planning and learning in model-based reinforcement learning.

Estimates of predictive uncertainty are important for accurate model-based planning and reinforcement learning. However, predictive uncertainties---especially ones derived from modern deep learning systems---can be inaccurate and impose a bottleneck on performance. This paper explores which uncertainties are needed for…

MoMA improves model-based RL by using unrestricted policy classes.

Designing effective model-based reinforcement learning algorithms is difficult because the ease of data generation must be weighed against the bias of model-generated data. In this paper, we study the role of model usage in policy optimization both theoretically and empirically. We first formulate and analyze a model-b…

We give an explicit algorithm and source code for constructing risk models based on machine learning techniques. The resultant covariance matrices are not factor models. Based on empirical backtests, we compare the performance of these machine learning risk models to other constructions, including statistical risk mode…

An evolutionary algorithm (EA) is developed as an alternative to the EM algorithm for parameter estimation in model-based clustering. This EA facilitates a different search of the fitness landscape, i.e., the likelihood surface, utilizing both crossover and mutation. Furthermore, this EA represents an efficient approac…

Unified framework for policy improvement in RL with benefits in data efficiency and computation.

VMBPO optimizes model and policy jointly using variational lower-bound.

Model-based reinforcement learning (RL) has proven to be a data efficient approach for learning control tasks but is difficult to utilize in domains with complex observations such as images. In this paper, we present a method for learning representations that are suitable for iterative model-based policy improvement, e…

Unified Latent Dynamics unifies model-free and model-based reinforcement learning.

FOCUS improves offline RL by incorporating causal structure into world-models.

Adaptive discretization improves model-based RL in large spaces.

Model-based clustering is a popular approach for clustering multivariate data which has seen applications in numerous fields. Nowadays, high-dimensional data are more and more common and the model-based clustering approach has adapted to deal with the increasing dimensionality. In particular, the development of variabl…

This study tackles adversarial corruption in model-based reinforcement learning.

In this paper, we introduce a new model for the risk process based on general compound Hawkes process (GCHP) for the arrival of claims. We call it risk model based on general compound Hawkes process (RMGCHP). The Law of Large Numbers (LLN) and the Functional Central Limit Theorem (FCLT) are proved. We also study the ma…

New algorithms accelerate model-based optimization for stochastic problems.

Curious Replay improves model-based reinforcement learning agents' adaptability.

Paper proposes BMPO to optimize policies using bidirectional models.

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-func…

State-of-the-art efficient model-based Reinforcement Learning (RL) algorithms typically act by iteratively solving empirical models, i.e., by performing \emph{full-planning} on Markov Decision Processes (MDPs) built by the gathered experience. In this paper, we focus on model-based RL in the finite-state finite-horizon…

Significant progress has been made in the area of model-based reinforcement learning. State-of-the-art algorithms are now able to match the asymptotic performance of model-free methods while being significantly more data efficient. However, this success has come at a price: state-of-the-art model-based methods require …

New RL framework models continuous-time dynamics using neural ODEs.

Traditional model-based reinforcement learning approaches learn a model of the environment dynamics without explicitly considering how it will be used by the agent. In the presence of misspecified model classes, this can lead to poor estimates, as some relevant available information is ignored. In this paper, we introd…

New hybrid RL algorithm outperforms model-free and model-based methods.

MAGE optimizes policies using action gradients from model-based learning.

Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, mode…

Proposes H-UCRL for efficient model-based RL with sublinear regret.

We prove strong consistency and asymptotic normality of least squares estimators for the subcritical Heston model based on continuous time observations. We also present some numerical illustrations of our results.

Improved model-based reinforcement learning for interactive dialogue tasks reduces sample needs and improves performance.

MOPO optimizes offline RL by penalizing dynamics uncertainty.

Proposes modifications to model-based forests for HTE estimation in observational data.

We study asymptotic properties of some (essentially conditional least squares) parameter estimators for the subcritical Heston model based on discrete time observations derived from conditional least squares estimators of some modified parameters.

A contraction analysis improves model-based RL's error recovery.

We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…

A method to reduce bias in model-based policy evaluation by shifting operators.