arXiv research

RL struggles with generalization due to implicit partial observability.

Enhanced feedback model improves sample-efficiency in POMDPs.

New method approximates POMDPs with PB-MDPs, providing error bounds and practical algorithms.

PS4POMDPs algorithm simplifies online learning for episodic POMDPs with unknown models.

Dynamical-VAE learns causal dynamics from POMDPs using future information.

This research develops an efficient reinforcement learning method for undercomplete POMDPs.

New algorithms learn POMDPs efficiently with hindsight observability.

Solves POMDPs with recurrent neural networks and natural policy gradient.

New algorithm learns POMDPs without computational oracles.

New algorithm proves RL from partial obs is feasible.

We solve POMDPs by approximating them as finite-state MDPs.

New algorithm for planning in observable POMDPs in quasi-polynomial time.

This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments. Off-policy evaluation under partial observability is inherently prone to bias, with risk of arbitrarily large errors. We define the problem of off-policy evaluation for Partially Observable Mark…

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

New method optimizes decision-making in uncertain environments.

New method for off-policy evaluation in POMDPs using future-dependent value functions.

Paper proposes a policy gradient method for confounded POMDPs.

New algorithm learns POMDPs with known observation model efficiently.

New framework learns policies for partially observable systems.

Efficient RL in large POMDPs with latent determinism and embeddings.

A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real…

FORBES learns flexible belief states for POMDPs using normalizing flows.

Partially observable Markov decision processes (POMDPs) are a powerful abstraction for tasks that require decision making under uncertainty, and capture a wide range of real world tasks. Today, effective planning approaches exist that generate effective strategies given black-box models of a POMDP task. Yet, an open qu…

Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often converge to maxima that are far from optimal. This paper considers a variable-size FSC to rep…

This paper establishes strong lower bounds for learning in revealing POMDPs.

We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, …

We solve a broad class of sequential decision-making problems with partially observed states.

Study on natural actor-critic for POMDPs with finite memory.

The paper develops methods to estimate POMDPs from partial information.

The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…

ETC learns minimal representations for reinforcement learning in POMDPs.

We consider the estimation of the policy gradient in partially observable Markov decision processes (POMDP) with a special class of structured policies that are finite-state controllers. We show that the gradient estimation can be done in the Actor-Critic framework, by making the critic compute a "value" function that …

Study optimizes inspection and monitoring of deteriorating structures using POMDPs.

In Passive POMDPs actions do not affect the world state, but still incur costs. When the agent is bounded by information-processing constraints, it can only keep an approximation of the belief. We present a variational principle for the problem of maintaining the information which is most useful for minimizing the cost…

A new reinforcement learning method for medical decisions with limited data.

Paper introduces a new framework to improve sample efficiency in POMDPs learning.

RL approach for target tracking with unknown dynamics and sensor control.

Bayesian reinforcement learning (BRL) encodes prior knowledge of the world in a model and represents uncertainty in model parameters by maintaining a probability distribution over them. This paper presents Monte Carlo BRL (MC-BRL), a simple and general approach to BRL. MC-BRL samples a priori a finite set of hypotheses…

Paper solves POMDPs in continuous time and discrete spaces.

Second-order methods fail to fully quantify epistemic uncertainty, leading to biased predictions.

Bayesian Neural Networks show unexpected collapse of epistemic uncertainty with large models and little data.

This paper advances sample-efficient learning for partially observable RL by introducing B-stability and new algorithms.

In real-world scenarios, the observation data for reinforcement learning with continuous control is commonly noisy and part of it may be dynamically missing over time, which violates the assumption of many current methods developed for this. We addressed the issue within the framework of partially observable Markov Dec…

Overparametrized neural networks retain significant epistemic uncertainty even with sufficient data.

DEUP directly predicts epistemic uncertainty, improving model optimization and exploration.

CLEAR calibrates both aleatoric and epistemic uncertainties for better predictive intervals.

Improved POMDP regret to sqrt(T) with known observation model.