arXiv research

The paper addresses statistical estimation in MDPs with confounders using instrumental variables.

New method for estimating and optimizing MDPs without stationarity.

We decode latent states in Block MDPs and learn near-optimal policies.

New method removes oracle and reduces memory usage for robust MDPs.

New method tackles MDPs by learning normalized representations efficiently.

We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value e…

New RL algorithm achieves nearly optimal performance for linear MDPs.

Efficient RL for linear MDPs with unknown transitions.

New spectral methods improve matrix estimation in RL with low-rank structure.

Algorithm learns mixtures of Markov chains and MDPs from short trajectories.

A new parallel algorithm for learning optimal policies in MDPs with low communication costs.

We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL$^+$ is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bo…

Approximate Newton methods are a standard optimization tool which aim to maintain the benefits of Newton's method, such as a fast rate of convergence, whilst alleviating its drawbacks, such as computationally expensive calculation or estimation of the inverse Hessian. In this work we investigate approximate Newton meth…

Paper optimizes learning MDPs with adversarial rewards using linear models.

We study the online estimation of the optimal policy of a Markov decision process (MDP). We propose a class of Stochastic Primal-Dual (SPD) methods which exploit the inherent minimax duality of Bellman equations. The SPD methods update a few coordinates of the value and policy estimates as a new state transition is obs…

Actor-Critic method achieves optimal regret for unichain MDPs.

Paper develops IV method for consistent OPE in confounded MDPs.

We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as possible. Similarly to active exploration in multi-armed bandit (MAB), states may have…

New algorithm reduces reinforcement learning regret for linear MDPs with unknown transitions.

New method for optimistic planning in MDPs using regularization.

Improved regret bound for adversarial MDPs with linear function approximation.

New RL algorithm maximizes CVaR in low-rank MDPs with provable efficiency.

MOReL learns offline RL policies using pessimistic MDPs.

Paper develops a new estimator for MDPs' risk functionals with lower variance and bias.

We develop efficient and sharp bounds on policy value under perturbations in MDPs.

This work explores efficient reinforcement learning with density features in low-rank MDPs.

Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent's behavior originates in its policy and MDP policies depend on both the stochastic system dynamics as well as the reward function, the …

Study preference-based reinforcement learning in episodic kernel MDPs.

Bayesian regularization improves policy performance in noisy MDPs.

Dividend yields have been widely used in previous research to relate stock market valuations to cash flow fundamentals. However, this approach relies on the assumption that dividend yields are stationary. Due to the failure to reject the hypothesis of a unit root in the classical dividend-price ratio for the US stock m…

Optimal sample complexity analysis for plug-in approach in average-reward MDPs.

Adaptive reduction scheme approximates optimal policy in regularized MDPs.

Improved off-policy evaluation for MDPs with weak distributional overlap.

We study Exo-MDPs to reduce sample complexity in reinforcement learning.

Improves reinforcement learning agent's scene-specific value function.

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…

We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the mean Bellman residual. Our algorithm uses a Kalman filter model to estimate those …

Logarithmic regret for continuous-time reinforcement learning.

PRL improves off-policy evaluation in partially observed MDPs.

Paper establishes new lower bounds for MDPs with changing transition kernels.

Trajectory data suffices for efficient RL in linear MDPs.

We consider the Markov Decision Process (MDP) of selecting a subset of items at each step, termed the Select-MDP (S-MDP). The large state and action spaces of S-MDPs make them intractable to solve with typical reinforcement learning (RL) algorithms especially when the number of items is huge. In this paper, we present …

RLMM extends psychometric models to larger tasks.

Prognosticator improves performance in non-stationary MDPs.

A quantum reinforcement learning algorithm reduces sample complexity.

In this paper, we propose a novel reinforcement- learning algorithm consisting in a stochastic variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs). Stochastic variance-reduced gradient (SVRG) methods have proven to be very successful in supervised learning. However, their adaptation…

Study on future-dependent value functions for off-policy evaluation in complex environments.

Off-policy evaluation for MNAR rewards in MDPs