arXiv research

New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.

Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.

We develop robust Markov Decision Processes with risk measures for uncertain environments.

Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.

A new method for risk-averse decision-making in Markov processes with improved regret bounds.

Q-learning for average cost MDPs gets a concentration bound.

New algorithm identifies best policy in MDPs faster.

Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.

This paper analyzes risk-sensitive reinforcement learning with Conditional Value-at-Risk (CVaR) for robust Markov Decision Processes.

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…

Develops new reinforcement learning methods for complex constrained decision-making problems.

Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.

Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…

New algorithm tackles constrained Markov decision processes with peak constraints.

Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse …

This paper solves the open problem of computing Bayes optimal prediction for decision trees using a Markov chain Monte Carlo method.

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

Novel algorithm for Markov decision processes using rank-one approximation.

The paper tackles robust policy learning in MDPs using statistical methods.

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we sh…

Decision makers, such as doctors and judges, make crucial decisions such as recommending treatments to patients, and granting bails to defendants on a daily basis. Such decisions typically involve weighting the potential benefits of taking an action against the costs involved. In this work, we aim to automate this task…

Study minimizes risk in MDPs with spectral measures.

New Q-learning method achieves optimal sample complexity for average-reward problems.

Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…

New algorithm reduces sample complexity for planning in MDPs.

This work extends reinforcement learning to handle non-cumulative objectives.

New methods improve temporal difference learning for policy evaluation in Markov decision processes.

Optimizes mobile notifications for multiple objectives using reinforcement learning.

Framework for optimizing portfolios under model uncertainty.

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

Proposes a Riemannian optimization for policy improvement in MDPs.

The Markov decision process (MDP) formulation used to model many real-world sequential decision making problems does not efficiently capture the setting where the set of available decisions (actions) at each time step is stochastic. Recently, the stochastic action set Markov decision process (SAS-MDP) formulation has b…

New RL theory reduces sample complexity for mixing MDPs.

We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of variables that is linear in the number of states. To address this issue, recent …

Bayesian framework for learning optimal action-value function in MDPs.

Guarantees for third-person imitation learning from offline data.

Develops methods for finding counterfactual explanations in sequential decision making.

Paper tackles non-monotonic resource utilization in sequential decision-making.

State variables are easily the most subtle dimension of sequential decision problems. This is especially true in the context of active learning problems (bandit problems") where decisions affect what we observe and learn. We describe our canonical framework that models {\it any} sequential decision problem, and present…

Optimizes control of hybrid systems with multiple switching processes.

Develops an actor-critic algorithm for risk-sensitive Markov decision processes.

New RL approach tackles constrained Markov decision processes.

Paper approximates solutions for complex decision processes with limited precision.

We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an algorithm whose regret with respect to any policy in a comparison class grows as t…

The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…

Reward suffices for convex MDPs, expanding RL to new problems.