arXiv research

Bayesian method infers local rules for collective animal movement.

We study the problem of online learning in a class of Markov decision processes known as linearly solvable MDPs. In the stationary version of this problem, a learner interacts with its environment by directly controlling the state transitions, attempting to balance a fixed state-dependent cost and a certain smooth cost…

Free Random Projection enhances reinforcement learning by naturally incorporating hierarchical structure.

New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.

Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…

Develops an actor-critic algorithm for risk-sensitive Markov decision processes.

Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.

Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.

Risk measures applied to dynamic Markov processes with varying risk aversion.

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

New algorithm identifies best policy in MDPs faster.

We develop robust Markov Decision Processes with risk measures for uncertain environments.

Q-learning for average cost MDPs gets a concentration bound.

New Q-learning method achieves optimal sample complexity for average-reward problems.

We introduce Threatened Markov Decision Processes (TMDPs) as an extension of the classical Markov Decision Process framework for Reinforcement Learning (RL). TMDPs allow suporting a decision maker against potential opponents in a RL context. We also propose a level-k thinking scheme resulting in a novel learning approa…

This paper analyzes risk-sensitive reinforcement learning with Conditional Value-at-Risk (CVaR) for robust Markov Decision Processes.

We characterize value functions in partially observable MDPs as semi-algebraic sets.

New RL framework models continuous-time dynamics using neural ODEs.

Study minimax-optimal rates for offline decision-making with function approximation.

Study minimax optimal RL in factored MDPs with bonus exploration.

The paper tackles robust policy learning in MDPs using statistical methods.

Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.

New algorithm reduces sample complexity for planning in MDPs.

Introduces DF framework for sampling decisions from target distributions.

We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Thompson Sampling (PSRL) philosophies, and work in the general setting of continuous state and action …

New RL approach tackles constrained Markov decision processes.

Develops new reinforcement learning methods for complex constrained decision-making problems.

The paper introduces a method to learn Markov state abstractions for reinforcement learning.

We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty in the latent parameters and the state space. We also replace the original Gaussi…

New methods improve temporal difference learning for policy evaluation in Markov decision processes.

Algorithm learns mixtures of Markov chains and MDPs from short trajectories.

This work extends reinforcement learning to handle non-cumulative objectives.

Studies on massive open online courses (MOOCs) users discuss the existence of typical profiles and their impact on the learning process of the students. However defining the typical behaviors as well as classifying the users accordingly is a difficult task. In this paper we suggest two methods to model MOOC users behav…

A new method for risk-averse decision-making in Markov processes with improved regret bounds.

A new algorithm trains experts to safely guide agents in partially observed environments.

Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

Develops methods for finding counterfactual explanations in sequential decision making.

Paper analyzes history-based RL methods for MDPs, introduces a theoretical framework and practical algorithm.

Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse …

The paper extends conformal prediction to MDP trajectories for autonomous systems.

Develops robust MDPs for unknown disturbances with performance guarantees.

The paper studies risk-sensitive MDPs with recursive risk measures.

Paper approximates solutions for complex decision processes with limited precision.