arXiv research

Improved FPL algorithms for adversarial MDPs with better regret bounds.

We introduce Threatened Markov Decision Processes (TMDPs) as an extension of the classical Markov Decision Process framework for Reinforcement Learning (RL). TMDPs allow suporting a decision maker against potential opponents in a RL context. We also propose a level-k thinking scheme resulting in a novel learning approa…

New framework for robust reinforcement learning policies in uncertain environments.

Algorithm for online decision making with unknown dynamics and aggregate feedback.

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner. We show $\tilde{O}(L|X|\sqrt{|A|T})$ regret bound, where $T$ is the number of episodes, $X$ is the state space, $A…

New method minimizes regret in AMDP with high probability.

In several reinforcement learning (RL) scenarios, mainly in security settings, there may be adversaries trying to interfere with the reward generating process. In this paper, we introduce Threatened Markov Decision Processes (TMDPs), which provide a framework to support a decision maker against a potential adversary in…

We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an algorithm whose regret with respect to any policy in a comparison class grows as t…

New model considers unfairness complaints to ensure multiple fairness criteria.

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

FTPL method shows near-optimal regret bounds for AMDPs with bandit feedback.

New approach for online learning with adaptive adversaries, simpler and more effective.

New algorithms achieve no-regret learning even with adversarial transitions and losses.

New RL algorithm tackles adversarial RMAB with unknown transitions and bandit feedback.

Two algorithms improve online reinforcement learning in adversarial linear MDPs with bandit feedback.

Develops robust ML systems for predictive uncertainties and adversarial examples.

New algorithms for learning MDPs with linear approximations in infinite-horizon settings.

New approach optimizes policies in adversarial MDPs using adversarial learning.

New algorithm reduces regret and constraint violation in adversarial CMDP learning.

Study optimal policy regret in partially observable Markov games with adaptive opponents.

New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.

New algorithms reduce dynamic regret in online MDPs with changing losses.

Study online learning in MDPs with aggregate bandit feedback, achieving low regret in both stochastic and adversarial settings.

A Markov Decision Process (MDP) is a popular model for reinforcement learning. However, its commonly used assumption of stationary dynamics and rewards is too stringent and fails to hold in adversarial, nonstationary, or multi-agent problems. We study an episodic setting where the parameters of an MDP can differ across…

New algorithm reduces regret in RL with adversarial corruption.

DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…

Optimistic PPO variant solves linear MDPs with improved regret bound.

Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.

Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…

Develops an actor-critic algorithm for risk-sensitive Markov decision processes.

Model predicts stock prices using GAN and RoI Pooling.

We consider Markov Decision Processes (MDPs) where the rewards are unknown and may change in an adversarial manner. We provide an algorithm that achieves state-of-the-art regret bound of $O( \sqrt{τ(\ln|S|+\ln|A|)T}\ln(T))$, where $S$ is the state space, $A$ is the action space, $τ$ is the mixing time of the MDP, and $…

Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.

This work tackles the problem of robust zero-shot planning in non-stationary stochastic environments. We study Markov Decision Processes (MDPs) evolving over time and consider Model-Based Reinforcement Learning algorithms in this setting. We make two hypotheses: 1) the environment evolves continuously with a bounded ev…

Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.

Risk measures applied to dynamic Markov processes with varying risk aversion.

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…

Bayesian method infers local rules for collective animal movement.

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

New algorithm identifies best policy in MDPs faster.

We develop efficient and sharp bounds on policy value under perturbations in MDPs.

We develop robust Markov Decision Processes with risk measures for uncertain environments.

Distributional reinforcement learning (distributional RL) has seen empirical success in complex Markov Decision Processes (MDPs) in the setting of nonlinear function approximation. However, there are many different ways in which one can leverage the distributional approach to reinforcement learning. In this paper, we p…

New algorithm identifies near-optimal policies in adversarial distributed RL settings.

Q-learning for average cost MDPs gets a concentration bound.