arXiv research

Study minimax-optimal rates for offline decision-making with function approximation.

A new algorithm for cryo-EM data collection that balances reward and latency.

CB-RL solves complex decision-making problems with contextual information and exogenous events.

Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.

New framework handles dynamic contexts in reinforcement learning.

The paper develops a reinforcement learning model to estimate ad impact considering delayed and cumulative effects.

We consider a planning problem where the dynamics and rewards of the environment depend on a hidden static parameter referred to as the context. The objective is to learn a strategy that maximizes the accumulated reward across all contexts. The new model, called Contextual Markov Decision Process (CMDP), can model a cu…

New algorithm reduces regret in RL with adversarial corruption.

In order to make good decision under uncertainty an agent must learn from observations. To do so, two of the most common frameworks are Contextual Bandits and Markov Decision Processes (MDPs). In this paper, we study whether there exist algorithms for the more general framework (MDP) which automatically provide the bes…

Improved algorithm for misspecified MLMDPs with bounded regret and space/time complexities.

We introduce Parameterized Exploration (PE), a simple family of methods for model-based tuning of the exploration schedule in sequential decision problems. Unlike common heuristics for exploration, our method accounts for the time horizon of the decision problem as well as the agent's current state of knowledge of the …

Human irrationality can improve AI design, study shows.

We consider a reinforcement learning (RL) setting in which the agent interacts with a sequence of episodic MDPs. At the start of each episode the agent has access to some side-information or context that determines the dynamics of the MDP for that episode. Our setting is motivated by applications in healthcare where ba…

We introduce the Contextual Graph Markov Model, an approach combining ideas from generative models and neural networks for the processing of graph data. It founds on a constructive methodology to build a deep architecture comprising layers of probabilistic models that learn to encode the structured information in an in…

Paper develops a new estimator for MDPs' risk functionals with lower variance and bias.

AACC improves RL performance in changing environments.

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

In this paper, we consider the problem of online learning of Markov decision processes (MDPs) with very large state spaces. Under the assumptions of realizable function approximation and low Bellman ranks, we develop an online learning algorithm that learns the optimal value function while at the same time achieving ve…

This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …

Minimax PAC bounds for learning in exogenous contextual MDPs

A new method scores contextual Markov networks without assuming chordality.

A new algorithm uses concavity in Gaussian processes to optimize decisions in bandit problems.

New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.

Unified framework for human-like decision making in various sequential tasks.

New algorithm for partially observable contexts in finance.

Robust OPE framework uses human inputs to improve policy evaluation in changing environments.

Novel algorithm reduces feature inclusion in online decision-making.

A new estimator for evaluating policies in unknown environments.

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.

RL applied to finance tasks, highlighting challenges and future directions.

Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…

Develops an actor-critic algorithm for risk-sensitive Markov decision processes.

Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.

Efficient algorithm for CMDPs reduces to offline density estimation.

Paper studies linear CMDPs, improving sample complexity for both models.

Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.

Novel framework for contextual anomaly detection models uncertainty.

Risk measures applied to dynamic Markov processes with varying risk aversion.

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…

Bayesian method infers local rules for collective animal movement.

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

New algorithm identifies best policy in MDPs faster.

Study off-policy evaluation and learning in dynamic pricing with context.

Adaptive anomaly detection for IoT data reduces delay without sacrificing accuracy.

We consider the task of Inverse Reinforcement Learning in Contextual Markov Decision Processes (MDPs). In this setting, contexts, which define the reward and transition kernel, are sampled from a distribution. In addition, although the reward is a function of the context, it is not provided to the agent. Instead, the a…

We develop robust Markov Decision Processes with risk measures for uncertain environments.