arXiv research

Efficient algorithm for CMDPs reduces to offline density estimation.

New algorithm reduces regret in CMDPs without cancellation of errors.

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

Paper tackles offline CMDP problems with near-optimal algorithm and sample complexity bound.

New algorithm achieves sublinear regret in CMDPs without error cancellations.

Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updat…

Paper studies linear CMDPs, improving sample complexity for both models.

Practical reinforcement learning problems are often formulated as constrained Markov decision process (CMDP) problems, in which the agent has to maximize the expected return while satisfying a set of prescribed safety constraints. In this study, we propose a novel simulator-based method to approximately solve a CMDP pr…

New bounds for learning near-optimal policies in CMDPs with constraints.

Algorithm reduces episode count for CMDPs with constraints.

New algorithm solves constrained MDPs with generative model, reducing sample complexity.

Study uses DRL with Lagrangian relaxation to solve temporal control tasks with STL constraints.

OSIL learns safe policies from unsafe demonstrations.

A new algorithm finds optimal solutions for constrained decision processes.

Paper proposes a model-free algorithm for CMDPs with long-term constraints, achieving optimal regret bounds.

We consider a planning problem where the dynamics and rewards of the environment depend on a hidden static parameter referred to as the context. The objective is to learn a strategy that maximizes the accumulated reward across all contexts. The new model, called Contextual Markov Decision Process (CMDP), can model a cu…

New algorithm reduces sample complexity for safe reinforcement learning.

AACC improves RL performance in changing environments.

New method optimizes costly evaluations in Bayesian optimization.

We consider online learning for episodic stochastically constrained Markov decision processes (CMDPs), which plays a central role in ensuring the safety of reinforcement learning. Here the loss function can vary arbitrarily across the episodes, and both the loss received and the budget consumption are revealed at the e…

In many real-world reinforcement learning (RL) problems, besides optimizing the main objective function, an agent must concurrently avoid violating a number of constraints. In particular, besides optimizing performance it is crucial to guarantee the safety of an agent during training as well as deployment (e.g. a robot…

Study examines sample complexity for RL with safety constraints.

Proposes method to discover diverse near-optimal policies in reinforcement learning.

Develops new reinforcement learning methods for complex constrained decision-making problems.

A new RL model ensures safe learning in uncertain environments.

We study the Safe Reinforcement Learning (SRL) problem using the Constrained Markov Decision Process (CMDP) formulation in which an agent aims to maximize the expected total reward subject to a safety constraint on the expected total value of a utility function. We focus on an episodic setting with the function approxi…

This paper presents a distributionally robust Q-Learning algorithm (DrQ) which leverages Wasserstein ambiguity sets to provide idealistic probabilistic out-of-sample safety guarantees during online learning. First, we follow past work by separating the constraint functions from the principal objective to create a hiera…

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

We consider a reinforcement learning (RL) setting in which the agent interacts with a sequence of episodic MDPs. At the start of each episode the agent has access to some side-information or context that determines the dynamics of the MDP for that episode. Our setting is motivated by applications in healthcare where ba…

Paper uses RL to solve constrained combinatorial optimization problems.

CoCoRL learns safe constraints from demonstrations with unknown rewards.

New algorithm for reinforcement learning in uncertain environments with unknown thresholds.

Survey of mathematical foundations for reinforcement learning.

ClauseLens uses reinforcement learning to price reinsurance treaties transparently and auditably.

The paper develops a reinforcement learning model to estimate ad impact considering delayed and cumulative effects.

FedSGM tackles constrained federated learning with unified framework.

Paper tackles robust MDPs with sample complexity guarantees.

CB-RL solves complex decision-making problems with contextual information and exogenous events.

New algorithm tackles constrained Markov decision processes with peak constraints.

We study continuous action reinforcement learning problems in which it is crucial that the agent interacts with the environment only through safe policies, i.e.,~policies that do not take the agent to undesirable situations. We formulate these problems as constrained Markov decision processes (CMDPs) and present safe p…

We consider reinforcement learning (RL) in Markov Decision Processes in which an agent repeatedly interacts with an environment that is modeled by a controlled Markov process. At each time step $t$, it earns a reward, and also incurs a cost-vector consisting of $M$ costs. We design model-based RL algorithms that maximi…