arXiv research

ASE safely explores unknown MDPs with unknown dynamics, improving sample efficiency.

Revel tackles safe exploration in RL with verified symbolic policies.

Proposes a method to accelerate safe sequential learning using offline data.

Adapts safe policies for exploration in high-risk settings.

We study the problem of safe learning and exploration in sequential control problems. The goal is to safely collect data samples from operating in an environment, in order to learn to achieve a challenging control goal (e.g., an agile maneuver close to a boundary). A central challenge in this setting is how to quantify…

In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…

In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in applications, on real-world systems they must provably avoid unsafe decisions. To this end, safe IML algo…

SAMBA improves safe reinforcement learning with active exploration metrics.

Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…

Safe actions learned in finite trials, without infinite exploration.

Safe exploration method for RL under disturbance ensures safety with probabilistic guarantees.

Safe exploration in RF-RL doesn't increase sample complexity.

Safe learning of stochastic dynamics with safety constraints.

We show that when a third party, the adversary, steps into the two-party setting (agent and operator) of safely interruptible reinforcement learning, a trade-off has to be made between the probability of following the optimal policy in the limit, and the probability of escaping a dangerous situation created by the adve…

This paper proposes a method to safely adjust exploration in RL to satisfy constraints.

There have been numerous advances in reinforcement learning, but the typically unconstrained exploration of the learning process prevents the adoption of these methods in many safety critical applications. Recent work in safe reinforcement learning uses idealized models to achieve their guarantees, but these models do …

The paper tackles safe exploration in RL by a conservative safety critic.

Safe Bayesian optimization method using information theory.

Safe reinforcement learning with logical constraints for optimal policy synthesis.

Bandit algorithms have various application in safety-critical systems, where it is important to respect the system constraints that rely on the bandit's unknown parameters at every round. In this paper, we formulate a linear stochastic multi-armed bandit problem with safety constraints that depend (linearly) on an unkn…

Monotonicity helps in safely optimizing unknown functions.

Meta-learning priors improves safe Bayesian optimization.

New algorithm optimizes reward while ensuring safety in complex decision-making problems.

Using reinforcement learning to learn control policies is a challenge when the task is complex with potentially long horizons. Ensuring adequate but safe exploration is also crucial for controlling physical systems. In this paper, we use temporal logic to facilitate specification and learning of complex tasks. We combi…

In this paper, we study the problem of safe online learning to re-rank, where user feedback is used to improve the quality of displayed lists. Learning to rank has traditionally been studied in two settings. In the offline setting, rankers are typically learned from relevance labels created by judges. This approach has…

New method speeds up solving L0-regularized least-squares problems.

Safe RL with binary feedback using SABRE algorithm.

ARTEO algorithm optimizes safety-critical systems with uncertainty.

Machine learning can provide efficient solutions to the complex problems encountered in autonomous driving, but ensuring their safety remains a challenge. A number of authors have attempted to address this issue, but there are few publicly-available tools to adequately explore the trade-offs between functionality, scal…

Reinforcement learning algorithms rely on exploration to discover new behaviors, which is typically achieved by following a stochastic policy. In continuous control tasks, policies with a Gaussian distribution have been widely adopted. Gaussian exploration however does not result in smooth trajectories that generally c…

New method avoids failures in physics-constrained systems using active learning.

Reinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actions, which may be harmful for real-world systems. As a consequence, learning algorithms are rarely applied on safety-crit…

Safe screening improves generalized CGM's feature selection stability.

Develops a learning model predictive controller for competitive racing.

Safe RL in linear systems achieves $\sqrt{T}$-regret.

Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.

In this work we seek for an approach to integrate safety in the learning process that relies on a partly known state-space model of the system and regards the unknown dynamics as an additive bounded disturbance. We introduce a framework for safely learning a control strategy for a given system with an additive disturba…

StepMix algorithm ensures safe exploration in reinforcement learning with near-optimal performance.

Algorithm ensures safe optimization under unknown constraints.

This work establishes safe reinforcement learning for LQR with nonlinear baselines.

This paper presents a distributionally robust Q-Learning algorithm (DrQ) which leverages Wasserstein ambiguity sets to provide idealistic probabilistic out-of-sample safety guarantees during online learning. First, we follow past work by separating the constraint functions from the principal objective to create a hiera…

We introduce the safe linear stochastic bandit framework---a generalization of linear stochastic bandits---where, in each stage, the learner is required to select an arm with an expected reward that is no less than a predetermined (safe) threshold with high probability. We assume that the learner initially has knowledg…

Study best arm identification with safety constraints in bandit problems.

Safe active learning for time-series models with Gaussian processes.

CoCoRL learns safe constraints from demonstrations with unknown rewards.

Reinforcement learning (RL) for robotics is challenging due to the difficulty in hand-engineering a dense cost function, which can lead to unintended behavior, and dynamical uncertainty, which makes exploration and constraint satisfaction challenging. We address these issues with a new model-based reinforcement learnin…

SafePILCO is a Python tool for safe reinforcement learning.

High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this …