New approach to reinforcement learning that balances safety and performance against adversaries.
problem Balancing safety and performance in reinforcement learning against potential adversaries.
method Developed a new reinforcement learning framework that integrates interruptibility, resilience, and safe exploration.
result Achieved both interruptibility and resilience to adversaries without sacrificing optimal policy probability.
In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order to prevent dangerous situations from happening. Yet, as part of their learning process, agents may link these interruptions, that impact the…
Proposes E-MIIM for better mobile phone interruptions management.
problem Improper phone interruptions in daily life activities.
method Ensemble learning based on context-aware mobile telephony model.
result E-MIIM outperforms existing MIIM models in prediction accuracy.
SilentPhone identifies opportune times to silence phones to reduce interruptions.
problem Inappropriate phone notifications cause interruptions for users and others.
method Data-driven approach using past phone log data to infer unavailability.
result Identifies opportune moments for call interruptions and generates silent mode rules.
Deep RL predicts equipment maintenance from sensor data.
problem Equipment downtime due to sensor data overload.
method Model-free Deep Reinforcement Learning for optimal maintenance policy.
result Automatic maintenance policy learning from sensor data.
A new bandit problem where experiments can be interrupted if results are not promising.
problem Interruptible multi-armed bandit problem with a threshold for cumulative reward.
method Formalized survival regret, identified key components (regret and probability of ruin), derived lower bounds and optimal policies.
result No policy can achieve sublinear survival regret, but optimal policies minimize survival regret in a Pareto sense.
Using public data (Forbes Global 2000) we show that the asset sizes for the largest global firms follow a Pareto distribution in an intermediate range, that is ``interrupted'' by a sharp cut-off in its upper tail, where it is totally dominated by financial firms. This flattening of the distribution contrasts with a lar…
Safe linear stochastic bandits ensure safe exploration with optimal regret.
problem Ensuring safe exploration in stochastic bandits while minimizing regret.
method Combining known safe arms with exploratory arms to safely expand the set of safe arms over time.
result The algorithm achieves an expected regret of O ( T log ( T ) ) O(\sqrt{T}\log (T)) O ( T log ( T )) . CoCoRL learns safe constraints from demonstrations with unknown rewards.
problem Learning safe constraints from demonstrations with different unknown rewards.
method Convex Constraint Learning for Reinforcement Learning (CoCoRL) constructs a convex safe set based on demonstrations.
result CoCoRL learns constraints that lead to safe driving behavior and can safely transfer to different tasks and environments.
SafePILCO is a Python tool for safe reinforcement learning.
problem Safe and efficient policy synthesis in reinforcement learning.
method Extends PILCO algorithm with safety features, implemented in Python.
result Safe and data-efficient policy synthesis achieved.
Robust regression model for safe exploration in control problems.
problem Learning and exploring safely in sequential control problems.
method Deep robust regression model trained to predict uncertainty bounds.
result Empirically outperforms conventional GP-based safe exploration.
Safe learning of stochastic dynamics with safety constraints.
problem Learning controlled stochastic dynamics with safety constraints.
method Iterative expansion of a safe control set using kernel-based confidence bounds.
result The method ensures safe exploration and efficient estimation of system dynamics.
High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this …
Study on neural networks to identify redundancy issues in safe machine learning.
problem Identifying redundancy in neural network architectures for safe machine learning.
method Experiments with MNIST database using neural network classifiers.
result Underlines difficulties in using neural network classifiers for safe systems.
We study safe screening for metric learning. Distance metric learning can optimize a metric over a set of triplets, each one of which is defined by a pair of same class instances and an instance in a different class. However, the number of possible triplets is quite huge even for a small dataset. Our safe triplet scree…
Bitcoin fails to prove safe haven status during pandemic.
problem Determining if Bitcoin is a reliable safe haven asset during crises.
method Quantile correlations of Bitcoin with S&P500, VIX, and gold.
result Gold is a better safe haven during crises, not Bitcoin.
Proposes a method to accelerate safe sequential learning using offline data.
problem Limited exploration due to disconnected safe regions and slow task learning.
method Safe transfer sequential learning using Gaussian processes and offline data.
result Enhances global exploration across multiple disjoint safe regions with lower data consumption.
Meta-learning priors improves safe Bayesian optimization.
problem Optimizing robot controllers under safety constraints.
method Meta-learning priors from offline data using F-PACOH.
result Meta-learned priors accelerate safe BO convergence.
Revel tackles safe exploration in RL with verified symbolic policies.
problem Computational infeasibility of verifying neural networks in RL learning loops.
method Two policy classes: neurosymbolic with approximate gradients and symbolic policies for efficient verification. Mirror descent over policies to safely update and project policies.
result Revel discovers policies that outperform prior approaches to verified exploration.
Safe exploration framework for IML algorithms.
problem Safe decision-making in IML without unsafe outcomes.
method Exploits Gaussian process prior to efficiently learn safe decisions.
result Outperforms other algorithms empirically.
ASE safely explores unknown MDPs with unknown dynamics, improving sample efficiency.
problem Balancing exploration and safety in unknown MDPs with stochastic dynamics.
method Exploits analogies between state-action pairs to safely learn near-optimal policies.
result Empirically improves sample efficiency compared to existing methods.
Framework for safe reinforcement learning using expert demonstrations.
problem Ensuring safe behavior in reinforcement learning with unknown reward functions.
method Combines expert demonstrations with optimization methods to find safe reward functions.
result Trained agent safely avoids harmful states while mimicking expert behavior.
Safe reinforcement learning for autonomous vehicles using prediction constraints.
problem Safe reinforcement learning for safety-critical applications like autonomous vehicles.
method Use prediction to constrain exploration in reinforcement learning models.
result Successfully learned intersection handling behaviors on an autonomous vehicle.
Safe-House secures DeFi by limiting losses and enhancing security.
problem Ongoing hacks and security concerns in DeFi.
method Safe-House uses blockchain principles to secure asset movements.
result Safe-House limits maximum one-time loss to specified limits.
StageOpt efficiently optimizes safe decisions by separating safety and utility stages.
problem Optimizing unknown utility with safety constraints in sequential decisions.
method Develops StageOpt, a two-stage safe Bayesian optimization algorithm.
result StageOpt is more efficient and applicable to broader problems than existing methods.
OSIL learns safe policies from unsafe demonstrations.
problem Offline safe imitation learning with implicit safety.
method Formulates CMDP, infers safety from non-preferred trajectories, learns cost model.
result OSIL learns safer policies without degrading reward performance.
The problem of learning a sparse model is conceptually interpreted as the process of identifying active features/samples and then optimizing the model over them. Recently introduced safe screening allows us to identify a part of non-active features/samples. So far, safe screening has been individually studied either fo…
A new screening rule 'dynamic Sasvi' improves sparse optimization speed.
problem Sparse optimization problem identification.
method Flexible framework based on Fenchel-Rockafellar duality for norm-regularized least squares.
result Dynamic Sasvi can eliminate more features and increase solver speed.
This paper tackles safe global optimization of noisy functions with a Lipschitz condition.
problem Safe global maximization of expensive, noisy, Lipschitz functions.
method Develops a δ-Lipschitz framework and two algorithms to ensure safety constraints are met.
result The proposed methods ensure safety constraints are met before evaluating noisy functions.
New algorithm for safe bandits with unknown parameters and constraints.
problem Safety-critical systems with unknown parameters and constraints.
method Safe-LUCB algorithm with two phases: pure exploration and safe exploration-exploitation.
result General and problem-dependent regret bounds for the Safe-LUCB algorithm.
SRF learns sparse rule models by screening out features efficiently.
problem Learning optimal sparse rule models is computationally intractable due to the large number of possible rules.
method SRF uses meta safe screening (mSS) to efficiently screen out multiple features, improving the learning of sparse rule models.
result SRF provides a general framework for fitting sparse rule models and can handle group regularization.
Safe autonomous decisions made with machine learning predictions using Conformal Decision Theory.
problem Safe decisions from imperfect machine learning predictions.
method Conformal Decision Theory framework for producing safe decisions.
result Safe decisions with provable statistical guarantees of low risk.
Crypto-assets perform better than gold as safe-havens during market crashes.
problem Evaluating safe-haven properties of crypto-assets and gold during the 2020 market crash.
method Comparative analysis of Crypto-assets (Tether, Cardano, Dogecoin, Bitcoin, Ethereum, Litecoin, Ripple) and gold for European indices.
result Tether, Cardano, and Dogecoin exhibited hedging properties similar to gold, while gold was not more efficient as a safe-haven.
Safe Bayesian optimization method using information theory.
problem Optimizing unknown functions while respecting safety constraints.
method Information-theoretic exploration criterion for continuous domains.
result The method learns the value of the safe optimum up to arbitrary precision.
Safe sample screening improves RSVM performance without sacrificing accuracy.
problem Improving RSVM performance under noisy conditions.
method Proposed two safe sample screening rules based on CCCP framework for RSVM.
result Significant reduction in computational time for RSVM.
SafeRNet uses IoT and cloud computing to provide real-time safe routes.
problem High traffic fatality rates despite advanced technology.
method Bayesian network for safe route modeling.
result Demonstrated effectiveness with real traffic data.
Safe screening rules reduce computation time in logistic regression with ℓ 0 − ℓ 2 \ell_0-\ell_2 ℓ 0 − ℓ 2 regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
In high dimensional regression settings, sparsity enforcing penalties have proved useful to regularize the data-fitting term. A recently introduced technique called screening rules propose to ignore some variables in the optimization leveraging the expected sparsity of the solutions and consequently leading to faster s…
Safe-M 3 ^3 3 -UCRL learns safe policies for multi-agent systems with global constraints.
problem Global constraints in mean-field reinforcement learning for multi-agent systems.
method Safe-M 3 ^3 3 -UCRL uses epistemic uncertainty and log-barrier approach to ensure constraints satisfaction. result Safe-M 3 ^3 3 -UCRL learns safe policies for multi-agent systems with global constraints. SAMBA improves safe reinforcement learning with active exploration metrics.
problem Safe reinforcement learning in dynamic systems.
method Combines probabilistic modelling, information theory, and statistics. Uses novel metrics for out-of-sample Gaussian process evaluation.
result Orders of magnitude reduction in samples and violations compared to state-of-the-art methods.
Paper improves safe policy improvement with estimated baseline policy.
problem Unreliable batch Reinforcement Learning algorithms in real-world applications.
method Apply SPIBB algorithms with an estimated baseline policy.
result Safe policy improvement guarantees over true baseline without direct access.
During times of extreme market turmoil, it is acknowledged that there is a tendency towards "flight to safety". A strong (weak) safe haven is defined as an asset that has a significant positive (negative) return in periods where another asset is in distress, while hedge has to be negatively correlated (uncorrelated) on…
Adapts safe policies for exploration in high-risk settings.
problem Balancing safety and exploration in high-risk environments.
method Uses conformal calibration on a safe reference policy to determine aggressive action limits.
result Safe exploration improves performance without requiring model class identification or hyperparameter tuning.
Paper defines ε-Safe Decision Regions for exponential family distributions and approximates them for unbalanced data.
problem Need probabilistic guarantees for reliable predictions in machine learning.
method Formalizes ε-Safe Decision Regions, proves their form for exponential family distributions, and develops Multi Cost SVM for unbalanced data.
result Formal definition and analytical determination of ε-Safe Decision Regions for exponential family distributions.
WiseMove framework for safe deep RL in autonomous driving.
problem Ensuring safety in deep reinforcement learning for autonomous driving.
method Modular learning architecture for motion planning.
result Demonstrated on a common traffic scenario, WiseMove supports safe learning.
Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…
Safe actions learned in finite trials, without infinite exploration.
problem Learning safe actions in unknown environments efficiently.
method Defining a handicap metric and using sequential probability ratio test for discarding unsafe actions.
result Achieves constant handicap, discarding unsafe machines with probability one in finite rounds.
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.