StageOpt efficiently optimizes safe decisions by separating safety and utility stages.
problem Optimizing unknown utility with safety constraints in sequential decisions.
method Develops StageOpt, a two-stage safe Bayesian optimization algorithm.
result StageOpt is more efficient and applicable to broader problems than existing methods.
Safe exploration framework for IML algorithms.
problem Safe decision-making in IML without unsafe outcomes.
method Exploits Gaussian process prior to efficiently learn safe decisions.
result Outperforms other algorithms empirically.
Accumulator module improves reinforcement learning by delaying decisions based on evidence.
problem Incomplete information, limited sensing, and stochastic environments lead to risky decisions.
method Integrates evidence for each action, delays action until confident, using dynamic competition.
result Accumulator module outperforms traditional reinforcement learning methods in a guessing game.
ARTEO algorithm optimizes safety-critical systems with uncertainty.
problem Decision-making under uncertainty with safety constraints in real-time optimization.
method ARTEO algorithm uses multi-armed bandits as a mathematical programming problem subject to safety constraints, learning unknown characteristics through exploration and incorporating uncertainty quantification.
result ARTEO achieves less cumulative regret with accurate and safe decisions.
Safe Bayesian optimization method using information theory.
problem Optimizing unknown functions while respecting safety constraints.
method Information-theoretic exploration criterion for continuous domains.
result The method learns the value of the safe optimum up to arbitrary precision.
Safe Gaussian Process Bandit Optimization with sub-linear regret bounds.
problem Sequential decision-making under uncertainty and safety constraints.
method Developed SGP-UCB, a safe variant of GP-UCB with modifications to respect safety constraints.
result First sub-linear regret bounds for safe Gaussian Process Bandit Optimization.
A deep reinforcement learning method with rule-based constraints improves safe and efficient lane changes in autonomous driving.
problem Complex and uncertain traffic environment challenges autonomous driving decision-making.
method Deep Q-Network (DQN) combined with rule-based constraints for lane change decision-making.
result The proposed rule-based DQN method outperforms both rule-based and DQN approaches in a real-world simulator.
Safe reinforcement learning framework using optimal transport for robustness.
problem Robustness and safety in deep reinforcement learning with limited data assumptions.
method Optimal transport perturbations to construct worst-case virtual state transitions.
result Significantly improved safety at deployment time compared to standard methods.
Safe autonomous decisions made with machine learning predictions using Conformal Decision Theory.
problem Safe decisions from imperfect machine learning predictions.
method Conformal Decision Theory framework for producing safe decisions.
result Safe decisions with provable statistical guarantees of low risk.
Investigates safe decision-making in interactive environments.
problem Learning the best safe decision in real-time systems.
method Reduces to a constrained linear bandits problem, proposes adaptive experimental design-based algorithm.
result First results on best-arm identification in linear bandits with safety constraints.
A new RL model ensures safe learning in uncertain environments.
problem Safe reinforcement learning in uncertain, partially observable environments.
method Lyapunov-based uncertainty quantification and Transformers for memory.
result Significant improvement in safety and optimality in grid-world tasks.
Risk-averse model uncertainty framework for safe reinforcement learning.
problem Safe decision making in uncertain environments.
method Risk-averse perspective towards model uncertainty using coherent distortion risk measures; equivalent to distributionally robust safe reinforcement learning problems; efficient, model-free implementation.
result Demonstrates robust performance and safety across perturbed test environments.
Paper defines ε-Safe Decision Regions for exponential family distributions and approximates them for unbalanced data.
problem Need probabilistic guarantees for reliable predictions in machine learning.
method Formalizes ε-Safe Decision Regions, proves their form for exponential family distributions, and develops Multi Cost SVM for unbalanced data.
result Formal definition and analytical determination of ε-Safe Decision Regions for exponential family distributions.
Examines fairness in ML for health, highlighting its importance and challenges.
problem Ensuring fairness in ML models for health to prevent health disparities.
method Reviews fairness notions in ML for health, including group, individual, and causal-based approaches.
result Discusses the importance and challenges of fairness in health-focused ML applications.
Algorithm reduces regret in safe Bayesian optimization with monotonicity constraints.
problem Sequentially maximize unknown function with safety constraints.
method Sequential algorithms using Gaussian processes with safety constraints modeled as monotonicity.
result Sublinear regret achieved for expanding safe region and finding optimal s s s . An important problem in sequential decision-making under uncertainty is to use limited data to compute a safe policy, i.e., a policy that is guaranteed to perform at least as well as a given baseline strategy. In this paper, we develop and analyze a new model-based approach to compute a safe policy when we have access …
Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.
problem Finding safe zones in policy Markov Decision Processes to limit trajectory escape.
method Bi-criteria approximation learning algorithm with polynomial sample complexity.
result Achieves almost 2 approximation for both escape probability and safe zone size.
This paper proposes a method to safely adjust exploration in RL to satisfy constraints.
problem Unsafe exploration in reinforcement learning violates constraints on controlled object states.
method Automatic adjustment of exploration inputs and variance-covariance matrix for safety.
result The method guarantees satisfaction of joint chance constraints with specified probability.
Paper tackles safe combinatorial semi-bandits with risk constraints.
problem Safe combinatorial semi-bandits with risk constraints.
method Formulated probably anytime-safe constraint, designed PASCombUCB algorithm.
result PASCombUCB is almost asymptotically optimal in minimizing regret.
New method provides scalable safety guarantees for RL agents.
problem Safe reinforcement learning in real-life scenarios.
method State-augmentation and shield design for probabilistic avoidance.
result Strict formal safety guarantees for RL agents, scalable and practical.
IDAS approach for autonomous vehicles to make decisions under merging scenarios.
problem Decision making for autonomous vehicles in merging scenarios with varying driver cooperativeness.
method IDAS approach using multi-agent reinforcement learning (MARL) with curriculum learning and masking mechanism.
result IDAS approach can handle uncertainties in real-world scenarios and make strategic decisions.
In many real-world reinforcement learning (RL) problems, besides optimizing the main objective function, an agent must concurrently avoid violating a number of constraints. In particular, besides optimizing performance it is crucial to guarantee the safety of an agent during training as well as deployment (e.g. a robot…
New algorithm optimizes reward while ensuring safety in complex decision-making problems.
problem Maximizing reward while adhering to safety constraints in complex decision-making problems.
method Optimistic Primal-Dual Proximal Policy Optimization (OPDOP) algorithm combining least-squares policy evaluation and a bonus term for safe exploration.
result Achieves i l d e O ( d H 2.5 T ) ilde{O}(d H^{2.5}\sqrt{T}) i l d e O ( d H 2.5 T ) regret and i l d e O ( d H 2.5 T ) ilde{O}(d H^{2.5}\sqrt{T}) i l d e O ( d H 2.5 T ) constraint violation. A new algorithm trains experts to safely guide agents in partially observed environments.
problem Existing imitation learning methods for POMDPs can lead to sub-optimal or unsafe policies.
method Derive an objective to encourage the expert to maximize the agent's reward, then use it to train both expert and agent.
result The algorithm produces an expert policy that the agent can safely imitate, outperforming fixed expert policies.
CoCoRL learns safe constraints from demonstrations with unknown rewards.
problem Learning safe constraints from demonstrations with different unknown rewards.
method Convex Constraint Learning for Reinforcement Learning (CoCoRL) constructs a convex safe set based on demonstrations.
result CoCoRL learns constraints that lead to safe driving behavior and can safely transfer to different tasks and environments.
In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…
New algorithm learns safe policies in unknown environments.
problem Learning safe policies in unknown, potentially unsafe environments.
method C-UCRL: Upper Confidence Reinforcement Learning for constrained MDPs.
result Achieves sub-linear regret while satisfying constraints.
Improves pre-trial risk assessments by making them safer without changing existing rules.
problem Improving pre-trial risk assessments while maintaining deterministic rules.
method Developed a maximin robust optimization approach to find a safer policy.
result Can safely improve certain components of the risk assessment instrument.
A new method for CMDP solving without compromising safety constraints.
problem Solving CMDP problems while adhering to safety constraints.
method Decomposition into reconnaissance and planning MDPs.
result Achieves safe policies for any safety constraint set.
Deep RL for autonomous highway driving avoids unexpected scenarios.
problem Unexpected scenarios in AV operation space lead to poor decision-making.
method Deep reinforcement learning with safety checks for decision-making.
result Enhanced learning efficiency and safe behavior in highway driving.
Develops new methods for risk-aware decision-making in medical bandits.
problem Risk-averse decision-making in medical contexts with limited data.
method Safe, anytime-valid concentration bounds, risk-aware contextual bandits, nonparametric algorithms.
result Improved decision-making algorithms for postoperative patient follow-up.
A framework identifies worst-case decision points in safety-critical scenarios, improving risk assessment by 10 hours.
problem Identifying worst-case outcomes in safety-critical decision-making under uncertainty.
method Explicitly estimating distributions of expected return to identify dead-ends, tuning based on risk tolerance.
result Significantly improves risk assessment, providing indications 10 hours earlier and increasing detection by 20%.
Safety is a desirable property that can immensely increase the applicability of learning algorithms in real-world decision-making problems. It is much easier for a company to deploy an algorithm that is safe, i.e., guaranteed to perform at least as well as a baseline. In this paper, we study the issue of safety in cont…
The paper uses deep reinforcement learning to control autonomous lane changes safely.
problem Safe and efficient autonomous lane changes in vehicles.
method Deep Q-networks and quadratic approximators for decision-making and control.
result Demonstrated effectiveness in simulations for decision-making and control.
Study best arm identification with safety constraints in bandit problems.
problem Real-world decision-making with safety constraints.
method Analyzed linear and monotonic reward and safety constraints, proposed algorithms.
result Guaranteed safe learning in both linear and general reward/safety constraint settings.
Paper proposes RL for real-time smart grid cyber attack detection.
problem Real-time detection of cyber-attacks in smart grids.
method Formulated as POMDP, uses model-free reinforcement learning.
result Effective in timely and accurate detection of cyber-attacks.
Bitcoin fails to prove safe haven status during pandemic.
problem Determining if Bitcoin is a reliable safe haven asset during crises.
method Quantile correlations of Bitcoin with S&P500, VIX, and gold.
result Gold is a better safe haven during crises, not Bitcoin.
OSIL learns safe policies from unsafe demonstrations.
problem Offline safe imitation learning with implicit safety.
method Formulates CMDP, infers safety from non-preferred trajectories, learns cost model.
result OSIL learns safer policies without degrading reward performance.
Develops optimal uncertainty quantification for risk-averse decision makers.
problem Quantifying prediction uncertainty for risk-sensitive domains.
method Decision-theoretic foundations connecting uncertainty quantification with risk-averse decision-making.
result Risk-Averse Calibration (RAC) algorithm provides optimal prediction sets for risk-averse decision makers.
New risk metric for AI systems reduces safety risks with minimal data.
problem Risk assessment in multi-agent AI systems.
method Free Energy Principle applied to risk metrics, introducing Cumulative Risk Exposure.
result Gatekeepers improve system safety in autonomous vehicle fleets.
New framework uses OR to ensure AI systems make safe decisions.
problem Ensuring generative AI systems make safe decisions as they gain autonomy.
method Developed a conceptual framework combining flow-based models and adversarial robustness.
result Increased autonomy requires new OR approaches for feasibility, robustness, and stress testing.
Safe actions learned in finite trials, without infinite exploration.
problem Learning safe actions in unknown environments efficiently.
method Defining a handicap metric and using sequential probability ratio test for discarding unsafe actions.
result Achieves constant handicap, discarding unsafe machines with probability one in finite rounds.
Fair ML systems can be safe ML systems by considering uncertainty.
problem Safety of data-driven decision systems is often neglected.
method Viewing ML systems as socio-technical, uncertainty-aware modeling.
result Fair models should be uncertainty-aware, e.g. through distributional regression.
DPG makes reinforcement learning safer with human advice.
problem Unsafe reinforcement learning in shared environments.
method Extends Policy Gradient to incorporate human directives.
result DPG learns faster and more safely than reward-based methods.
Proposes novel method for detecting novel scenarios in autonomous systems.
problem Detecting when a machine learning model makes a trustworthy prediction in dynamic, real-world situations.
method Leverages trained model's learned information and a new image similarity metric.
result Demonstrates the method's efficacy on real-world driving and indoor racing datasets.
Proposes Constrained Q-learning for reinforcement learning with constraints.
problem Optimizing multiple objectives while adhering to constraints in reinforcement learning.
method Directly restricts the action space in Q-update to learn optimal Q-function for constrained MDP.
result Improves safety and optimality in high-level decision making for autonomous driving.
Paper introduces a new power-dominance axis in estimator design.
problem Estimator design trade-off between bias and variance.
method Introduces a third power regime, `power-dominant', with an unavoidable error penalty.
result Any estimator in the `power-dominant' regime is structurally sub-optimal.
This study examines representation bias in open-source Qwen models for investment decisions.
problem Representation bias in financial applications of large language models.
method Balanced round-robin prompting over 150 U.S. equities, constrained decoding, token-logit aggregation.
result Firm size and valuation increase model confidence, while risk factors decrease it.