New approach to reinforcement learning that balances safety and performance against adversaries.
problem Balancing safety and performance in reinforcement learning against potential adversaries.
method Developed a new reinforcement learning framework that integrates interruptibility, resilience, and safe exploration.
result Achieved both interruptibility and resilience to adversaries without sacrificing optimal policy probability.
Safe-M 3 ^3 3 -UCRL learns safe policies for multi-agent systems with global constraints.
problem Global constraints in mean-field reinforcement learning for multi-agent systems.
method Safe-M 3 ^3 3 -UCRL uses epistemic uncertainty and log-barrier approach to ensure constraints satisfaction. result Safe-M 3 ^3 3 -UCRL learns safe policies for multi-agent systems with global constraints. In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order to prevent dangerous situations from happening. Yet, as part of their learning process, agents may link these interruptions, that impact the…
Framework for safe reinforcement learning using expert demonstrations.
problem Ensuring safe behavior in reinforcement learning with unknown reward functions.
method Combines expert demonstrations with optimization methods to find safe reward functions.
result Trained agent safely avoids harmful states while mimicking expert behavior.
New method provides scalable safety guarantees for RL agents.
problem Safe reinforcement learning in real-life scenarios.
method State-augmentation and shield design for probabilistic avoidance.
result Strict formal safety guarantees for RL agents, scalable and practical.
Framework trains safe agents avoiding deceptive behavior.
problem Training safe agents from unsafe incentives.
method Formal settings, causal influence analysis, maximizing non-mediated effects.
result Agents avoid manipulating delicate state for rewards.
A new algorithm trains experts to safely guide agents in partially observed environments.
problem Existing imitation learning methods for POMDPs can lead to sub-optimal or unsafe policies.
method Derive an objective to encourage the expert to maximize the agent's reward, then use it to train both expert and agent.
result The algorithm produces an expert policy that the agent can safely imitate, outperforming fixed expert policies.
DPG makes reinforcement learning safer with human advice.
problem Unsafe reinforcement learning in shared environments.
method Extends Policy Gradient to incorporate human directives.
result DPG learns faster and more safely than reward-based methods.
Adapts safe policies for exploration in high-risk settings.
problem Balancing safety and exploration in high-risk environments.
method Uses conformal calibration on a safe reference policy to determine aggressive action limits.
result Safe exploration improves performance without requiring model class identification or hyperparameter tuning.
OSIL learns safe policies from unsafe demonstrations.
problem Offline safe imitation learning with implicit safety.
method Formulates CMDP, infers safety from non-preferred trajectories, learns cost model.
result OSIL learns safer policies without degrading reward performance.
A new RL model ensures safe learning in uncertain environments.
problem Safe reinforcement learning in uncertain, partially observable environments.
method Lyapunov-based uncertainty quantification and Transformers for memory.
result Significant improvement in safety and optimality in grid-world tasks.
Paper tackles safe combinatorial semi-bandits with risk constraints.
problem Safe combinatorial semi-bandits with risk constraints.
method Formulated probably anytime-safe constraint, designed PASCombUCB algorithm.
result PASCombUCB is almost asymptotically optimal in minimizing regret.
Develops a learning model predictive controller for competitive racing.
problem Lack of exploration in state space and complexity in obstacle avoidance.
method Explores state space through multiple initializations and develops a new method for convex terminal set selection.
result Yields a richer terminal safe set and maintains convexity.
Safe reinforcement learning with nonconvex constraints using convex approximations.
problem Safe reinforcement learning with nonlinear function approximation.
method Constructing surrogate convex constrained optimization problems by replacing nonconvex functions with convex quadratic functions.
result Solutions to surrogate problems converge to a stationary point of the original nonconvex problem.
Safe reinforcement learning with logical constraints for optimal policy synthesis.
problem Ensuring safety during reinforcement learning while maximizing goal satisfaction.
method Adaptive safe padding that synthesizes optimal control policies satisfying temporal logic formulas.
result The proposed method handles the trade-off between exploration and safety with theoretical guarantees.
Safe reinforcement learning for continuous control using Lyapunov methods.
problem Ensuring safe interactions in continuous action reinforcement learning.
method Formulates CMDPs, uses Lyapunov-based projection for policy updates, integrates with PG methods.
result Demonstrates effectiveness in balancing performance and constraint satisfaction.
Safe RL approach using Lyapunov functions.
problem Concurrent optimization of performance and safety constraints in RL.
method Lyapunov-based approach for CMDPs, transforming DP and RL algorithms.
result Significant performance improvement in balancing constraints and performance.
Accumulator module improves reinforcement learning by delaying decisions based on evidence.
problem Incomplete information, limited sensing, and stochastic environments lead to risky decisions.
method Integrates evidence for each action, delays action until confident, using dynamic competition.
result Accumulator module outperforms traditional reinforcement learning methods in a guessing game.
The paper shows why AI safety doesn't generalize across tasks.
problem AI safety fails to generalize across unseen tasks.
method Theoretical analysis of linear-quadratic control with H ∞ H_{\infty} H ∞ -robustness, empirical demonstrations in simulated quadcopter navigation and CRM. result The mapping from task specification to an optimal controller has a higher Lipschitz constant with safety requirements than without, indicating inherent complexity of safety.
New approach to avoid bad incentives in reinforcement learning agents.
problem Designing safe reinforcement learning agents that avoid unnecessary disruptions.
method Break down side effects penalties into baseline state and deviation measure; introduce new stepwise inaction baseline and relative reachability deviation measure.
result Combination of new design choices avoids undesirable incentives, while simpler alternatives fail.
Insurance contracts for autonomous AI agents must be actuarially sound and resistant to gaming.
problem Designing insurance contracts for autonomous AI agents that are actuarially sound and resistant to gaming.
method Characterizing a five-attack space and proving the actuarial runtime is gaming-resistant.
result An incentive-compatible layer for actuarial control of autonomous-agent side effects.
Agents need world models to generalize multi-step tasks.
problem The necessity of world models for flexible, goal-directed behavior.
method Formal analysis and demonstration of the necessity of world models for agents to generalize multi-step tasks.
result World models are necessary for agents to generalize to multi-step goal-directed tasks.
DePAint solves MARL for agents with local constraints, privacy, and no central controller.
problem Training multi-agent systems to optimize rewards while adhering to safety constraints in a decentralized setting.
method Formulated as a decentralized constrained multi-agent Markov Decision Problem, proposed DePAint method using momentum-based decentralized policy gradient.
result First privacy-preserving fully decentralized MARL algorithm considering both peak and average constraints.
SafeMIL learns safer policies by avoiding risky behavior from non-preferred trajectories.
problem Learning safe imitation policies from non-preferred trajectories in risky environments.
method SafeMIL uses Multiple Instance Learning to learn a cost function from non-preferred trajectories.
result SafeMIL learns a safer policy that avoids non-preferred behaviors without sacrificing reward performance.
New risk metric for AI systems reduces safety risks with minimal data.
problem Risk assessment in multi-agent AI systems.
method Free Energy Principle applied to risk metrics, introducing Cumulative Risk Exposure.
result Gatekeepers improve system safety in autonomous vehicle fleets.
Safe RL with binary feedback using SABRE algorithm.
problem Safe reinforcement learning with binary safety feedback.
method SABRE algorithm, combining active learning and reinforcement learning.
result Provable safe policy with high probability, no unsafe actions during training.
Proposes Constrained Q-learning for reinforcement learning with constraints.
problem Optimizing multiple objectives while adhering to constraints in reinforcement learning.
method Directly restricts the action space in Q-update to learn optimal Q-function for constrained MDP.
result Improves safety and optimality in high-level decision making for autonomous driving.
In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…
This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.
problem Challenges in working with real robotic hardware, especially position-controlled robots.
method Combines RL with traditional force control techniques, implementing parallel position/force control and admittance control.
result Validated methods on both simulation and real robot (UR3 e-series) for force control.
Algorithm safely learns from sub-optimal baseline policies while satisfying constraints.
problem Safe reinforcement learning with constraints when baseline policy is sub-optimal.
method Iterative policy optimization alternating between return maximization, baseline distance minimization, and constraint projection.
result Consistently outperforms baselines, achieving 10x fewer constraint violations and 40% higher reward.
Reinforcement learning controls car speed for safe, efficient, and comfortable driving.
problem Safe, efficient, and comfortable car following during autonomous driving.
method Deep reinforcement learning with a reward function for safety, efficiency, and comfort.
result The model reduces dangerous minimum time to collision to 8% of human drivers and maintains efficient headways.
The paper develops an economic foundation for multi-agent learning in markets.
problem Learning dynamics in markets with strategic externalities.
method A two-phase incentive mechanism that estimates and uses implementable transfers to steer long-run dynamics.
result The mechanism achieves sublinear social-welfare regret and asymptotically optimal welfare under mild rationality and exploration conditions.
RFM models predict multi-agent behavior, offering insights and faster learning.
problem Understanding and improving multi-agent systems learning.
method Relational Forward Models (RFM) that learn to predict future agent behavior.
result RFM modules inside agents lead to faster learning compared to baseline methods.
A decentralized algorithm minimizes cumulative regret in stochastic linear bandits with safety constraints.
problem Efficiently solving a linear bandit-optimization problem over a network of agents with safety constraints.
method DLUCB: a fully decentralized algorithm that minimizes cumulative regret through UCB strategy and consensus procedure.
result Near-optimal regret performance of O ( d log N T N T ) \mathcal{O}(d\log{NT}\sqrt{NT}) O ( d log N T N T ) with O ( d N 2 ) \mathcal{O}(dN^2) O ( d N 2 ) communication rate. New RL approach tackles constrained Markov decision processes.
problem Applying RL to physical systems with safety constraints.
method Formulated as a Constrained Markov Decision Process (CMDP), introduced a safe policy improvement method.
result Agent learns to maximize returns while satisfying constraints.
Safe exploration in RF-RL doesn't increase sample complexity.
problem Achieving optimal policies with safety constraints in reward-free RL.
method Proposed SWEET framework for tabular and low-rank MDP settings, leveraging truncated value functions.
result Sample complexities match or outperform constraint-free counterparts, proving safety constraints have little impact.
New bounds on predicting agent behavior from behavior alone.
problem Predicting agent beliefs and intentions from observed behavior.
method Derivation of bounds on agent behavior in new environments under assumption of world model.
result Theoretical limits on predicting intentional agents from behavioral data.
Combines Lyapunov functions with controller synthesis for safe control policies.
problem Ensuring safety in controller design for nonlinear systems.
method Iterative algorithm combining Lyapunov function estimation and controller synthesis.
result Effective control policies with large safe regions are derived.
SafeCritic predicts safe trajectories for pedestrians and cyclists avoiding collisions.
problem Predicting safe trajectories for pedestrians and cyclists in urban environments.
method Generative adversarial networks and reinforcement learning to generate safe trajectories, evaluated by a Discriminator.
result Significant improvement over state-of-the-art models in safety classification.
The study improves gold's role as a hedge and safe haven using new correlation measures.
problem Investing in gold for medium and extreme market movements.
method Developed a new correlation measure based on fractal approach.
result Gold is a better hedge than safe haven for extreme market movements.
Machine learning identifies potential drugs for COVID-19.
problem Finding effective treatments for COVID-19.
method Trained neural network models on virus protein sequences and antiviral drugs.
result Identified potential drug candidates for treating COVID-19.
A new method for CMDP solving without compromising safety constraints.
problem Solving CMDP problems while adhering to safety constraints.
method Decomposition into reconnaissance and planning MDPs.
result Achieves safe policies for any safety constraint set.
Parenting algorithm improves AI safety by learning from human input.
problem Safety concerns in reinforcement learning, especially reward hacking and unsafe exploration.
method Inspired by parenting, a precise framework for learning from human input.
result Parenting algorithm solves safety problems in AI Safety gridworlds.
Study best arm identification with safety constraints in bandit problems.
problem Real-world decision-making with safety constraints.
method Analyzed linear and monotonic reward and safety constraints, proposed algorithms.
result Guaranteed safe learning in both linear and general reward/safety constraint settings.
A new runtime for AI agents calculates risks in real-time.
problem Managing risks and liabilities in autonomous AI actions.
method A time-consistent counterfactual actuarial layer with explicit underwriting boundaries.
result Establishes a well-defined toll and guarantees executed-action budgets.
Deep RL for autonomous highway driving avoids unexpected scenarios.
problem Unexpected scenarios in AV operation space lead to poor decision-making.
method Deep reinforcement learning with safety checks for decision-making.
result Enhanced learning efficiency and safe behavior in highway driving.
New algorithm optimizes reward while ensuring safety in complex decision-making problems.
problem Maximizing reward while adhering to safety constraints in complex decision-making problems.
method Optimistic Primal-Dual Proximal Policy Optimization (OPDOP) algorithm combining least-squares policy evaluation and a bonus term for safe exploration.
result Achieves i l d e O ( d H 2.5 T ) ilde{O}(d H^{2.5}\sqrt{T}) i l d e O ( d H 2.5 T ) regret and i l d e O ( d H 2.5 T ) ilde{O}(d H^{2.5}\sqrt{T}) i l d e O ( d H 2.5 T ) constraint violation. CARL safely adapts RL agents for safety-critical tasks.
problem Safety hazards in RL for safety-critical tasks.
method CARL combines model-based RL and cautious adaptation.
result CARL achieves higher rewards with fewer failures in safety-critical tasks.