arXiv research

The paper discusses safety assessment for AI systems, focusing on machine learning models.

RLVR maintains safety while improving reasoning capabilities in LLMs.

Fine-tuning LLMs improves capability but harms safety, study finds.

Safety filter for unknown discrete-time systems with learned models and noise covariance.

This paper formalizes AI safety using hypothesis testing in GenAI.

RAGuard improves safety in LLMs for offshore wind maintenance.

Researchers introduce a method to assess the safety of interpretable machine learning models.

Study best arm identification with safety constraints in bandit problems.

CARL safely adapts RL agents for safety-critical tasks.

Safe imitation learning with a safety layer for flexible training.

Failures are challenging for learning to control physical systems since they risk damage, time-consuming resets, and often provide little gradient information. Adding safety constraints to exploration typically requires a lot of prior knowledge and domain expertise. We present a safety measure which implicitly captures…

LLM safety alignment explained as divergence estimation.

Unified control theory and machine learning for safety in uncertain systems.

A new method reduces the computational burden of safety alignment for large language models.

High-dimensional models pose both safety benefits and risks.

By building on a recently introduced genetic-inspired attribute-based conceptual framework for safety risk analysis, we propose a novel methodology to compute construction univariate and bivariate construction safety risk at a situational level. Our fully data-driven approach provides construction practitioners and aca…

Safe reinforcement learning tackles safety constraints with linear approximations.

Each year, around 6 million car accidents occur in the U.S. on average. Road safety features (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety features is an important component of safety management systems for…

A new RL model ensures safe learning in uncertain environments.

Study evaluates conformal prediction methods for safety in vision models under shifts and long-tailed data.

A novel approach for safe offline RL using latent safety constraints.

Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we revi…

Machine learning algorithms increasingly influence our decisions and interact with us in all parts of our daily lives. Therefore, just as we consider the safety of power plants, highways, and a variety of other engineered socio-technical systems, we must also take into account the safety of systems involving machine le…

Reinforcement learning is a promising approach to synthesizing policies for challenging robotics tasks. A key problem is how to ensure safety of the learned policy---e.g., that a walking robot does not fall over or that an autonomous car does not run into an obstacle. We focus on the setting where the dynamics are know…

System predicts ice formation to improve road safety.

Survey of algorithms for testing AI-driven CPS safety.

Algorithm minimizes regret while adhering to unknown safety constraints.

A new framework ensures model safety by retaining old model capabilities while improving new tasks.

Urban traffic systems worldwide are suffering from severe traffic safety problems. Traffic safety is affected by many complex factors, and heavily related to all drivers' behaviors involved in traffic system. Drivers with aggressive driving behaviors increase the risk of traffic accidents. In order to manage the safety…

Safe learning of stochastic dynamics with safety constraints.

Model forecasts motor vehicle collision rates with high accuracy.

ELS framework improves safety alignment by dynamically steering LLMs towards helpful responses.

Reinforcement Learning (RL) algorithms have found limited success beyond simulated applications, and one main reason is the absence of safety guarantees during the learning process. Real world systems would realistically fail or break before an optimal controller can be learned. To address this issue, we propose a cont…

Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning appro…

Although aviation accidents are rare, safety incidents occur more frequently and require a careful analysis to detect and mitigate risks in a timely manner. Analyzing safety incidents using operational data and producing event-based explanations is invaluable to airline companies as well as to governing organizations s…

A new algorithm for identifying the best arm in linear feedback with safety constraints.

Machine learning algorithms are increasingly influencing our decisions and interacting with us in all parts of our daily lives. Therefore, just like for power plants, highways, and myriad other engineered sociotechnical systems, we must consider the safety of systems involving machine learning. In this paper, we first …

SafeML monitors ML systems for safety and security risks.

Meta-learning priors improves safe Bayesian optimization.

Paper defines and quantifies safety risks in deep neural networks.

A new error bound improves safety in Bayesian optimization.

AI systems that explain their decisions can be monitored for harmful intentions.

We propose Trusted Neural Network (TNN) models, which are deep neural network models that satisfy safety constraints critical to the application domain. We investigate different mechanisms for incorporating rule-based knowledge in the form of first-order logic constraints into a TNN model, where rules that encode safet…

Study examines sample complexity for RL with safety constraints.

Synthesizes machine learning applications in reliability and safety.

Develops a method to simulate rare dangerous events in autonomous systems.

SafeQ-Learning uses Wasserstein DRO to ensure safety in online learning.

Every year, thousands of people receive consumer product related injuries. Research indicates that online customer reviews can be processed to autonomously identify product safety issues. Early identification of safety issues can lead to earlier recalls, and thus fewer injuries and deaths. A dataset of product reviews …