Model forecasts motor vehicle collision rates with high accuracy.
problem Forecasting motor vehicle collision rates with high accuracy.
method Adopted Heston Stochastic Volatility model and extended it to account for seasonality and accelerated safety periods.
result Short-term forecasts show high accuracy (over 95%) and outperform existing models.
CARL safely adapts RL agents for safety-critical tasks.
problem Safety hazards in RL for safety-critical tasks.
method CARL combines model-based RL and cautious adaptation.
result CARL achieves higher rewards with fewer failures in safety-critical tasks.
ViTaX provides formal guarantees for targeted explanations in safety-critical systems.
problem Need trustworthy explanations for safety-critical deep neural networks.
method Formal reachability analysis for targeted, semifactual explanations.
result First method to provide formally guaranteed explanations of model resilience.
Adversarial attacks pose a threat to deep neural networks, especially in safety-critical applications.
problem Adversarial attacks can misclassify deep neural networks, leading to safety issues.
method Adversarial attacks are categorized into white-box and black-box attacks based on the attacker's knowledge. They can be targeted or non-targeted.
result Adversarial attacks are effective and can transfer between different models and real-world scenarios.
Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.
problem Optimality in safety-constrained Markov decision processes with multichain structure.
method Formulated as a zero-sum game, constructed asynchronous value iteration scheme and modified Q-learning algorithm.
result Resolved Bellman's principle of optimality for multichain Markov decision processes and provided learning algorithms.
The paper tackles safe exploration in RL by a conservative safety critic.
problem Safe exploration in reinforcement learning (RL) when partially trained policies are deployed.
method Learning a conservative safety estimate through a critic, provably bounding catastrophic failures.
result The approach provably converges to competitive task performance with significantly lower catastrophic failure rates.
ARTEO algorithm optimizes safety-critical systems with uncertainty.
problem Decision-making under uncertainty with safety constraints in real-time optimization.
method ARTEO algorithm uses multi-armed bandits as a mathematical programming problem subject to safety constraints, learning unknown characteristics through exploration and incorporating uncertainty quantification.
result ARTEO achieves less cumulative regret with accurate and safe decisions.
Safe Bayesian Optimization algorithms are improved to ensure safety in real-world applications.
problem Ensuring safety in Bayesian Optimization algorithms for real-world applications.
method Investigated and improved three safety-related issues of SafeOpt-type algorithms: frequentist uncertainty bounds, RKHS norm assumptions, and discrete search spaces.
result Introduced Real-{eta}-SafeOpt, Lipschitz-only Safe Bayesian Optimization (LoSBO), and Lipschitz-only GP-UCB (LoS-GP-UCB) algorithms that retain safety guarantees and superior performance.
Online monitoring system for safety classifiers with shift detection and conformal adaptation
problem Detecting and adapting to distributional shifts in deployed safety classifiers
method Calibrated sequential statistics for online monitoring, conformal abstention for adaptation
result 86.6% valid detection with mean latency of 39.5 steps
ContextBench benchmarks methods for generating linguistically fluent inputs that activate specific latent features in language models.
problem Identifying inputs that trigger specific behaviours or latent features in language models.
method Context modification and benchmarking methods like Evolutionary Prompt Optimisation (EPO) with LLM-assistance and diffusion model inpainting.
result Enhanced methods achieve state-of-the-art performance in balancing elicitation effectiveness and fluency.
SEEDA optimizes dose allocation in clinical trials to balance efficacy and safety.
problem Complex relationships between efficacy and toxicity in new drug trials.
method Adaptive clinical trial methodology that maximizes cumulative efficacy while ensuring safety constraints.
result SEEDA outperforms existing methods in finding optimal doses with higher success rates and fewer patients.
Safe Bayesian optimization tackles safety constraints in control engineering.
problem Handling safety constraints in parameter tuning of control systems.
method Lipschitz-only Safe Bayesian Optimization (LoSBO) and LoS-GP-UCB.
result SafeBO algorithms can violate safety constraints due to unreliable uncertainty bounds.
A new framework ensures model safety by retaining old model capabilities while improving new tasks.
problem Catastrophic forgetting in continual learning systems.
method Retention-centric framework with data-dependent constraints and an efficient constrained optimization algorithm.
result The method ensures that new models retain important capabilities of old models while improving new tasks.
Quantile regression attacks outperform shadow models in unseen class membership inference attacks.
problem Failure of shadow model attacks on unseen classes due to limited data access.
method Quantile regression attacks that learn features of member examples.
result Quantile regression attacks achieve up to 11x higher TPR than shadow model-based approaches.
Simple adaptive attacks can jailbreak state-of-the-art safety-aligned language models.
problem Vulnerability of safety-aligned language models to simple adaptive attacks.
method Designing adversarial prompt templates, random search on suffixes, and transfer/prefilling attacks.
result 100% attack success rate on various models including GPT-4, Vicuna, Mistral, and Claude.
Proposes a method to accelerate safe sequential learning using offline data.
problem Limited exploration due to disconnected safe regions and slow task learning.
method Safe transfer sequential learning using Gaussian processes and offline data.
result Enhances global exploration across multiple disjoint safe regions with lower data consumption.
This paper uses MIL and MHCNN-RNN to predict precursors to aviation safety events.
problem Identifying events that precede aviation safety incidents.
method Multiple-instance learning (MIL) framework combined with a Multi-Head Convolutional Neural Network-Recurrent Neural Network (MHCNN-RNN) architecture.
result Multiple binary classifiers outperform in predicting high speed and high path angle events during the approach phase.
Automated testing framework finds weaknesses in deep control policies.
problem Safety of deep neural network control policies is difficult to validate.
method Adversarial reinforcement learning to test and find weaknesses.
result Framework finds weaknesses not evident during manual testing.
We present a novel human-aware navigation approach, where the robot learns to mimic humans to navigate safely in crowds. The presented model, referred to as DeepMoTIon, is trained with pedestrian surveillance data to predict human velocity in the environment. The robot processes LiDAR scans via the trained network to n…
DL models for MTS regression are vulnerable to adversarial attacks, posing risks in safety-critical applications.
problem Vulnerability of DL models to adversarial examples in MTS regression.
method Adversarial attack generation techniques from image classification were adapted for MTS.
result All state-of-the-art DL regression models (CNN, LSTM, GRU) are vulnerable to adversarial attacks.
RLVR maintains safety while improving reasoning capabilities in LLMs.
problem Safety-capability tradeoff in fine-tuning LLMs.
method Reinforcement Learning with Verifiable Rewards (RLVR) and theoretical analysis.
result RLVR can enhance reasoning while maintaining safety guardrails.
The paper discusses safety assessment for AI systems, focusing on machine learning models.
problem Safety assessment of AI systems, especially machine learning models, in safety-related applications.
method Analyzed AI models as statistical models and proposed a new budget allocation for AI safety.
result Safety assessment of AI systems requires a new approach focusing on the model used, not just the system.
Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we revi…
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
In this work we seek for an approach to integrate safety in the learning process that relies on a partly known state-space model of the system and regards the unknown dynamics as an additive bounded disturbance. We introduce a framework for safely learning a control strategy for a given system with an additive disturba…
RAGuard improves safety in LLMs for offshore wind maintenance.
problem Conventional LLMs fail with specialised or unexpected scenarios in offshore wind maintenance.
method Integrates safety-critical documents alongside technical manuals in RAG framework.
result RAGuard increases safety recall from almost 0% to over 50% while maintaining technical recall above 60%.
Survey of algorithms for testing AI-driven CPS safety.
problem Testing AI-driven CPS for safety in complex environments.
method Survey of applied algorithms for safety validation.
result Survey of existing tools and techniques for safety validation.
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.
Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning appro…
This paper formalizes AI safety using hypothesis testing in GenAI.
problem Ensuring safety of generative AI tools that create realistic content.
method Formalization of computational safety through hypothesis testing and signal processing.
result Demonstrates how AI safety can be assessed quantitatively using mathematical frameworks.
A new algorithm for identifying the best arm in linear feedback with safety constraints.
problem Identifying the best arm in linear feedback with safety constraints.
method A gap-based algorithm that ensures safety while minimizing sample complexity.
result The algorithm achieves meaningful sample complexity while ensuring safety.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
Paper defines and quantifies safety risks in deep neural networks.
problem Safety concerns in deep neural networks applied to critical sectors.
method Defines safety property, computes maximum safe radius, identifies new risk class, develops algorithm.
result Method achieves competitive performance in safety quantification.
PWSHAP provides targeted explanations for complex models.
problem Inability of black-box models to explain targeted effects in sensitive domains.
method Augments model with DAG, uses Shapley values for causal pathway identification.
result Establishes error bounds and demonstrates resolution, interpretability, and locality.
Shai is a 10B model for asset management tasks, outperforming baselines.
problem Improving performance in asset management tasks.
method Continuous pre-training and fine-tuning on asset management-specific data.
result Shai outperforms baseline models in asset management tasks.
Study best arm identification with safety constraints in bandit problems.
problem Real-world decision-making with safety constraints.
method Analyzed linear and monotonic reward and safety constraints, proposed algorithms.
result Guaranteed safe learning in both linear and general reward/safety constraint settings.
Failures are challenging for learning to control physical systems since they risk damage, time-consuming resets, and often provide little gradient information. Adding safety constraints to exploration typically requires a lot of prior knowledge and domain expertise. We present a safety measure which implicitly captures…
Paper optimizes industrial refrigeration using adaptive exploration.
problem Challenges in optimizing real-time industrial processes with unknown characteristics and safety constraints.
method Adaptive and explorative real-time optimization framework with Gaussian process uncertainty quantification.
result Approach increases energy efficiency of refrigeration process, approximating complete information solutions.
New algorithm constructs PAC prediction sets for meta-learning.
problem Uncertainty quantification in safety-critical systems.
method Proposes a novel algorithm to construct PAC prediction sets.
result Prediction sets satisfy a PAC guarantee with high probability over future tasks.
Safe reinforcement learning tackles safety constraints with linear approximations.
problem Ensuring safety in reinforcement learning without violating constraints.
method Modeling safety as a linear cost function, developing SLUCB-QVI and RSLUCB-QVI algorithms for MDPs with linear function approximation.
result Achieved a nearly optimal regret bound for safe reinforcement learning, matching state-of-the-art unsafe algorithms.
Enhances safety of 3D object detection neural networks.
problem Ensuring robustness and safety of 3D object detection systems.
method Symbolic error propagation, specialized loss function, safety-aware non-max-inclusion algorithm.
result Improved safety and robustness of 3D object detection neural networks.
Each year, around 6 million car accidents occur in the U.S. on average. Road safety features (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety features is an important component of safety management systems for…
A novel approach for safe offline RL using latent safety constraints.
problem Balancing safety constraints and reward maximization in offline RL.
method Conditional Variational Autoencoders for latent safety modeling, Constrained Reward-Return Maximization.
result Our approach maintains safety compliance while optimizing rewards, outperforming existing methods.
New method improves safety analytics by addressing imbalanced data issues.
problem Imbalanced safety datasets lead to inaccurate predictions and management problems.
method Extended accident triangle theory and three oversampling methods.
result Robust improvements in machine learning algorithms for safety analytics.
Paper proposes a framework for reliable off-policy evaluation in reinforcement learning.
problem Quantifying uncertainty in off-policy estimates for safe deployment of target policies.
method Distributionally robust optimization for creating confidence bounds.
result Non-asymptotic and asymptotic guarantees for robust cumulative reward estimates.
Extends driving model to control agent behavior in simulations.
problem Simulate realistic driving behavior for autonomous systems.
method Introduces Control-ITRA method to influence agent behavior through waypoint assignment and target speed modulation.
result Demonstrates controllable, infraction-free trajectories while preserving realism.
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.