This paper explores formal verification for autonomous systems, identifying limitations and proposing improvements.
problem Ensuring safety of autonomous systems like self-driving cars and drones.
method Formal verification techniques based on formal methods, analyzing three assumptions and their limitations.
result Preliminary work to improve the strength of evidence provided by formal verification.
CitySim dataset captures vehicle trajectories for safety research.
problem Lack of fine-grain vehicle trajectories for safety-oriented research.
method Five-step procedure: video stabilization, object filtering, stitching, detection, and error filtering.
result CitySim dataset improves safety evaluations and facilitates digital-twin research.
Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning appro…
Deep RL drone trained to compete against classical path planning in drone racing.
problem Optimizing long-term drone racing strategies using reinforcement learning.
method Used PPO algorithm on a simulated quadrotor in a racing environment created with AirSim.
result Deep RL agent outperformed classical path planning in drone racing competitions.
Abstract Neural Networks (ANNs) improve DNN verification efficiency.
problem Efficiently verify safety-critical DNNs without slowing exponentially.
method Introduces ANNs that use abstract domains and activation functions to overapproximate DNNs.
result ANNs can soundly overapproximate DNNs with fewer nodes, improving verification efficiency.
A drone catches another agile drone using competitive reinforcement learning.
problem Intercepting an agile drone with another agile drone.
method Formulated as a Competitive Reinforcement Learning problem, trained with PPO, using a high-fidelity simulation environment.
result Trained policies outperform common heuristic baselines in catch rate, time to catch, and crash rate.
Enabling cellular connectivity for drones introduces a wide set of challenges and opportunities. Communication of cellular-connected drones is influenced by 3-dimensional mobility and line-of-sight channel characteristics which results in higher number of handovers with increasing altitude. Our cell planning simulation…
Drone optimizes collaborative learning for neural networks.
problem Training a shared neural network with heterogeneous data from multiple devices.
method Drone collects and aggregates model parameters from devices, optimizing trajectory for best accuracy.
result Significant improvement in final accuracy and speedup in training time.
Modern vision-based reinforcement learning techniques often use convolutional neural networks (CNN) as universal function approximators to choose which action to take for a given visual input. Until recently, CNNs have been treated like black-box functions, but this mindset is especially dangerous when used for control…
In this paper, we consider same-day delivery with vehicles and drones. Customers make delivery requests over the course of the day, and the dispatcher dynamically dispatches vehicles and drones to deliver the goods to customers before their delivery deadline. Vehicles can deliver multiple packages in one route but trav…
A drone-based MOT algorithm tracks vehicles using neural network detections and TPMBM filter.
problem Tracking multiple vehicles from drone-mounted cameras.
method Neural network for object detection, TPMBM filter for trajectory estimation, von-Mises Fisher distribution for DOA.
result TPMBM filter optimally estimates vehicle trajectories.
We present NAVREN-RL, an approach to NAVigate an unmanned aerial vehicle in an indoor Real ENvironment via end-to-end reinforcement learning RL. A suitable reward function is designed keeping in mind the cost and weight constraints for micro drone with minimum number of sensing modalities. Collection of small number of…
Deep reinforcement learning controls drones without model knowledge.
problem Real-time robot control without engineered models.
method Learnt probabilistic model of drone dynamics, model-based reinforcement learning.
result Controller and value function optimized through generated latent trajectories.
A new method for drone-based geo-localization using style and spatial alignment.
problem Geo-localization of drone-view images with satellite-view images using pre-annotated GPS tags.
method Orientation-based method to align patterns, new branch to extract aligned partial features, style alignment strategy.
result The proposed method outperforms state-of-the-art alternatives in geo-localization accuracy.
Smart and agile drones are fast becoming ubiquitous at the edge of the cloud. The usage of these drones are constrained by their limited power and compute capability. In this paper, we present a Transfer Learning (TL) based approach to reduce on-board computation required to train a deep neural network for autonomous n…
Air traffic control is a real-time safety-critical decision making process in highly dynamic and stochastic environments. In today's aviation practice, a human air traffic controller monitors and directs many aircraft flying through its designated airspace sector. With the fast growing air traffic complexity in traditi…
DePAint solves MARL for agents with local constraints, privacy, and no central controller.
problem Training multi-agent systems to optimize rewards while adhering to safety constraints in a decentralized setting.
method Formulated as a decentralized constrained multi-agent Markov Decision Problem, proposed DePAint method using momentum-based decentralized policy gradient.
result First privacy-preserving fully decentralized MARL algorithm considering both peak and average constraints.
Meta-learning improves drone trajectory design for dynamic wireless networks.
problem Designing optimal trajectories for energy-constrained drones in dynamic network environments.
method Proposes a meta-learning algorithm to adaptively tune a reinforcement learning solution for trajectory design.
result Meta-tuned RL yields faster convergence and improved communication performance compared to baseline algorithms.
Reinforcement learning (RL) has advanced greatly in the past few years with the employment of effective deep neural networks (DNNs) on the policy networks. With the great effectiveness came serious vulnerability issues with DNNs that small adversarial perturbations on the input can change the output of the network. Sev…
One of the greatest challenges in the design of a real-time perception system for autonomous driving vehicles and drones is the conflicting requirement of safety (high prediction accuracy) and efficiency. Traditional approaches use a single frame rate for the entire system. Motivated by the observation that the lack of…
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
Scenario-based testing for the safety validation of highly automated vehicles is a promising approach that is being examined in research and industry. This approach heavily relies on data from real-world scenarios to derive the necessary scenario information for testing. Measurement data should be collected at a reason…
The paper detects amateur drones using acoustic signals, overcoming interference.
problem Detection of unmonitored UAVs (amateur drones) for security and safety.
method Machine learning approach using Independent Component Analysis (ICA) for acoustic signal detection.
result The RMS values of PSD with KNN outperform other methods in detecting AmDr.
UAVs learn to collect data from IoT sensors efficiently.
problem Efficient UAV path planning for wireless data collection.
method Deep reinforcement learning (DDQN) with experience replay and convolutional layers.
result UAV control policy generalizes over changing scenario parameters.
Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of …
RLVR maintains safety while improving reasoning capabilities in LLMs.
problem Safety-capability tradeoff in fine-tuning LLMs.
method Reinforcement Learning with Verifiable Rewards (RLVR) and theoretical analysis.
result RLVR can enhance reasoning while maintaining safety guardrails.
We introduce a spatio-temporal convolutional neural network model for trajectory forecasting from visual sources. Applied in an auto-regressive way it provides an explicit probability distribution over continuations of a given initial trajectory segment. We discuss it in relation to (more complicated) existing work and…
The paper discusses safety assessment for AI systems, focusing on machine learning models.
problem Safety assessment of AI systems, especially machine learning models, in safety-related applications.
method Analyzed AI models as statistical models and proposed a new budget allocation for AI safety.
result Safety assessment of AI systems requires a new approach focusing on the model used, not just the system.
EMPIR combines low and full precision DNNs to enhance robustness against adversarial attacks.
problem Vulnerability of DNNs to adversarial attacks that misclassify inputs with small perturbations.
method Ensemble of quantized DNN models with different numerical precisions.
result EMPIR ensembles increase adversarial robustness by 42.6% on average across different tasks.
Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we revi…
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
RAGuard improves safety in LLMs for offshore wind maintenance.
problem Conventional LLMs fail with specialised or unexpected scenarios in offshore wind maintenance.
method Integrates safety-critical documents alongside technical manuals in RAG framework.
result RAGuard increases safety recall from almost 0% to over 50% while maintaining technical recall above 60%.
Survey of algorithms for testing AI-driven CPS safety.
problem Testing AI-driven CPS for safety in complex environments.
method Survey of applied algorithms for safety validation.
result Survey of existing tools and techniques for safety validation.
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.
This paper formalizes AI safety using hypothesis testing in GenAI.
problem Ensuring safety of generative AI tools that create realistic content.
method Formalization of computational safety through hypothesis testing and signal processing.
result Demonstrates how AI safety can be assessed quantitatively using mathematical frameworks.
A new algorithm for identifying the best arm in linear feedback with safety constraints.
problem Identifying the best arm in linear feedback with safety constraints.
method A gap-based algorithm that ensures safety while minimizing sample complexity.
result The algorithm achieves meaningful sample complexity while ensuring safety.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
Paper defines and quantifies safety risks in deep neural networks.
problem Safety concerns in deep neural networks applied to critical sectors.
method Defines safety property, computes maximum safe radius, identifies new risk class, develops algorithm.
result Method achieves competitive performance in safety quantification.
Study best arm identification with safety constraints in bandit problems.
problem Real-world decision-making with safety constraints.
method Analyzed linear and monotonic reward and safety constraints, proposed algorithms.
result Guaranteed safe learning in both linear and general reward/safety constraint settings.
Failures are challenging for learning to control physical systems since they risk damage, time-consuming resets, and often provide little gradient information. Adding safety constraints to exploration typically requires a lot of prior knowledge and domain expertise. We present a safety measure which implicitly captures…
Safe reinforcement learning tackles safety constraints with linear approximations.
problem Ensuring safety in reinforcement learning without violating constraints.
method Modeling safety as a linear cost function, developing SLUCB-QVI and RSLUCB-QVI algorithms for MDPs with linear function approximation.
result Achieved a nearly optimal regret bound for safe reinforcement learning, matching state-of-the-art unsafe algorithms.
Enhances safety of 3D object detection neural networks.
problem Ensuring robustness and safety of 3D object detection systems.
method Symbolic error propagation, specialized loss function, safety-aware non-max-inclusion algorithm.
result Improved safety and robustness of 3D object detection neural networks.
Each year, around 6 million car accidents occur in the U.S. on average. Road safety features (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety features is an important component of safety management systems for…
CARL safely adapts RL agents for safety-critical tasks.
problem Safety hazards in RL for safety-critical tasks.
method CARL combines model-based RL and cautious adaptation.
result CARL achieves higher rewards with fewer failures in safety-critical tasks.
A novel approach for safe offline RL using latent safety constraints.
problem Balancing safety constraints and reward maximization in offline RL.
method Conditional Variational Autoencoders for latent safety modeling, Constrained Reward-Return Maximization.
result Our approach maintains safety compliance while optimizing rewards, outperforming existing methods.
New method improves safety analytics by addressing imbalanced data issues.
problem Imbalanced safety datasets lead to inaccurate predictions and management problems.
method Extended accident triangle theory and three oversampling methods.
result Robust improvements in machine learning algorithms for safety analytics.
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.