arXiv research

Paper defines and quantifies safety risks in deep neural networks.

Enhances safety of 3D object detection neural networks.

Neural networks are increasingly deployed in real-world safety-critical domains such as autonomous driving, aircraft collision avoidance, and malware detection. However, these networks have been shown to often mispredict on inputs with minor adversarial or even accidental perturbations. Consequences of such errors can …

Study probabilistic safety of BNNs under adversarial attacks.

Each year, around 6 million car accidents occur in the U.S. on average. Road safety features (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety features is an important component of safety management systems for…

Extends neural net safety guarantees by proving structural properties.

New metric space for ReLU codes connects to network safety and robustness.

Paper proposes Vertex Networks for reinforcement learning of control systems with safety guarantees.

SafeML monitors ML systems for safety and security risks.

The prediction of workers' safety behaviour can help identify vulnerable workers who intend to undertake unsafe behaviours and be useful in the design of management practices to minimise the occurrence of accidents. The latest literature has evidenced that there is within-population diversity that leads people's intend…

This paper formalizes AI safety using hypothesis testing in GenAI.

Artificial Neural Networks (ANNs) have demonstrated remarkable utility in various challenging machine learning applications. While formally verified properties of their behaviors are highly desired, they have proven notoriously difficult to derive and enforce. Existing approaches typically formulate this problem as a p…

We propose Trusted Neural Network (TNN) models, which are deep neural network models that satisfy safety constraints critical to the application domain. We investigate different mechanisms for incorporating rule-based knowledge in the form of first-order logic constraints into a TNN model, where rules that encode safet…

The paper proves neural networks are almost always surjective, impacting model safety.

Deep neural networks have achieved impressive experimental results in image classification, but can surprisingly be unstable with respect to adversarial perturbations, that is, minimal changes to the input image that cause the network to misclassify it. With potential applications including perception modules and end-t…

System predicts ice formation to improve road safety.

An active area of research is to increase the safety of self-driving vehicles. Although safety cannot be guarenteed completely, the capability of a vehicle to predict the future trajectories of its surrounding vehicles could help ensure this notion of safety to a greater deal. We cast the trajectory forecast problem in…

Real time large scale streaming data pose major challenges to forecasting, in particular defying the presence of human experts to perform the corresponding analysis. We present here a class of models and methods used to develop an automated, scalable and versatile system for large scale forecasting oriented towards saf…

This paper compares uncertainty estimation methods for deep learning in autonomous vehicles.

The paper shows why AI safety doesn't generalize across tasks.

PEREGRiNN verifies safety of ReLU NNs by penalizing relaxation in a greedy manner.

Study evaluates conformal prediction methods for safety in vision models under shifts and long-tailed data.

A decentralized algorithm minimizes cumulative regret in stochastic linear bandits with safety constraints.

Classifier-based AI safety gates fail in self-improvement, even with advanced verification methods.

Abstract Neural Networks (ANNs) improve DNN verification efficiency.

Automated testing framework finds weaknesses in deep control policies.

Paper proposes a method to predict deep neural network confidences with guarantees.

Neural Networks are being integrated into safety critical systems, e.g., perception systems for autonomous vehicles, which require trained networks to perform safely in novel scenarios. It is challenging to verify neural networks because their decisions are not explainable, they cannot be exhaustively tested, and finit…

Video applications and analytics are routinely projected as a stressing and significant service of the Nationwide Public Safety Broadband Network. As part of a NIST PSCR funded effort, the New Jersey Office of Homeland Security and Preparedness and MIT Lincoln Laboratory have been developing a computer vision dataset o…

Adversarial attacks pose a threat to deep neural networks, especially in safety-critical applications.

Although aviation accidents are rare, safety incidents occur more frequently and require a careful analysis to detect and mitigate risks in a timely manner. Analyzing safety incidents using operational data and producing event-based explanations is invaluable to airline companies as well as to governing organizations s…

BODE enhances deep neural network predictions and uncertainty quantification in safety modeling.

RLVR maintains safety while improving reasoning capabilities in LLMs.

Advances in the field of Machine Learning and Deep Neural Networks (DNNs) has enabled rapid development of sophisticated and autonomous systems. However, the inherent complexity to rigorously assure the safe operation of such systems hinders their real-world adoption in safety-critical domains such as aerospace and med…

Deep-PrAE improves rare-event simulation for black-box systems.

Soft labeling impacts OOD detection in neural networks.

Generates multimodal safety-critical scenarios for robustness evaluation of decision-making algorithms.

The paper discusses safety assessment for AI systems, focusing on machine learning models.

This paper uses MIL and MHCNN-RNN to predict precursors to aviation safety events.

Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we revi…

The paper proposes a method to assess when automated predictions are reliable.

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial examples are imperceptible to …

Fine-tuning LLMs improves capability but harms safety, study finds.

RAGuard improves safety in LLMs for offshore wind maintenance.

Survey of algorithms for testing AI-driven CPS safety.

Can engineering neural networks be approached in a disciplined way similar to how engineers build software for civil aircraft? We present nn-dependability-kit, an open-source toolbox to support safety engineering of neural networks for autonomous driving systems. The rationale behind nn-dependability-kit is to consider…

Safe imitation learning with a safety layer for flexible training.

Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning appro…