Every year, thousands of people receive consumer product related injuries. Research indicates that online customer reviews can be processed to autonomously identify product safety issues. Early identification of safety issues can lead to earlier recalls, and thus fewer injuries and deaths. A dataset of product reviews …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “safety classifiers”
Classifier-based AI safety gates fail in self-improvement, even with advanced verification methods.
problem Maintaining reliable oversight of AI systems as they improve over iterations.
method Comprehensive empirical testing on neural controllers and MuJoCo benchmarks, using various classifiers and verification methods.
result Classifier-based safety gates fail in maintaining reliable oversight, even with advanced verification methods.
Develops probabilistic safety regions for scalable classifiers.
problem Minimizing misclassification errors in supervised classification.
method Introduces probabilistic safety regions and scalable classifiers.
result Probabilistic certifications for classifier performance.
Online monitoring system for safety classifiers with shift detection and conformal adaptation
problem Detecting and adapting to distributional shifts in deployed safety classifiers
method Calibrated sequential statistics for online monitoring, conformal abstention for adaptation
result 86.6% valid detection with mean latency of 39.5 steps
This paper uses MIL and MHCNN-RNN to predict precursors to aviation safety events.
problem Identifying events that precede aviation safety incidents.
method Multiple-instance learning (MIL) framework combined with a Multi-Head Convolutional Neural Network-Recurrent Neural Network (MHCNN-RNN) architecture.
result Multiple binary classifiers outperform in predicting high speed and high path angle events during the approach phase.
Mobile app development in recent years has resulted in new products and features to improve human life. Mobile telematics is one such development that encompasses multidisciplinary fields for transportation safety. The application of mobile telematics has been explored in many areas, such as insurance and road safety. …
Each year, around 6 million car accidents occur in the U.S. on average. Road safety features (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety features is an important component of safety management systems for…
SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measurecs.LG
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
Quantum model improves safety in machine learning.
problem Improving safety and robustness in machine learning models.
method Variational quantum classifier with amplitude encoding and SAFE-AI metrics.
result Quantum model provides competitive performance and improved robustness.
Efficient classifier with uncertainty bounds for safety-critical applications.
problem Lack of uncertainty bounds in high-accuracy classifiers for safety-critical tasks.
method Nadaraya-Watson estimator with frequentist bounds.
result Competitive accuracy and uncertainty bounds at reduced computational cost.
We explore adversarial robustness in the setting in which it is acceptable for a classifier to abstain---that is, output no class---on adversarial examples. Adversarial examples are small perturbations of normal inputs to a classifier that cause the classifier to give incorrect output; they present security and safety …
Improves safety region certification for smoothed classifiers without changing smoothing scheme.
problem Certified safety regions for smoothed classifiers are often small compared to optimal.
method Generalizes certified radius calculation as nested optimization problem, uses 0th-1st order information, and designs efficient estimators.
result Certified safety regions are significantly larger than current methods, achieving significant improvements on various metrics.
Theoretical limits on verifying self-improving systems without risking unbounded utility.
problem Formalizing and proving the limits of safety verification for self-improving systems.
method Developed dual conditions and used Holder's inequality, NP counting method, and Lipschitz bounds to establish impossibility and ceiling results.
result A classifier-based safety gate cannot simultaneously permit unbounded beneficial self-modification and bounded cumulative risk.
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
Semantic Embeddings are a popular way to represent knowledge in the field of zero-shot learning. We observe their interpretability and discuss their potential utility in a safety-critical context. Concretely, we propose to use them to add introspection and error detection capabilities to neural network classifiers. Fir…
Online monitor detects classifier drift and adapts predictions.
problem Silent degradation of classifier accuracy under distributional shift.
method Sliding-window KS statistic with calibrated alarm thresholds.
result 86.6% valid detection across various shift conditions.
PEOC uses policy entropy to detect untrained states in RL.
problem Detecting untrained states in reinforcement learning for safety.
method Policy entropy based one-class classifier.
result PEOC is highly competitive and reliable.
Time Series Classification (TSC) problems are encountered in many real life data mining tasks ranging from medicine and security to human activity recognition and food safety. With the recent success of deep neural networks in various domains such as computer vision and natural language processing, researchers started …
Recent work has shown that state-of-the-art classifiers are quite brittle, in the sense that a small adversarial change of an originally with high confidence correctly classified input leads to a wrong classification again with high confidence. This raises concerns that such classifiers are vulnerable to attacks and ca…
Machine learning (ML) classification is increasingly used in safety-critical systems. Protecting ML classifiers from adversarial examples is crucial. We propose that the main threat is that of an attacker perturbing a confidently classified input to produce a confident misclassification. To protect against this we devi…
In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researc…
OOD detection methods often misidentify OOD points, leading to ineffective safety improvements.
problem Improving model safety through OOD detection methods often leads to incorrect identification of out-of-distribution points.
method Re-examine popular OOD detection procedures based on predictive uncertainty or features of supervised models trained on in-distribution data.
result Popular OOD detection methods incorrectly conflate high uncertainty and far feature-space distance with being out-of-distribution.
Motivated by safety-critical classification problems, we investigate adversarial attacks against cost-sensitive classifiers. We use current state-of-the-art adversarially-resistant neural network classifiers [1] as the underlying models. Cost-sensitive predictions are then achieved via a final processing step in the fe…
Suitability filter detects model performance degradation in real-world deployment.
problem Ensuring model reliability in safety-critical domains without access to ground truth labels.
method Uses suitability signals to evaluate classifier performance on unlabeled user data.
result The suitability filter reliably detects performance deviations due to covariate shift.
Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems using machine-learning and deep-learning models, such as automated-driving vehicles. Quality assurance frameworks are required for such machine learning systems, but there are no widely accepted and established quality-assurance …
Generative model improves safety in self-driving simulators and human motion generation.
problem Improving generative models for constrained domains like safety-critical applications.
method Developed Gen-neG, a denoising diffusion model that uses oracle-assisted guidance.
result Empirically validated Gen-neG for collision avoidance and safety-guarded human motion generation.
Paper proposes a method to predict deep neural network confidences with guarantees.
problem Quantifying uncertainty in deep neural networks for safety-critical applications.
method Uses Clopper-Pearson confidence intervals and histogram binning for calibrated prediction.
result Demonstrates the effectiveness of predicted confidences in improving DNN performance and safety.
Proposes a method to ensure accurate estimation of rare events in AI systems.
problem Lack of efficiency guarantees in black-box systems for rare-event simulation.
method Integrates deep learning with importance sampling to create a statistically guaranteed estimator.
result Demonstrates effective estimation of rare-event probabilities in AI systems.
It has been shown that neural network classifiers are not robust. This raises concerns about their usage in safety-critical systems. We propose in this paper a regularization scheme for ReLU networks which provably improves the robustness of the classifier by maximizing the linear regions of the classifier as well as t…
RLVR maintains safety while improving reasoning capabilities in LLMs.
problem Safety-capability tradeoff in fine-tuning LLMs.
method Reinforcement Learning with Verifiable Rewards (RLVR) and theoretical analysis.
result RLVR can enhance reasoning while maintaining safety guardrails.
Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due to lack of training …
In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Unlike recent robustness research, this benchmark evaluates performan…
Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we revi…
Probabilistic classifiers output a probability distribution on target classes rather than just a class prediction. Besides providing a clear separation of prediction and decision making, the main advantage of probabilistic models is their ability to represent uncertainty about predictions. In safety-critical applicatio…
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
BagCert efficiently certifies robustness against adversarial patches on image classifiers.
problem Adversarial patches pose a threat to autonomous systems' perception component.
method BagCert combines model architecture and certification procedure for efficient inference.
result BagCert certifies 10,000 examples in 43 seconds on a single GPU, achieving 86% clean and 60% certified accuracy against 5x5 patches.
RAGuard improves safety in LLMs for offshore wind maintenance.
problem Conventional LLMs fail with specialised or unexpected scenarios in offshore wind maintenance.
method Integrates safety-critical documents alongside technical manuals in RAG framework.
result RAGuard increases safety recall from almost 0% to over 50% while maintaining technical recall above 60%.
Study improves image classifier robustness to random p-norm corruptions.
problem Improving robustness of image classifiers to real-world imperceptible corruptions.
method Training and testing with random p-norm corruptions, evaluating robustness against different p-norms.
result Training with a combination of p-norm corruptions significantly improves robustness.
Survey of algorithms for testing AI-driven CPS safety.
problem Testing AI-driven CPS for safety in complex environments.
method Survey of applied algorithms for safety validation.
result Survey of existing tools and techniques for safety validation.
Linear classifiers can resist adversarial attacks on Gaussian data.
problem Adversarial attacks on high-dimensional data.
method Adversarial training of linear classifiers on Gaussian data.
result Linear classifiers can resist adversarial attacks on Gaussian data.
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.
Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning appro…
This paper formalizes AI safety using hypothesis testing in GenAI.
problem Ensuring safety of generative AI tools that create realistic content.
method Formalization of computational safety through hypothesis testing and signal processing.
result Demonstrates how AI safety can be assessed quantitatively using mathematical frameworks.
A new algorithm for identifying the best arm in linear feedback with safety constraints.
problem Identifying the best arm in linear feedback with safety constraints.
method A gap-based algorithm that ensures safety while minimizing sample complexity.
result The algorithm achieves meaningful sample complexity while ensuring safety.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
Before deploying autonomous agents in the real world, we need to be confident they will perform safely in novel situations. Ideally, we would expose agents to a very wide range of situations during training, allowing them to learn about every possible danger, but this is often impractical. This paper investigates safet…
Paper defines and quantifies safety risks in deep neural networks.
problem Safety concerns in deep neural networks applied to critical sectors.
method Defines safety property, computes maximum safe radius, identifies new risk class, develops algorithm.
result Method achieves competitive performance in safety quantification.
Study best arm identification with safety constraints in bandit problems.
problem Real-world decision-making with safety constraints.
method Analyzed linear and monotonic reward and safety constraints, proposed algorithms.
result Guaranteed safe learning in both linear and general reward/safety constraint settings.