Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we revi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “machine learning safety”
On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscs.CY
Machine learning algorithms increasingly influence our decisions and interact with us in all parts of our daily lives. Therefore, just as we consider the safety of power plants, highways, and a variety of other engineered socio-technical systems, we must also take into account the safety of systems involving machine le…
Researchers introduce a method to assess the safety of interpretable machine learning models.
problem Ensuring safety in machine learning models that are easy to understand.
method Introduce maximum deviation as an optimization problem to find the largest deviation from a safe reference model.
result Interpretability helps in assessing the safety of machine learning models.
Machine learning algorithms are increasingly influencing our decisions and interacting with us in all parts of our daily lives. Therefore, just like for power plants, highways, and myriad other engineered sociotechnical systems, we must consider the safety of systems involving machine learning. In this paper, we first …
Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functionseess.SY
Unified control theory and machine learning for safety in uncertain systems.
problem Safety guarantees for systems with measurement model uncertainty.
method Measurement-Robust Control Barrier Functions (MR-CBFs) for control synthesis.
result MR-CBFs ensure safety in perception systems with measurement model uncertainty.
SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measurecs.LG
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
New method improves safety analytics by addressing imbalanced data issues.
problem Imbalanced safety datasets lead to inaccurate predictions and management problems.
method Extended accident triangle theory and three oversampling methods.
result Robust improvements in machine learning algorithms for safety analytics.
Synthesizes machine learning applications in reliability and safety.
problem Navigating the fragmented literature on ML for reliability and safety.
method Overview of ML categories, review of applications, discussion of Deep Learning.
result Machine learning can provide novel insights and improve accident prevention.
System predicts ice formation to improve road safety.
problem Improving road safety through accurate ice formation prediction.
method Combining physical models and machine learning for data correction.
result Machine learning effectively corrects incorrect data in forecasts.
Quantum model improves safety in machine learning.
problem Improving safety and robustness in machine learning models.
method Variational quantum classifier with amplitude encoding and SAFE-AI metrics.
result Quantum model provides competitive performance and improved robustness.
Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems using machine-learning and deep-learning models, such as automated-driving vehicles. Quality assurance frameworks are required for such machine learning systems, but there are no widely accepted and established quality-assurance …
An Overview and Prospective Outlook on Robust Training and Certification of Machine Learning Modelscs.LG
Paper reviews robustness in machine learning models and discusses training and certification methods.
problem Ensuring reliability of machine learning models in safety-critical systems.
method Reviews formalisms and discusses training and certification techniques.
result Identifies future research directions in robust machine learning.
Study investigates how machine learning models degrade over time, leading to patient safety issues.
problem Overtime degradation of machine learning models in clinical settings.
method Used MIMIC-IV dataset to train models replicating commercial approaches, observing and analyzing degradation over a decade.
result An RNN model built on Epic features degrades from 0.729 AUC to 0.525 AUC over a decade, highlighting technical and clinical drift as root causes.
Develops probabilistic safety regions for scalable classifiers.
problem Minimizing misclassification errors in supervised classification.
method Introduces probabilistic safety regions and scalable classifiers.
result Probabilistic certifications for classifier performance.
Teaches uncertainty in ML through practical examples.
problem Lack of uncertainty teaching in ML curricula.
method Developed a curriculum and use cases.
result Motivates adoption of uncertainty concepts in AI courses.
In this paper we discuss how systems with Artificial Intelligence (AI) can undergo safety assessment. This is relevant, if AI is used in safety related applications. Taking a deeper look into AI models, we show, that many models of artificial intelligence, in particular machine learning, are statistical models. Safety …
A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysiscs.LG
Survey on biases in image analysis for industrial safety.
problem Bias in machine learning algorithms affects industrial safety-critical applications.
method Survey and analysis of recent advances in bias detection and mitigation.
result Need for new methods to detect and mitigate biases in image analysis for safety-critical applications.
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learni…
This paper uses MIL and MHCNN-RNN to predict precursors to aviation safety events.
problem Identifying events that precede aviation safety incidents.
method Multiple-instance learning (MIL) framework combined with a Multi-Head Convolutional Neural Network-Recurrent Neural Network (MHCNN-RNN) architecture.
result Multiple binary classifiers outperform in predicting high speed and high path angle events during the approach phase.
Artificial Neural Networks (ANNs) have demonstrated remarkable utility in various challenging machine learning applications. While formally verified properties of their behaviors are highly desired, they have proven notoriously difficult to derive and enforce. Existing approaches typically formulate this problem as a p…
This paper significantly improves on, and finishes to validate, an approach proposed in previous research in which safety outcomes were predicted from attributes with machine learning. Like in the original study, we use Natural Language Processing (NLP) to extract fundamental attributes from raw incident reports and ma…
Affective states have a critical role in driving performance and safety. They can degrade driver situation awareness and negatively impact cognitive processes, severely diminishing road safety. Therefore, detecting and assessing drivers' affective states is crucial in order to help improve the driving experience, and i…
The safety and resilience of fully autonomous vehicles (AVs) are of significant concern, as exemplified by several headline-making accidents. While AV development today involves verification, validation, and testing, end-to-end assessment of AV systems under accidental faults in realistic driving scenarios has been lar…
This paper compares machine learning methods for recognizing lane change intentions from vehicle trajectories.
problem Accurately detecting and predicting lane change processes in autonomous vehicles.
method Comparison of different machine learning methods on high-dimensional time series data.
result Ensemble methods reduce Type II and Type III classification errors, while LightGBM outperforms XGBoost in training efficiency.
This work improves motion planning for quadcopters by learning and reasoning about controller performance.
problem Improving motion planning for quadcopters with safety margins and execution reliability.
method Introspective learning and reasoning to correct execution bias and improve collision checking.
result Substantial reduction in safety margins for motion actions, leading to safer execution.
Paper proposes a dataset quality process for ML systems.
problem Inadequate standards for ML datasets in safety-critical systems.
method Proposes a dataset specification and verification process.
result Demonstrates the process on a railway signal recognition system.
Study improves self-driving safety in dynamic environments.
problem Safe self-driving in non-stationary urban settings.
method Neurosymbolic Meta-Reinforcement Lookahead Learning (NUMERLA).
result Self-driving agents can adapt safely in real-time.
Framework for safely updating machine learning models.
problem Continuous updates to machine learning models can lead to unintended consequences.
method Formalizes the problem as computing the largest locally invariant domain (LID), uses tractable primal-dual formulation.
result Matches or exceeds heuristic baselines for avoiding forgetting while providing formal safety guarantees.
A framework integrates machine learning with robust control for safer, more reliable systems.
problem Combining machine learning with robust control for systems with stringent safety and reliability requirements.
method Integrates Gaussian Process Regression and state-of-the-art robust controller synthesis within a framework that provides rigorous guarantees.
result Demonstrated improved performance with more data while maintaining rigorous guarantees.
Combining feature importance estimates improves reliability of machine learning predictions.
problem Lack of consensus on feature importance quantification makes explanations unreliable.
method Proposes a feature importance fusion framework combining multiple quantifiers.
result Feature importance ensembles reduce prediction error by 15%.
Survey of technologies for trustworthy machine learning systems.
problem Building machine learning systems that are fair, explainable, auditable, and secure.
method Survey of technologies across data and model stages of machine learning.
result Four categories of system properties (fairness, explainability, auditability, safety & security) are essential for trustworthy systems.
This paper presents a distributionally robust Q-Learning algorithm (DrQ) which leverages Wasserstein ambiguity sets to provide idealistic probabilistic out-of-sample safety guarantees during online learning. First, we follow past work by separating the constraint functions from the principal objective to create a hiera…
Machine and reinforcement learning (RL) are increasingly being applied to plan and control the behavior of autonomous systems interacting with the physical world. Examples include self-driving vehicles, distributed sensor networks, and agile robots. However, when machine learning is to be applied in these new settings,…
Paper shows how to quantify uncertainty in medical ML models.
problem Uncertainty in opaque ML models can lead to safety risks in medical applications.
method Introduces Uncertainty Wrapper to quantify uncertainty transparently.
result Demonstrates practical utility of Uncertainty Wrapper in flow cytometry.
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpr…
Machine learning (ML) techniques are increasingly applied to decision-making and control problems in Cyber-Physical Systems among which many are safety-critical, e.g., chemical plants, robotics, autonomous vehicles. Despite the significant benefits brought by ML techniques, they also raise additional safety issues beca…
RLVR maintains safety while improving reasoning capabilities in LLMs.
problem Safety-capability tradeoff in fine-tuning LLMs.
method Reinforcement Learning with Verifiable Rewards (RLVR) and theoretical analysis.
result RLVR can enhance reasoning while maintaining safety guardrails.
Safe Bayesian optimization tackles safety constraints in control engineering.
problem Handling safety constraints in parameter tuning of control systems.
method Lipschitz-only Safe Bayesian Optimization (LoSBO) and LoS-GP-UCB.
result SafeBO algorithms can violate safety constraints due to unreliable uncertainty bounds.
The use of machine learning (ML) is on the rise in many sectors of software development, and automotive software development is no different. In particular, Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) are two areas where ML plays a significant role. In automotive development, safety is…
Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are saf…
Machine learning can provide efficient solutions to the complex problems encountered in autonomous driving, but ensuring their safety remains a challenge. A number of authors have attempted to address this issue, but there are few publicly-available tools to adequately explore the trade-offs between functionality, scal…
Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning appro…
Safe imitation learning with a safety layer for flexible training.
problem Flexible yet safe imitation learning for complex tasks.
method Theory and modular method with a safety layer for continuous policy, adversarial training, and worst-case safety guarantees.
result Robustness advantage of safety layer during training compared to test time.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
The paper proposes a method to assess when automated predictions are reliable.
problem Ensuring reliability and safety of automated decision-making in machine learning.
method Clustering to measure distances between outputs and class centroids, defining a safety threshold based on these distances.
result The proposed metric can efficiently determine when automated predictions are acceptable and when they should be deferred.
Despite the tremendous advances that have been made in the last decade on developing useful machine-learning applications, their wider adoption has been hindered by the lack of strong assurance guarantees that can be made about their behavior. In this paper, we consider how formal verification techniques developed for …
Failures are challenging for learning to control physical systems since they risk damage, time-consuming resets, and often provide little gradient information. Adding safety constraints to exploration typically requires a lot of prior knowledge and domain expertise. We present a safety measure which implicitly captures…