DL models for MTS regression are vulnerable to adversarial attacks, posing risks in safety-critical applications.
problem Vulnerability of DL models to adversarial examples in MTS regression.
method Adversarial attack generation techniques from image classification were adapted for MTS.
result All state-of-the-art DL regression models (CNN, LSTM, GRU) are vulnerable to adversarial attacks.
Abstract Neural Networks (ANNs) improve DNN verification efficiency.
problem Efficiently verify safety-critical DNNs without slowing exponentially.
method Introduces ANNs that use abstract domains and activation functions to overapproximate DNNs.
result ANNs can soundly overapproximate DNNs with fewer nodes, improving verification efficiency.
Paper proposes a dataset quality process for ML systems.
problem Inadequate standards for ML datasets in safety-critical systems.
method Proposes a dataset specification and verification process.
result Demonstrates the process on a railway signal recognition system.
New algorithm constructs PAC prediction sets for meta-learning.
problem Uncertainty quantification in safety-critical systems.
method Proposes a novel algorithm to construct PAC prediction sets.
result Prediction sets satisfy a PAC guarantee with high probability over future tasks.
SGPA calibrates transformer uncertainty for safety-critical tasks.
problem Uncertainty estimation in transformer models for safety-critical domains.
method Bayesian inference in transformer's output space using sparse Gaussian processes.
result SGPA-based Transformers improve in-distribution calibration and out-of-distribution robustness.
Semantic Embeddings are a popular way to represent knowledge in the field of zero-shot learning. We observe their interpretability and discuss their potential utility in a safety-critical context. Concretely, we propose to use them to add introspection and error detection capabilities to neural network classifiers. Fir…
Develops a method to simulate rare dangerous events in autonomous systems.
problem Rare dangerous events in safety-critical systems are hard to test in real-world settings.
method Combines exploration, exploitation, and optimization techniques for rare-event simulation.
result Provides rigorous guarantees for the performance of the method.
The paper tackles MAP inference over non-convex constraints in safety-critical settings.
problem Efficiently computing MAP predictions subject to non-convex constraints is challenging.
method The paper investigates conditions for exact and efficient MAP inference over continuous variables and devises scalable algorithms for both tractable and general cases.
result The proposed methods outperform constraint-agnostic baselines and scale to complex densities.
A framework identifies worst-case decision points in safety-critical scenarios, improving risk assessment by 10 hours.
problem Identifying worst-case outcomes in safety-critical decision-making under uncertainty.
method Explicitly estimating distributions of expected return to identify dead-ends, tuning based on risk tolerance.
result Significantly improves risk assessment, providing indications 10 hours earlier and increasing detection by 20%.
Suitability filter detects model performance degradation in real-world deployment.
problem Ensuring model reliability in safety-critical domains without access to ground truth labels.
method Uses suitability signals to evaluate classifier performance on unlabeled user data.
result The suitability filter reliably detects performance deviations due to covariate shift.
ViTaX provides formal guarantees for targeted explanations in safety-critical systems.
problem Need trustworthy explanations for safety-critical deep neural networks.
method Formal reachability analysis for targeted, semifactual explanations.
result First method to provide formally guaranteed explanations of model resilience.
Framework for safely updating machine learning models.
problem Continuous updates to machine learning models can lead to unintended consequences.
method Formalizes the problem as computing the largest locally invariant domain (LID), uses tractable primal-dual formulation.
result Matches or exceeds heuristic baselines for avoiding forgetting while providing formal safety guarantees.
CARL safely adapts RL agents for safety-critical tasks.
problem Safety hazards in RL for safety-critical tasks.
method CARL combines model-based RL and cautious adaptation.
result CARL achieves higher rewards with fewer failures in safety-critical tasks.
Survey of algorithms for testing AI-driven CPS safety.
problem Testing AI-driven CPS for safety in complex environments.
method Survey of applied algorithms for safety validation.
result Survey of existing tools and techniques for safety validation.
New method calibrates models under covariate shifts.
problem Calibration of models can be lost under covariate shifts.
method Importance sampling based approach.
result Efficacy demonstrated on real-world and synthetic datasets.
New metric predicts neural network reliability under novel conditions.
problem Verifying neural networks' safety in novel scenarios.
method ML Dependability metric, Task Undependability, Harmful Undependability.
result Accurately predicts reliability under novel conditions.
The paper evaluates and improves uncertainty estimates in neural networks for safety-critical applications.
problem Quantifying uncertainty in neural networks for safety-critical systems.
method Proposes a statistical test for evaluating uncertainty realism in neural networks and transfers a classification architecture to image-to-image tasks.
result The variational U-Net architecture significantly improves uncertainty realism in image-to-image tasks compared to a plain model.
Resource scheduling and coordination is an NP-hard optimization requiring an efficient allocation of agents to a set of tasks with upper- and lower bound temporal and resource constraints. Due to the large-scale and dynamic nature of resource coordination in hospitals and factories, human domain experts manually plan a…
TransCal calibrates DA models with lower bias and variance.
problem Calibrating DA models to estimate accurate predictive uncertainty.
method Transferable Calibration (TransCal) in a unified hyperparameter-free optimization framework.
result TransCal achieves more accurate calibration with lower bias and variance.
Generates multimodal safety-critical scenarios for robustness evaluation of decision-making algorithms.
problem Lack of comprehensive evaluation of neural network robustness under real-world scenarios.
method Proposes a flow-based multimodal scenario generator using weighted likelihood maximization and gradient-based sampling.
result Demonstrates improved testing efficiency and multimodal modeling capability compared to traditional methods.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
The paper provides tighter error bounds for GPR under bounded support noise.
problem Rigorous error quantification for safety-critical applications with bounded noise.
method Using concentration inequalities and low complexity assumptions in RKHS, the paper derives probabilistic and deterministic error bounds for GPR.
result The derived error bounds are substantially tighter than existing state-of-the-art bounds and are particularly well-suited for GPR with neural network kernels.
Deep neural networks are complex and opaque. As they enter application in a variety of important and safety critical domains, users seek methods to explain their output predictions. We develop an approach to explaining deep neural networks by constructing causal models on salient concepts contained in a CNN. We develop…
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
Reinforcement Learning (RL) is a widely employed machine learning architecture that has been applied to a variety of control problems. However, applications in safety-critical domains require a systematic and formal approach to specifying requirements as tasks or goals. We propose a model-free RL algorithm that enables…
This work introduces a bias-variance decomposition for proper scores, improving uncertainty estimation in predictive models.
problem Reliable uncertainty estimation for predictions in safety-critical applications, especially under domain drift.
method Developed a general bias-variance decomposition for proper scores, introducing the Bregman Information as the variance term.
result The decomposition provides novel formulations for different predictive tasks, including classification and model ensembles.
Deep neural networks (DNN) have achieved unprecedented success in numerous machine learning tasks in various domains. However, the existence of adversarial examples has raised concerns about applying deep learning to safety-critical applications. As a result, we have witnessed increasing interests in studying attack an…
Modern vision-based reinforcement learning techniques often use convolutional neural networks (CNN) as universal function approximators to choose which action to take for a given visual input. Until recently, CNNs have been treated like black-box functions, but this mindset is especially dangerous when used for control…
Machine learning (ML) classification is increasingly used in safety-critical systems. Protecting ML classifiers from adversarial examples is crucial. We propose that the main threat is that of an attacker perturbing a confidently classified input to produce a confident misclassification. To protect against this we devi…
This paper compares uncertainty estimation methods for deep learning in autonomous vehicles.
problem Ensuring safety in autonomous vehicles through accurate uncertainty quantification in deep learning models.
method A comparative survey of uncertainty quantification methods in deep neural networks.
result Different methods for uncertainty quantification in DNNs have advantages and downsides for specific AV tasks and types of uncertainty.
A rise in popularity of Deep Neural Networks (DNNs), attributed to more powerful GPUs and widely available datasets, has seen them being increasingly used within safety-critical domains. One such domain, self-driving, has benefited from significant performance improvements, with millions of miles having been driven wit…
Diverging Flows detects extrapolations in flow models, ensuring reliable predictions.
problem Flow models extrapolate into invalid data, leading to silent failures.
method Structurally enforce inefficient transport for off-manifold inputs.
result Effective detection of extrapolations without compromising predictive fidelity or inference latency.
Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems using machine-learning and deep-learning models, such as automated-driving vehicles. Quality assurance frameworks are required for such machine learning systems, but there are no widely accepted and established quality-assurance …
TensorFI injects faults in TensorFlow programs to assess their reliability.
problem Ensuring reliability of machine learning systems in safety-critical domains.
method TensorFI is a flexible fault injection framework for TensorFlow applications.
result TensorFI evaluates the resilience of 12 ML programs, including autonomous vehicle DNNs.
Research tackles safety of deep learning in safety-critical tasks.
problem Safety concerns of deep learning in perception tasks for autonomous agents.
method Technical enumeration and discussions on safety concerns and mitigation methods.
result Need for more mitigation methods to ensure safety of deep learning.
Generative model improves safety in self-driving simulators and human motion generation.
problem Improving generative models for constrained domains like safety-critical applications.
method Developed Gen-neG, a denoising diffusion model that uses oracle-assisted guidance.
result Empirically validated Gen-neG for collision avoidance and safety-guarded human motion generation.
Adversarial attacks on probabilistic state-space models affect latent state and policy decisions.
problem Robust reinforcement learning under adversarial observability.
method Analyzing adversarial attacks on linear probabilistic state-space models.
result Demonstrating the influence of adversarial observations on latent state and policy decisions.
In exchange for large quantities of data and processing power, deep neural networks have yielded models that provide state of the art predication capabilities in many fields. However, a lack of strong guarantees on their behaviour have raised concerns over their use in safety-critical applications. A first step to unde…
Study evaluates uncertainty in BP estimation from PPG signals under domain shift.
problem Uncertainty quantification in healthcare, especially for cuffless BP estimation.
method Compared deep ensembles, Monte Carlo dropout, and various recalibration techniques.
result Deep ensembles provide stronger robustness under domain shift.
Data-driven models are subject to model errors due to limited and noisy training data. Key to the application of such models in safety-critical domains is the quantification of their model error. Gaussian processes provide such a measure and uniform error bounds have been derived, which allow safe control based on thes…
BagCert efficiently certifies robustness against adversarial patches on image classifiers.
problem Adversarial patches pose a threat to autonomous systems' perception component.
method BagCert combines model architecture and certification procedure for efficient inference.
result BagCert certifies 10,000 examples in 43 seconds on a single GPU, achieving 86% clean and 60% certified accuracy against 5x5 patches.
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learni…
Adversarial attacks pose a threat to deep neural networks, especially in safety-critical applications.
problem Adversarial attacks can misclassify deep neural networks, leading to safety issues.
method Adversarial attacks are categorized into white-box and black-box attacks based on the attacker's knowledge. They can be targeted or non-targeted.
result Adversarial attacks are effective and can transfer between different models and real-world scenarios.
The paper tackles safe exploration in RL by a conservative safety critic.
problem Safe exploration in reinforcement learning (RL) when partially trained policies are deployed.
method Learning a conservative safety estimate through a critic, provably bounding catastrophic failures.
result The approach provably converges to competitive task performance with significantly lower catastrophic failure rates.
A new framework for robust transfer learning that avoids negative transfer in domains with unequal information.
problem Negative transfer in unsupervised domain adaptation, especially when source and target domains have different levels of informativeness.
method Decision-theoretic framework based on Le Cam's theory of statistical experiments, using constructive approximations to replace strict invariance with directional simulability.
result Le Cam Distortion achieves near-perfect frequency estimation and zero source utility loss in various domains, demonstrating superior performance compared to traditional methods.
New bounds for kernel regression under non-Gaussian noise.
problem Uncertainty quantification for function estimates from noisy observations.
method Novel non-asymptotic probabilistic uniform error bounds for kernel-based regression.
result Proposed bounds apply to a broad class of non-Gaussian noise distributions.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
The paper tackles uncertainty in multi-objective decision-making.
problem Learning Pareto-efficient decisions with statistical confidence in uncertain outcomes.
method Adapting Pareto-efficient decisions to uncertainty, using conformal prediction.
result Statistical guarantees for efficient decisions in uncertain contexts.