Framework classifies machine learning failures into intentional and unintentional.
problem Understanding and preventing failures in machine learning systems.
method Developed a taxonomy of machine learning failure modes.
result Stakeholders found the framework useful for discussing machine learning failures.
Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.
problem Stability and sample quality issues in GAN training.
method Spectral normalization controls gradient explosion and vanishing, improving GAN training stability and sample quality.
result Bidirectional Scaled Spectral Normalization (BSSN) outperforms standard spectral normalization in sample quality and training stability.
Training models to prefer certain responses can unintentionally shift probability to harmful ones.
problem Likelihood displacement in DPO models, leading to unintended unalignment.
method Characterized and mitigated likelihood displacement using CHES score.
result Training models to prefer certain responses can unintentionally shift probability mass to harmful responses.
Hardware architectures composed of resistive cross-point device arrays can provide significant power and speed benefits for deep neural network training workloads using stochastic gradient descent (SGD) and backpropagation (BP) algorithm. The training accuracy on this imminent analog hardware however strongly depends o…
Predictive models are increasingly deployed for the purpose of determining access to services such as credit, insurance, and employment. Despite potential gains in productivity and efficiency, several potential problems have yet to be addressed, particularly the potential for unintentional discrimination. We present an…
Modeling HFT interactions reveals market instability.
problem Market instability caused by HFT dynamic coupling.
method Developed a recurrence relations framework to model HFT interactions.
result Unexpected latency and feedback can trigger market instability.
The paper assesses fairness in AI for financial services, using statistical methods.
problem Unintentional bias and insufficient model validation in AI applications.
method Statistical methods for imbalanced data treatment and bias mitigation.
result Fairness evaluation metrics applied to a credit card default payment example.
Unified framework improves fair classification by selecting representative data points.
problem Improving fair classification outcomes in the presence of unintentional biases.
method Develops a unified framework to jointly optimize accuracy and fairness, recasting as mixed-integer convex programs.
result The framework can be used to enhance classification fairness by selecting more representative data points.
Our goal is to identify beneficial interventions from observational data. We consider interventions that are narrowly focused (impacting few covariates) and may be tailored to each individual or globally enacted over a population. For applications where harmful intervention is drastically worse than proposing no change…
The paper proposes a test for trading intelligence similar to the Turing Test.
problem Determining if trading programs exhibit intelligent behavior.
method Develops a test based on the Turing Test for financial trading programs.
result Introduces a method to assess trading intelligence in financial markets.
Economics tool predicts failure times in reliability systems.
problem Predicting optimal failure times in weighted k-out-of-n reliability systems with heterogeneous component failure.
method Using rational expectations to analyze and predict failure times in reliability systems with heterogeneous component failure.
result Different measures are optimal for predicting system failure depending on component failure distributions.
Unintentional falls can cause severe injuries and even death, especially if no immediate assistance is given. The aim of Fall Detection Systems (FDSs) is to detect an occurring fall. This information can be used to trigger the necessary assistance in case of injury. This can be done by using either ambient-based sensor…
Unified model predicts multi-mode failure with multi-sensor data.
problem Independent failure mode and RUL prediction ignores inherent relationship.
method Hierarchical Bayesian framework with Cox model, Gaussian process, and multinomial distributions.
result Robust uncertainty quantification and accurate prediction of multi-mode failure.
PAGER detects failures in deep regression models using a new framework.
problem Detecting failures in deep regression models.
method PAGER uses a combination of epistemic uncertainty and manifold non-conformity scores.
result PAGER accurately characterizes and detects failures in deep regressors.
The study tests and optimizes fairness in credit scoring models.
problem Discrimination in credit scoring models based on protected attributes.
method Formal testing and variable identification to optimize fairness and performance.
result Guidance on monitoring and improving algorithmic fairness in credit scoring.
Risk Advisor predicts and mitigates ML deployment failures.
problem Predicting and mitigating test-time failure risks of ML systems.
method Post-hoc meta-learner for estimating failure risks and uncertainties.
result Reliably predicts deployment-time failure risks across various ML models.
Prognostics and Health Management (PHM) is an emerging engineering discipline which is concerned with the analysis and prediction of equipment health and performance. One of the key challenges in PHM is to accurately predict impending failures in the equipment. In recent years, solutions for failure prediction have evo…
CalNF models rare failures with limited data, improving safety in autonomous systems.
problem Challenges in modeling and debugging rare safety-critical failures due to limited data.
method CalNF, a self-regularized framework for posterior learning from limited data.
result Achieves state-of-the-art performance on data-limited failure modeling and inverse problems.
This work improves safety validation of autonomous vehicles by finding interpretable failures.
problem Finding interpretable failures of autonomous systems in simulation.
method Signal temporal logic expressions optimized for high likelihood and human interpretability.
result Our methodology finds more interpretable failures with higher likelihood compared to baseline approaches.
Framework predicts remaining useful life of DSH subsystems under unknown failure modes.
problem Predicting remaining useful life of DSH subsystems with unknown failure modes.
method Unsupervised framework using mixture of Gaussian regressions and Expectation-Maximization algorithm.
result Improved prediction accuracy and interpretability of RUL.
The paper calculates the likelihood of a financial market failure involving multiple major banks.
problem Estimating the probability of a market failure involving multiple globally important banks.
method Multivariate Cox process across G-SIBs, deriving various theorems on market failure probabilities.
result The probability of a market failure increases with the number of G-SIBs and is inevitable if there are too many.
Paper proposes a method to predict disk failures using multi-layer domain adaptive learning.
problem Traditional machine learning models struggle to predict disk failures due to limited data.
method Multi-layer domain adaptive learning with source and target domains.
result The proposed method improves failure prediction accuracy on disk data with few failure samples.
We aim to predict and explain service failures in supply-chain networks, more precisely among last-mile pickup and delivery services to customers. We analyze a dataset of 500,000 services using (1) supervised classification with Random Forests, and (2) Association Rules. Our classifier reaches an average sensitivity of…
GE finds failures in autonomous systems without domain heuristics.
problem Finding failures in autonomous systems without domain-specific heuristics.
method Adaptive stress testing using go-explore (GE) algorithm.
result GE finds failures in scenarios other RL techniques cannot solve.
In the artificial intelligence field, learning often corresponds to changing the parameters of a parameterized function. A learning rule is an algorithm or mathematical expression that specifies precisely how the parameters should be changed. When creating an artificial intelligence system, we must make two decisions: …
Method distinguishes between failures and domain shifts in industrial data streams.
problem Confusing domain shifts with failures in industrial data.
method Modified Page-Hinkley changepoint detector and supervised domain-adaptation-based anomaly detection.
result Allows differentiation between failures and domain shifts.
PEF identifies the best subgroup performance balance for fairness.
problem Fairness constraints can degrade performance in skewed datasets.
method PEF identifies the closest operating point on the Pareto curve of subgroup performances.
result PEF achieves Pareto levels in accuracy for all subgroups.
System predicts respiratory failure up to 8 hours early.
problem Early detection of respiratory failure in ICU patients.
method Machine learning on ICU patient monitoring data.
result System outperforms traditional clinical decision-making.
New method avoids failures in physics-constrained systems using active learning.
problem Handling fatal failures in systems governed by physics constraints.
method Develops a novel active learning method that considers implicit physics constraints.
result Achieves zero-failure in composite fuselage assembly process without explicit failure regions.
This paper addresses the problem of evaluating learning systems in safety critical domains such as autonomous driving, where failures can have catastrophic consequences. We focus on two problems: searching for scenarios when learned agents fail and assessing their probability of failure. The standard method for agent e…
DeepSIP predicts network failures' impact using CNN from syslog and traffic data.
problem Predicting service impact from network failures.
method Temporal multimodal CNN for predicting time to recovery and traffic loss.
result DeepSIP reduced prediction error by approximately 50%.
RODMAN improves ML-based disk failure prediction accuracy in cloud environments.
problem Imperfect data quality in real-world cloud environments degrades ML-based disk failure prediction accuracy.
method RODMAN uses three data preprocessing techniques: failure-type filtering, spline-based data filling, and automated pre-failure backtracking.
result RODMAN significantly improves prediction accuracy compared to no preprocessing.
Two BO methods improve reliability optimization for rare failures.
problem Maximizing reliability of designs subject to random perturbations.
method Bayesian optimization with Thompson sampling and knowledge gradient.
result Proposed methods outperform existing techniques in extreme failure probability scenarios.
New method finds failures in high-fidelity simulators with fewer steps.
problem Finding failures in high-fidelity simulators is expensive and impractical.
method Adaptive stress testing with backward algorithm adaptation from low-fidelity to high-fidelity.
result Significantly fewer high-fidelity simulation steps needed to find failures.
Entropy-based GP adaptive design improves failure probability estimation.
problem Limited accuracy in failure probability estimation due to model evaluation costs.
method Entropy-based Gaussian process (GP) adaptive design combined with multifidelity importance sampling (MFIS).
result More accurate failure probability estimates and higher confidence.
Brain uses synaptic failure to sample from posterior distributions.
problem Bayesian inference in the brain's probabilistic computations.
method Adapting synaptic failure to sample posterior predictive distributions.
result Synaptic failure enables sampling of complete posterior predictive distributions.
Study constructs balanced datasets for seismic failure prediction.
problem Imbalanced datasets limit machine learning performance in seismic failure prediction.
method Framework with three steps: GMF identification, probability density estimation, and sample transformation.
result Framework improves machine learning performance in seismic failure mode prediction.
Proposes a deep neural network for early disk drive failure prediction.
problem Early prediction of disk drive failure using multivariate time series sensor data.
method Enriched features derived from sensor data through transformations, combined with ensemble learning and deep neural network architecture.
result Significantly improved classification accuracy in predicting disk drive failure.
Improves FI-PINNs by combining re-sampling and subset simulation for better failure probability estimation.
problem Estimating failure probability in physics-informed neural networks (PINNs).
method Adaptive sampling with re-sampling and subset simulation, using cosine-annealing for uniform to adaptive transition.
result Significant improvement in estimating failure probability and generating new training points in the failure region.
A new algorithm learns from failures to optimize under constraints efficiently.
problem Optimizing under unknown constraints with limited failure tolerance.
method Excursion search controls risk as a function of a failure budget.
result The algorithm achieves lower regret and uses failures budget more efficiently.
Determining possible failure scenarios is a critical step in the evaluation of autonomous vehicle systems. Real-world vehicle testing is commonly employed for autonomous vehicle validation, but the costs and time requirements are high. Consequently, simulation-driven methods such as Adaptive Stress Testing (AST) have b…
New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.
problem Efficiently discovering rare failure events in autonomous vehicle simulations.
method Approximate dynamic programming and scene decomposition to estimate failure distribution.
result Increased number of failures discovered compared to baseline approaches.
Adaptive PINNs improve accuracy by adding points where solutions are uncertain.
problem Inadequate sampling in PINNs leads to inaccurate solutions, especially near singularities.
method FI-PINNs use failure probability to dynamically add points, improving numerical accuracy.
result FI-PINNs achieve better accuracy through adaptive sampling, as proven by rigorous error bounds.
Bayesian inference models failure distributions in autonomous systems.
problem Estimating the distribution of failures in complex systems.
method Bayesian inference using system dynamics rollouts and gradient computation.
result Improves sample efficiency and parameter space coverage in autonomous systems.
We develop a new method to estimate failure probabilities in complex systems.
problem Estimating failure probabilities in safety-critical autonomous systems is challenging due to the rarity of failures and large state spaces.
method We propose an adaptive importance sampling algorithm that minimizes forward Kullback-Leibler divergence and uses Markov score ascent methods.
result Our method provides more accurate failure probability estimates than existing techniques.
Machine learning predicts failure in brittle materials with high accuracy.
problem Predicting failure in brittle materials under repetitive loads.
method Phase-field model combined with supervised machine learning.
result Framework predicts failure with acceptable accuracy even in noisy data.
Study predicts P2P lending platform failures using machine learning.
problem Predicting failures of P2P lending platforms in China.
method Used machine learning models with filter and wrapper methods, forward selection, and backward elimination.
result Identified robust variables for predicting platform failures with high AUC and F1 scores.
This study uses TDA to map corporate failure, revealing distinct regions of risk.
problem Understanding and predicting corporate default risk.
method Topological Data Analysis (TDA) applied to Altman's Z-score model.
result Firms do not cluster neatly along default predictors, suggesting complex risk landscapes.