New method predicts rare failures in aerospace systems.
problem Rare failure prediction in aerospace applications.
method Event matching based on technical system peculiarities.
result Illustrated the method's applicability on aircraft operations.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “rare failures”
Paper proposes a method to efficiently test rare vehicle failures using kernel methods.
problem Efficiently test rarely occurring but critical vehicle failures.
method Uses kernel methods to approximate and construct sampling distributions for rare event sets.
result Proposed method robustly identifies rare vehicle failures and significantly reduces evaluation time.
Adversarial method finds rare catastrophic failures in safety-critical agents.
problem Evaluating safety-critical learning systems for catastrophic failures.
method Adversarial evaluation approach focusing on rare adversarial situations.
result Adversarial evaluation finds catastrophic failures and estimates failure rates faster.
CalNF models rare failures with limited data, improving safety in autonomous systems.
problem Challenges in modeling and debugging rare safety-critical failures due to limited data.
method CalNF, a self-regularized framework for posterior learning from limited data.
result Achieves state-of-the-art performance on data-limited failure modeling and inverse problems.
A new method estimates rare failure events in complex systems.
problem Estimating the probability of rare failure events in non-linear systems.
method Stochastic Spectral Embedding (SSE) combined with modifications for efficient rare event estimation.
result Rare failure probability decomposed into conditional probabilities for easier computation.
New framework tests AVs as a black box, prioritizing rare failure modes.
problem Lack of rigorous and scalable testing methods for AVs.
method Developed a simulation testing framework that learns to identify and rank failure scenarios via adaptive importance-sampling methods.
result First independent evaluation of a full-stack commercial AV system (Comma AI's OpenPilot).
DG separates successes and failures by gating updates with advantage and surprisal.
problem Negative learning from surprising data in distributed reinforcement learning.
method DG gates each update with the product of advantage and surprisal, suppressing failures and preserving successes.
result DG outperforms other methods in various challenging reinforcement learning tasks.
Two BO methods improve reliability optimization for rare failures.
problem Maximizing reliability of designs subject to random perturbations.
method Bayesian optimization with Thompson sampling and knowledge gradient.
result Proposed methods outperform existing techniques in extreme failure probability scenarios.
Schervish (1985b) showed that every forecasting system is noncalibrated for uncountably many data sequences that it might see. This result is strengthened here: from a topological point of view, failure of calibration is typical and calibration rare. Meanwhile, Bayesian forecasters are certain that they are calibrated-…
Proposal-Guided Greedy Surrogate Refinement for PDE-Driven High-Dimensional Rare-Event Estimationmath.NA
Proposes a method to refine PDE-driven high-dimensional rare-event simulation.
problem Challenges in constructing accurate surrogates for rare-event simulation.
method Adaptive importance sampling framework that refines a locally constructed surrogate.
result Achieves accuracy comparable to true-model adaptive importance sampling with fewer high-fidelity evaluations.
Develops a method to simulate rare dangerous events in autonomous systems.
problem Rare dangerous events in safety-critical systems are hard to test in real-world settings.
method Combines exploration, exploitation, and optimization techniques for rare-event simulation.
result Provides rigorous guarantees for the performance of the method.
Active Kriging Monte Carlo simulation method with conformal certification for failure probability estimation
problem Failure probability estimation in structural reliability analysis
method Active learning framework with conformal prediction
result Improved uncertainty quantification and reliability of failure probability estimates
Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decompositioncs.RO
New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.
problem Efficiently discovering rare failure events in autonomous vehicle simulations.
method Approximate dynamic programming and scene decomposition to estimate failure distribution.
result Increased number of failures discovered compared to baseline approaches.
Efficiently estimates rare events using multifidelity modeling.
problem Estimating rare events with computationally expensive models.
method Active learning with multifidelity modeling, adapting the number of high-fidelity simulations based on problem complexity and desired accuracy.
result Significantly reduced the number of high-fidelity model calls while maintaining accuracy.
Evaluating the Role of Data Enrichment Approaches Towards Rare Event Analysis in Manufacturingstat.ML
This paper evaluates data enrichment techniques for rare event detection in manufacturing.
problem Rare events in manufacturing lead to unplanned downtime and high energy consumption.
method Time series data augmentation, sampling, and imputation techniques combined with supervised machine learning.
result Data enrichment enhances rare failure event detection and prediction by up to 48%.
Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.
problem Rare events and metastable trapping in sampling physical systems with rough energy landscapes.
method Introduces Jeffreys Flow, a robust generative framework using Parallel Tempering distillation and symmetric Jeffreys divergence to mitigate mode collapse and improve mode coverage.
result Minimizing Jeffreys divergence suppresses mode collapse and corrects inaccuracies in multi-modal distributions.
Proposes a federated learning approach for RUL prediction from nonparametric degradation and failure signals.
problem Cost-effective RUL prediction from limited, non-shared CM signals with unknown parametric forms.
method Joint modeling of nonlinear degradation signals and failure events using federated learning.
result Superior RUL prediction compared to alternatives, validated through simulations and real data.
This paper presents a method to efficiently estimate rare event probabilities using a combination of high and low-fidelity models.
problem Estimating the probability of failure for complex systems using high-fidelity models is expensive and inaccurate for rare events.
method The paper introduces a multi-fidelity surrogate modeling strategy using active learning and subset simulation to merge high and low-fidelity models.
result The method significantly reduces computational cost while maintaining high accuracy in estimating rare event probabilities.
Low-rank MPPCA improves importance sampling in high dimensions.
problem Estimating full-rank GMM covariance matrices in high dimensions is numerically unstable.
method Use MPPCA mixtures as low-rank proposals for importance sampling in high-dimensional spaces.
result Consistent gains in sample efficiency and quality of failure distribution characterization.
Conformal prediction fails to cover minority classes in imbalanced datasets, but a class-conditional fix improves coverage.
problem Conformal prediction fails to cover minority classes in imbalanced datasets, leading to poor performance on rare labels.
method Class-conditional conformal prediction to improve coverage of minority classes.
result Class-conditional conformal prediction restores minority coverage to target with a modest increase in prediction-set size.
The paper tackles model failure detection and refitting in real-world systems.
problem Real-world data often fails statistical models due to heterogeneity.
method Develops tools for detecting and identifying model failures and refitting to improve accuracy.
result Empirical and theoretical results show the effectiveness of the proposed methodology.
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how…
CyPhERS provides real-time event info for CPSs, avoiding downtime.
problem Real-time event identification in CPSs is challenging due to complex interdependencies and rare events.
method CyPhERS integrates cyber and physical components, generating event signatures for known and unknown events.
result Event signatures provide relevant and inferable information on both known and unknown event types.
Causal Explainability of Machine Learning in Heart Failure Prediction from Electronic Health Recordsstat.ML
The paper investigates causal relationships in heart failure prediction using machine learning.
problem Understanding the causal relationships between clinical variables and heart failure.
method Proposes a new computational framework for causal structure discovery (CSD) of mixed-type clinical variables for binary disease outcomes.
result Feature importance from nonlinear classifiers strongly correlates with causal strength of variables, but not differentiating cause and effect.
New method calculates sensitivity of system failure probability.
problem Difficulty in computing sensitivity of failure probability.
method Monte Carlo strategy using response gradient and kernel smoothing.
result Single Monte Carlo run for sensitivity estimates.
The paper examines how few neuron failures affect neural network performance.
problem Understanding the impact of random neuron failures on neural network output.
method Probabilistic analysis of neural networks with small random crashes using Taylor expansion in the continuous limit.
result Provable bounds on error in neural network output under small random crashes.
Method quantifies sensitivity of reliability analysis to uncertainty sources.
problem Computational expense in reliability analysis of complex models.
method Gaussian process surrogate model, active learning, sensitivity analysis.
result Reduces main source of error in estimating rare event probabilities.
The study uses a multi-armed bandit model to analyze and mitigate hiring discrimination.
problem Hiring discrimination due to insufficient data on worker skill and characteristics.
method Multi-armed bandit model to simulate firms' learning process and policy solutions.
result Temporary affirmative actions effectively alleviate discrimination caused by data insufficiency.
Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcs.LG
Hidden stratification causes machine learning models to fail on rare but important patient subgroups.
problem Machine learning models fail on rare patient subgroups not identified during training or testing.
method Assessed techniques for measuring and describing hidden stratification effects on multiple medical imaging datasets.
result Evidence of hidden stratification leading to over 20% performance differences on clinically important subsets.
We study large deviations and rare default clustering events in a dynamic large heterogeneous portfolio of interconnected components. Defaults come as Poisson events and the default intensities of the different components in the system interact through the empirical default rate and via systematic effects that are comm…
AI-assisted framework detects and predicts rare extreme events.
problem Detecting rare extreme events in complex systems.
method Combining BED with DNOs for active learning and forecasting.
result Framework outperforms GPs and uncovers extremes without initial data.
GRAM addresses healthcare data insufficiency and interpretation challenges using graph-based attention.
problem Data insufficiency and lack of interpretability in healthcare predictive modeling.
method GRAM integrates EHR with medical ontologies, using attention mechanisms to represent medical concepts.
result GRAM outperforms RNN in accuracy and interpretability, using less data.
Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applicationsstat.OT
The book examines statistical issues with fat-tailed distributions and proposes remedies.
problem Misapplication of conventional statistical techniques to fat-tailed distributions.
method Investigates the limitations of traditional asymptotics and proposes remedies.
result Traditional statistical techniques often fail when applied to fat-tailed distributions.
Rare features reduce analysis effectiveness; new method aggregates them.
problem Reducing analysis effectiveness due to rare features in high-dimensional data.
method Aggregating rare features into denser features using a tree-based similarity encoding.
result Our method achieves high accuracy in predicting hotel ratings from reviews.
Deep learning boosts rare disease detection from medical claims.
problem Improving diagnosis and treatment of rare diseases.
method Generative adversarial networks (GANs) and recurrent neural networks for sequence modeling.
result Accurate prediction with 0.56 PR-AUC, outperforming benchmarks.
Paper uses GANs for efficient rare disease detection.
problem Efficient detection of rare diseases with limited labeled data.
method Semi-supervised learning with GANs.
result Best precision-recall scores compared to baseline techniques.
A new method tunes LOF hyperparameters for better anomaly detection.
problem Tuning hyperparameters for LOF improves anomaly detection performance.
method Heuristic methodology to tune LOF hyperparameters.
result Tuned LOF model outperforms in simulations and real data sets.
New backdoor attacks in FL can fool models on rare inputs.
problem Federated Learning's vulnerability to adversarial backdoors.
method Introducing edge-case backdoor attacks and proving their effectiveness.
result Edge-case backdoors can fool FL models on rare inputs, posing a significant threat.
Generative model simulates rare events for better decision making.
problem Rare events impact decision making but are hard to sample.
method Normalizing Flow coupled with Importance Sampling.
result Accurate estimation of rare events improves decision outcomes.
Rational expectations as a tool for predicting failure of weighted k-out-of-n reliability systemsphysics.soc-ph
Economics tool predicts failure times in reliability systems.
problem Predicting optimal failure times in weighted k-out-of-n reliability systems with heterogeneous component failure.
method Using rational expectations to analyze and predict failure times in reliability systems with heterogeneous component failure.
result Different measures are optimal for predicting system failure depending on component failure distributions.
New algorithm detects rare categories with few labels.
problem Detecting rare categories with limited labeled data.
method Dimension-driven statistics and kappa-profile.
result Algorithm performs well on separable and non-separable classes.
AI boosts study of rare weather extremes with lower costs.
problem Difficulty in studying rare weather events due to limited data and models.
method Coupling AI forecasts with physics models using rare-event algorithms.
result Efficiently characterizes very rare events like once-per-millennium heatwaves.
Graphical model predicts rare disease physicians, improving accuracy.
problem Identifying rare disease physicians from imbalanced patient data.
method Factor Graph Approach modeling physician and patient features.
result Graphical model outperforms existing targeting methodologies.
Bayesian Joint Model of Multi-Sensor and Failure Event Data for Multi-Mode Failure Predictionstat.ME
Unified model predicts multi-mode failure with multi-sensor data.
problem Independent failure mode and RUL prediction ignores inherent relationship.
method Hierarchical Bayesian framework with Cox model, Gaussian process, and multinomial distributions.
result Robust uncertainty quantification and accurate prediction of multi-mode failure.
Develops a method to estimate rare-event probabilities under distributional uncertainty.
problem Distributional uncertainty limits the effectiveness of rare-event simulation techniques.
method Wasserstein distributionally robust rare-event simulation (DRIS) framework.
result DRIS achieves vanishing relative error in estimating rare-event probabilities.
Improved MCMC for rare events in hidden Markov models.
problem Slow inference and prediction for rare latent states in hidden Markov models.
method Targeted sub-sampling (TASS) over-samples rare latent states, reducing variance in gradient estimation.
result Substantial gains in predictive and inferential accuracy on real and synthetic examples.
Framework classifies machine learning failures into intentional and unintentional.
problem Understanding and preventing failures in machine learning systems.
method Developed a taxonomy of machine learning failure modes.
result Stakeholders found the framework useful for discussing machine learning failures.
RaRecognize learns to recognize rare classes in a stream of data.
problem Learning to recognize rare classes in a continuous stream of data.
method Estimates a general decision boundary, learns individual rare subclasses, flags new subclasses.
result RaRecognize outperforms state-of-the-art baselines on real-world datasets.