Develops FSC for maxima nominated samples, improving classification in rare-event data.
problem Combining labeled and unlabeled data in rare-event scenarios.
method Introduces a latent representation to account for maxima nomination sampling.
result Improves classification performance in rare-event contamination mixtures.
Develops a method to estimate rare-event probabilities under distributional uncertainty.
problem Distributional uncertainty limits the effectiveness of rare-event simulation techniques.
method Wasserstein distributionally robust rare-event simulation (DRIS) framework.
result DRIS achieves vanishing relative error in estimating rare-event probabilities.
Generative model simulates rare events for better decision making.
problem Rare events impact decision making but are hard to sample.
method Normalizing Flow coupled with Importance Sampling.
result Accurate estimation of rare events improves decision outcomes.
AI boosts study of rare weather extremes with lower costs.
problem Difficulty in studying rare weather events due to limited data and models.
method Coupling AI forecasts with physics models using rare-event algorithms.
result Efficiently characterizes very rare events like once-per-millennium heatwaves.
Study compares resampling methods for rare event prediction in longitudinal studies.
problem Predicting rare events in longitudinal follow-up studies.
method Comparison of resampling methods to improve standard regression models.
result Effect of sampling rate on model predictive performance.
Proposes a method to ensure accurate estimation of rare events in AI systems.
problem Lack of efficiency guarantees in black-box systems for rare-event simulation.
method Integrates deep learning with importance sampling to create a statistically guaranteed estimator.
result Demonstrates effective estimation of rare-event probabilities in AI systems.
The paper analyzes logistic regression for rare events data, deriving new insights on estimator efficiency and sampling strategies.
problem Binary logistic regression for rare events data with significantly fewer events than controls.
method Derives asymptotic distribution of MLE, proves under-sampling advantage, and compares over-sampling efficiency.
result Under-sampling a small proportion of nonevents can improve efficiency in rare events data analysis.
NOFIS uses normalizing flows to estimate rare event probabilities more efficiently.
problem Accurate estimation of rare event probabilities using conventional methods is inefficient and resource-intensive.
method NOFIS learns a sequence of proposal distributions by minimizing KL divergence losses and estimates rare event probability using importance sampling.
result NOFIS outperforms baseline approaches in estimating rare event probabilities across 10 distinct test cases.
Enhanced diffusion sampling improves rare event sampling in biomolecular simulations.
problem Efficiently sampling rare transition events in biomolecular systems.
method Quantitative steering protocols to generate biased ensembles and exact reweighting.
result Fast, accurate, and scalable estimation of equilibrium properties.
Enhanced diffusion sampling tackles rare event sampling in biomolecular simulations.
problem Efficiently sampling rare transition events in biomolecular simulations.
method Quantitative steering protocols to generate biased ensembles, followed by exact reweighting.
result Fast, accurate, and scalable estimation of equilibrium properties for folding free energies.
PHINN: A generative model for rare-event time series using persistent homology
problem Generating rare events in time series
method Flow-matching framework with dynamic Betti curves and persistence landscape loss
result Outperforms statistical and diffusion baselines in topological fidelity and tail coverage
Study rare-event simulation for neural networks and random forests.
problem Safety evaluation and robustness quantification of machine learning models.
method Importance sampling scheme integrating large deviations and sequential mixed integer programming.
result Efficiency guarantees and numerical demonstrations for various neural network architectures.
A new method estimates rare events using tensor trains.
problem Estimating rare event probabilities in high-dimensional problems.
method Approximating optimal importance distribution via tensor-train decompositions and compositions.
result Better variance reduction and efficient computation of rare event probabilities.
New test uncovers causal links in rare event dynamics.
problem Causal discovery for rare event phenomena in dynamic systems.
method Nonparametric conditional independence test on time-invariant data.
result Validated across simulated and real-world datasets.
Adaptive Multilevel Splitting improves rare event pricing for financial derivatives.
problem Efficient pricing of binary options in rare event regimes with discontinuous payoffs.
method Adaptive Multilevel Splitting (AMS) reformulates rare-event problem as conditional events.
result AMS achieves up to 200-fold improvements over standard Monte Carlo, preserving unbiasedness.
AUC is unreliable in rare event settings but stable with moderate numbers of events.
problem Misleading performance metrics in rare event settings.
method Simulation study varying dataset sizes and event rates.
result AUC is unreliable in rare event settings but stable with moderate numbers of events.
A real-world dataset is provided from a pulp-and-paper manufacturing industry. The dataset comes from a multivariate time series process. The data contains a rare event of paper break that commonly occurs in the industry. The data contains sensor readings at regular time-intervals (x's) and the event label (y). The pri…
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
Reduces false positives in classifying rare online platforms.
problem Challenges in accurately identifying rare online platforms with ML.
method Calibrated probabilities and ensembles to reduce bias.
result Significantly reduces false positives in rare event detection.
A cased-based reasoning method predicts rare events on strategic sites using satellite imagery.
problem Manual prediction of rare events on strategic sites is impractical due to large datasets.
method Case-based reasoning approach incorporating expert knowledge for irregular time series and small datasets.
result The method significantly outperforms random selection on challenging applications.
MIM-based GAN improves rare event generation in GANs.
problem Improving rare event generation in GANs.
method Adopting MIM (exponential form of information metric) to replace KL divergence in GANs.
result MIM-based GAN achieves state-of-the-art performance in anomaly detection.
New method uses active importance sampling for rare event optimization in high-dimensional problems.
problem Optimizing complex, high-dimensional functions with rare events.
method Combines rare events sampling with neural network optimization.
result Importance sampling reduces asymptotic variance, improving generalization.
Proposes a method to improve rare event prediction in healthcare.
problem Rare event classification in healthcare with low prevalence labels.
method Variational disentanglement approach to semi-parametric learning.
result Outperforms existing alternatives in mortality prediction on COVID-19 cohort.
Develops RES metrics for stable rare-event forecasting evaluation.
problem Challenges in evaluating forecasts of rare events.
method Rare-event-stable (RES) metrics designed to maintain stable thresholds under extreme rarity.
result RES metrics maintain stable thresholds, consistent model rankings, and near-complete prevalence invariance.
Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.
problem Rare events and metastable trapping in sampling physical systems with rough energy landscapes.
method Introduces Jeffreys Flow, a robust generative framework using Parallel Tempering distillation and symmetric Jeffreys divergence to mitigate mode collapse and improve mode coverage.
result Minimizing Jeffreys divergence suppresses mode collapse and corrects inaccuracies in multi-modal distributions.
Proposes a method to refine PDE-driven high-dimensional rare-event simulation.
problem Challenges in constructing accurate surrogates for rare-event simulation.
method Adaptive importance sampling framework that refines a locally constructed surrogate.
result Achieves accuracy comparable to true-model adaptive importance sampling with fewer high-fidelity evaluations.
SS-GEN simulates rare events in heavy and light-tailed data.
problem Estimating probabilities of extreme events in multivariate data.
method Self-Similar Generative Estimation (SS-GEN) decomposes tail distribution into radial and angular components.
result SS-GEN generates representative extreme scenarios and estimates rare-event probabilities beyond observed data.
PRESTO improves rare event prediction by shrinking towards proportional odds model.
problem Difficult to predict rare events due to class imbalance.
method PRESTO relaxes proportional odds model by estimating separate weights for transitions between categories, imposing L1 penalty to shrink towards proportional odds.
result PRESTO consistently estimates decision boundary weights under sparsity assumption, improving rare probability estimation.
This paper evaluates data enrichment techniques for rare event detection in manufacturing.
problem Rare events in manufacturing lead to unplanned downtime and high energy consumption.
method Time series data augmentation, sampling, and imputation techniques combined with supervised machine learning.
result Data enrichment enhances rare failure event detection and prediction by up to 48%.
A contamination in a 3-manifold is an object interpolating between the contact structure and the lamination. Contaminations seem to provide a link between 3-dimensional contact geometry and the classical topology of 3-manifolds, as described in a separate paper. In this paper we deal with contaminations carried by bran…
A robust loss for anomaly mitigation and unsupervised contamination classification
problem Detecting and mitigating contamination in supervised and unsupervised settings
method Neural Bayesian Anomaly Mitigation (NBAM)
result Recovering the structure of contamination and identifying label-flip pairs
New method learns diffusion bridges for rare events.
problem Simulating rare events in diffusion processes.
method Iterative online learning based on self-consistency.
result Strong performance in various empirical settings.
Study minimax robustness in statistical estimation under Wasserstein contamination.
problem Adversarial perturbations in statistical data.
method Developed minimax theory for ℓqr losses under Wasserstein-r contaminations. result Exact minimax risk identified for joint contaminations in location estimation and prediction in linear regression.
Study shows robust method for estimating density ratios even with heavy contamination.
problem Estimating density ratios in the presence of heavy contamination.
method Weighted density ratio estimation (DRE) with doubly strong robustness.
result Weighted DRE achieves sparse consistency under heavy contamination.
Generative models can still learn from contaminated data, but with limitations.
problem How much contamination can generative models tolerate?
method Characterized robustness under contaminated enumerations, proving generation is achievable for all countable collections if contamination fraction converges to zero.
result Generation under contamination is achievable for all countable collections if contamination fraction converges to zero, but dense generation is strictly less robust.
New algorithm identifies best arm in rare event scenarios.
problem Identifying the best arm with tiny reward probability.
method Approximated Compound Poisson process for faster algorithms.
result Improved computational efficiency with minor sample complexity increase.
Algorithm identifies Pareto set in bandits with contaminated feedback.
problem Identifying Pareto set in multi-objective bandits with adversarial contamination.
method Sample median-based multi-objective adaptive elimination algorithm.
result Sample complexity bound that depends on contamination probability.
The paper uses machine learning to compute rare event probabilities in stochastic systems.
problem Characterizing rare events in stochastic dynamical systems with weak noise.
method Developed a neural network framework for computing quasipotential, most probable paths, and prefactors.
result Demonstrated higher effectiveness and accuracy of the algorithm in calculating mean exit times.
The paper explains how importance sampling can be used for optimization of rare events.
problem Minimizing tail risks in stochastic optimization formulations.
method Importance sampling for reducing sample requirements in estimating rare events.
result Effective importance sampling techniques for optimization of rare events.
New algorithm resists contamination in high-dimensional regression with optimal performance.
problem Adversarial and measurement errors in high-dimensional data.
method Adversarial Contamination-resistant Iterative Hard Thresholding (AC-IHT) algorithm.
result Achieves minimax near-optimal estimation and signal-adaptive support recovery.
Develops methods for spectral estimation and rare-event prediction in complex systems.
problem Challenges in understanding dynamics in complex systems with many degrees of freedom.
method Inexact iterative numerical linear algebra methods for spectral estimation and rare-event prediction.
result Demonstrates methods on low-dimensional and high-dimensional models, showing their effectiveness.
Motivated by applications of bandit algorithms in education, we consider a stochastic multi-armed bandit problem with ε-contaminated rewards. We allow an adversary to give arbitrary unbounded contaminated rewards with full knowledge of the past and future. We impose the constraint that for each time t the…
Paper introduces RAS for robust MTL with contamination.
problem Representation-based multi-task learning struggles with contamination.
method Robust and Adaptive Spectral (RAS) method.
result RAS prevents negative transfer and performs well with up to 80% contamination.
Trimming helps in conformal prediction when it separates anomaly scores.
problem Effectiveness of trimming in conformal prediction under contamination.
method Analyse fixed-threshold trimming as a replacement of the contaminated calibration law with a retained law.
result Trimming helps when it separates anomaly scores, reducing clean-target coverage to a one-dimensional score-CDF transfer problem.
New deep learning method handles rare and imbalanced events in time series.
problem Challenges in event detection in time series data, especially rare and imbalanced events.
method Supervised regression-based deep learning approach that handles various types of events.
result Superior performance across diverse domains, particularly for rare events and imbalanced datasets.
Improved robust regression with clean covariates achieves better rates than Huber's model.
problem Robust regression under adaptive contamination of responses with clean covariates.
method Exploiting clean covariates to construct an estimator achieving better rates than Huber's model.
result Improved estimation rate even with constant contamination, achieving consistency.
Bayesian method estimates contamination factor for unsupervised anomaly detection.
problem No good methods for estimating contamination factor in unsupervised anomaly detection.
method Bayesian approach using mixture formulation of anomaly detector outputs.
result Estimated contamination factor distribution is well-calibrated and improves anomaly detection performance.
New test ensures quality of shared data in machine learning.
problem Ensuring quality of external data in machine learning tasks.
method Distribution-free two-sample testing procedures grounded in conformal outlier detection.
result Identifies valuable external data agents for model personalization.