Generative model simulates rare events for better decision making.
problem Rare events impact decision making but are hard to sample.
method Normalizing Flow coupled with Importance Sampling.
result Accurate estimation of rare events improves decision outcomes.
Enhanced diffusion sampling tackles rare event sampling in biomolecular simulations.
problem Efficiently sampling rare transition events in biomolecular simulations.
method Quantitative steering protocols to generate biased ensembles, followed by exact reweighting.
result Fast, accurate, and scalable estimation of equilibrium properties for folding free energies.
Enhanced diffusion sampling improves rare event sampling in biomolecular simulations.
problem Efficiently sampling rare transition events in biomolecular systems.
method Quantitative steering protocols to generate biased ensembles and exact reweighting.
result Fast, accurate, and scalable estimation of equilibrium properties.
The paper analyzes logistic regression for rare events data, deriving new insights on estimator efficiency and sampling strategies.
problem Binary logistic regression for rare events data with significantly fewer events than controls.
method Derives asymptotic distribution of MLE, proves under-sampling advantage, and compares over-sampling efficiency.
result Under-sampling a small proportion of nonevents can improve efficiency in rare events data analysis.
NOFIS uses normalizing flows to estimate rare event probabilities more efficiently.
problem Accurate estimation of rare event probabilities using conventional methods is inefficient and resource-intensive.
method NOFIS learns a sequence of proposal distributions by minimizing KL divergence losses and estimates rare event probability using importance sampling.
result NOFIS outperforms baseline approaches in estimating rare event probabilities across 10 distinct test cases.
Proposes a method to ensure accurate estimation of rare events in AI systems.
problem Lack of efficiency guarantees in black-box systems for rare-event simulation.
method Integrates deep learning with importance sampling to create a statistically guaranteed estimator.
result Demonstrates effective estimation of rare-event probabilities in AI systems.
Develops a method to estimate rare-event probabilities under distributional uncertainty.
problem Distributional uncertainty limits the effectiveness of rare-event simulation techniques.
method Wasserstein distributionally robust rare-event simulation (DRIS) framework.
result DRIS achieves vanishing relative error in estimating rare-event probabilities.
A new method estimates rare events using tensor trains.
problem Estimating rare event probabilities in high-dimensional problems.
method Approximating optimal importance distribution via tensor-train decompositions and compositions.
result Better variance reduction and efficient computation of rare event probabilities.
This paper evaluates data enrichment techniques for rare event detection in manufacturing.
problem Rare events in manufacturing lead to unplanned downtime and high energy consumption.
method Time series data augmentation, sampling, and imputation techniques combined with supervised machine learning.
result Data enrichment enhances rare failure event detection and prediction by up to 48%.
AI boosts study of rare weather extremes with lower costs.
problem Difficulty in studying rare weather events due to limited data and models.
method Coupling AI forecasts with physics models using rare-event algorithms.
result Efficiently characterizes very rare events like once-per-millennium heatwaves.
New method uses active importance sampling for rare event optimization in high-dimensional problems.
problem Optimizing complex, high-dimensional functions with rare events.
method Combines rare events sampling with neural network optimization.
result Importance sampling reduces asymptotic variance, improving generalization.
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
The paper explains how importance sampling can be used for optimization of rare events.
problem Minimizing tail risks in stochastic optimization formulations.
method Importance sampling for reducing sample requirements in estimating rare events.
result Effective importance sampling techniques for optimization of rare events.
Study compares resampling methods for rare event prediction in longitudinal studies.
problem Predicting rare events in longitudinal follow-up studies.
method Comparison of resampling methods to improve standard regression models.
result Effect of sampling rate on model predictive performance.
Proposes a method to refine PDE-driven high-dimensional rare-event simulation.
problem Challenges in constructing accurate surrogates for rare-event simulation.
method Adaptive importance sampling framework that refines a locally constructed surrogate.
result Achieves accuracy comparable to true-model adaptive importance sampling with fewer high-fidelity evaluations.
Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.
problem Rare events and metastable trapping in sampling physical systems with rough energy landscapes.
method Introduces Jeffreys Flow, a robust generative framework using Parallel Tempering distillation and symmetric Jeffreys divergence to mitigate mode collapse and improve mode coverage.
result Minimizing Jeffreys divergence suppresses mode collapse and corrects inaccuracies in multi-modal distributions.
AUC is unreliable in rare event settings but stable with moderate numbers of events.
problem Misleading performance metrics in rare event settings.
method Simulation study varying dataset sizes and event rates.
result AUC is unreliable in rare event settings but stable with moderate numbers of events.
Study rare-event simulation for neural networks and random forests.
problem Safety evaluation and robustness quantification of machine learning models.
method Importance sampling scheme integrating large deviations and sequential mixed integer programming.
result Efficiency guarantees and numerical demonstrations for various neural network architectures.
Develops a method to simulate rare dangerous events in autonomous systems.
problem Rare dangerous events in safety-critical systems are hard to test in real-world settings.
method Combines exploration, exploitation, and optimization techniques for rare-event simulation.
result Provides rigorous guarantees for the performance of the method.
New test uncovers causal links in rare event dynamics.
problem Causal discovery for rare event phenomena in dynamic systems.
method Nonparametric conditional independence test on time-invariant data.
result Validated across simulated and real-world datasets.
Low-rank MPPCA improves importance sampling in high dimensions.
problem Estimating full-rank GMM covariance matrices in high dimensions is numerically unstable.
method Use MPPCA mixtures as low-rank proposals for importance sampling in high-dimensional spaces.
result Consistent gains in sample efficiency and quality of failure distribution characterization.
Adaptive Multilevel Splitting improves rare event pricing for financial derivatives.
problem Efficient pricing of binary options in rare event regimes with discontinuous payoffs.
method Adaptive Multilevel Splitting (AMS) reformulates rare-event problem as conditional events.
result AMS achieves up to 200-fold improvements over standard Monte Carlo, preserving unbiasedness.
A new method estimates corporate bond defaults in financial networks efficiently.
problem Challenges in valuing corporate bonds in interconnected financial systems.
method Bi-Level Importance Sampling with Splitting
result The method efficiently estimates rare default events in financial networks.
A new method uses deep learning to efficiently sample rare transitions for estimating committor functions.
problem Efficiently sampling rare transitions to estimate committor functions in high-dimensional problems.
method DASTR (Deep Adaptive Sampling on Transition Paths) method using deep generative models.
result Significantly improved accuracy in approximating committor functions through efficient sampling.
SS-GEN simulates rare events in heavy and light-tailed data.
problem Estimating probabilities of extreme events in multivariate data.
method Self-Similar Generative Estimation (SS-GEN) decomposes tail distribution into radial and angular components.
result SS-GEN generates representative extreme scenarios and estimates rare-event probabilities beyond observed data.
Develops FSC for maxima nominated samples, improving classification in rare-event data.
problem Combining labeled and unlabeled data in rare-event scenarios.
method Introduces a latent representation to account for maxima nomination sampling.
result Improves classification performance in rare-event contamination mixtures.
Proposes a method to improve rare event prediction in healthcare.
problem Rare event classification in healthcare with low prevalence labels.
method Variational disentanglement approach to semi-parametric learning.
result Outperforms existing alternatives in mortality prediction on COVID-19 cohort.
Optimizes sampling for faster convergence in Bayesian experimental design and uncertainty quantification.
problem Efficiently selecting samples for faster convergence in Bayesian experimental design and uncertainty quantification.
method Output-weighted acquisition functions leveraging likelihood ratio to guide sampling towards relevant regions.
result Superiority of the proposed method in uncertainty quantification and rare event identification.
Develops methods for spectral estimation and rare-event prediction in complex systems.
problem Challenges in understanding dynamics in complex systems with many degrees of freedom.
method Inexact iterative numerical linear algebra methods for spectral estimation and rare-event prediction.
result Demonstrates methods on low-dimensional and high-dimensional models, showing their effectiveness.
New algorithm identifies best arm in rare event scenarios.
problem Identifying the best arm with tiny reward probability.
method Approximated Compound Poisson process for faster algorithms.
result Improved computational efficiency with minor sample complexity increase.
This paper shows that one cannot learn the probability of rare events without imposing further structural assumptions. The event of interest is that of obtaining an outcome outside the coverage of an i.i.d. sample from a discrete distribution. The probability of this event is referred to as the "missing mass". The impo…
A new method reduces uncertainty in predicting rare extreme events without assuming their presence in training data.
problem Predicting rare and extreme events in complex systems with high uncertainty.
method Extreme Event Aware (e2a or η) learning, which enforces extreme event statistics during training.
result Models generate unprecedented extreme events even when training data lacks extremes.
A new framework uses stochastic optimal control to estimate rare events more accurately.
problem Estimating rare events like chemical reactions in biomolecules is computationally challenging.
method The approach casts committor estimation as a stochastic optimal control problem, developing direct and off-policy Value Matching losses.
result The framework yields more accurate committor estimates, reaction rates, and equilibrium constants.
New deep learning method handles rare and imbalanced events in time series.
problem Challenges in event detection in time series data, especially rare and imbalanced events.
method Supervised regression-based deep learning approach that handles various types of events.
result Superior performance across diverse domains, particularly for rare events and imbalanced datasets.
While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the de facto evaluation environment, places the public in danger, and, due to the rare nature of accidents, will require billions of miles in or…
Reduces false positives in classifying rare online platforms.
problem Challenges in accurately identifying rare online platforms with ML.
method Calibrated probabilities and ensembles to reduce bias.
result Significantly reduces false positives in rare event detection.
Paper proposes active learning for sound event detection with reduced annotation effort.
problem Reducing annotation effort for sound event detection.
method Change point detection for candidate selection, mismatch-first farthest-traversal for selection, training with context recordings.
result The proposed system achieves similar performance to full annotation with only 2% of data, reducing annotation effort.
Markov chain Monte Carlo (MCMC) algorithms for hidden Markov models often rely on the forward-backward sampler. This makes them computationally slow as the length of the time series increases, motivating the development of sub-sampling-based approaches. These approximate the full posterior by using small random subsequ…
The paper explores how control variates can reduce variance in Monte Carlo simulations, especially for Sobolev functions.
problem Efficiency of control variates in reducing variance for Monte Carlo simulations.
method Study of a specific quadrature rule using nonparametric regression-adjusted control variates.
result A specific quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under sufficient smoothness assumptions.
Importance sampling has been known as a powerful tool to reduce the variance of Monte Carlo estimator for rare event simulation. Based on the criterion of minimizing the variance of Monte Carlo estimator within a parametric family, we propose a general account for finding the optimal tilting measure. To this end, when …
A real-world dataset is provided from a pulp-and-paper manufacturing industry. The dataset comes from a multivariate time series process. The data contains a rare event of paper break that commonly occurs in the industry. The data contains sensor readings at regular time-intervals (x's) and the event label (y). The pri…
Modern treatments for Type 1 diabetes (T1D) use devices known as artificial pancreata (APs), which combine an insulin pump with a continuous glucose monitor (CGM) operating in a closed-loop manner to control blood glucose levels. In practice, poor performance of APs (frequent hyper- or hypoglycemic events) is common en…
PHINN: A generative model for rare-event time series using persistent homology
problem Generating rare events in time series
method Flow-matching framework with dynamic Betti curves and persistence landscape loss
result Outperforms statistical and diffusion baselines in topological fidelity and tail coverage
The committor function is a central object of study in understanding transitions between metastable states in complex systems. However, computing the committor function for realistic systems at low temperatures is a challenging task, due to the curse of dimensionality and the scarcity of transition data. In this paper,…
Develops RES metrics for stable rare-event forecasting evaluation.
problem Challenges in evaluating forecasts of rare events.
method Rare-event-stable (RES) metrics designed to maintain stable thresholds under extreme rarity.
result RES metrics maintain stable thresholds, consistent model rankings, and near-complete prevalence invariance.
Neural network approximates diffusion bridges for efficiency and robustness.
problem Efficient simulation of conditioned diffusion processes, especially rare events and multimodal distributions.
method Trains a neural network to approximate bridge dynamics, eliminating MCMC and score modeling.
result Efficient sampling of conditioned diffusion bridges at comparable cost to unconditioned process.
Quantum model captures rare financial events not seen by Gaussian statistics.
problem Underestimation of rare financial events by Gaussian statistics.
method Quantum Bohmian Mechanics applied to multifractal random walk (MRW) models.
result Rare financial events generate a potential barrier in quantum potentials.
Algorithmic trading systems on DEXs reject most candidate tokens, but the counterfactual outcome of rejected candidates is rarely measured.
problem Measuring the counterfactual outcome of rejected tokens in algorithmic trading systems on decentralized exchanges.
method Post-Rejection Follow-up Sampling (PRFS) introduces a methodology for collecting and analyzing the data needed to evaluate filter precision against actual market outcomes of rejected candidates.
result PRFS produces the data needed to evaluate filter precision against actual market outcomes of rejected candidates, not against synthetic backtest reconstructions.