Study improves estimation of rare language model outputs.
problem Estimating probabilities of rare outputs in language models.
method Importance sampling vs. activation extrapolation for low probability estimation.
result Importance sampling outperforms activation extrapolation.
New algorithms estimate and test collision probability with near-optimal sample complexity.
problem Estimating and testing collision probability in discrete distributions.
method Developed algorithms for ( α , β ) (α, β) ( α , β ) -local differential privacy and sequential testing. result Achieved nearly optimal sample complexity for estimating and testing collision probability.
Proposes a neural network method to combine nonprobability and probability survey samples.
problem Combining nonprobability and probability survey samples for accurate population mean estimation.
method Uses a deep neural network to estimate sampling scores from nonprobability samples and combines them with probability sample information.
result Proposed estimators improve robustness to parametric propensity-score misspecification, especially for nonlinear selection mechanisms.
New methods for estimating causal effects with limited overlap, using Stable Probability Weighting.
problem Estimating causal effects with limited overlap in multivalued treatments.
method Stable Probability Weighting (SPW) and Finite-Sample Stable Probability Weighting (FPW) methods.
result SPW and FPW provide practical solutions for estimating and inferring causal effects with limited overlap.
The paper examines how sampling data affects the performance of submodular maximization.
problem Performance loss due to probability sampling in submodular maximization.
method Examines a simple probability sampling method where each data point is selected with probability at least r.
result The sampling gap is both upper and lower bounded by 1/r for policywise submodular utility functions.
We develop a new method to estimate failure probabilities in complex systems.
problem Estimating failure probabilities in safety-critical autonomous systems is challenging due to the rarity of failures and large state spaces.
method We propose an adaptive importance sampling algorithm that minimizes forward Kullback-Leibler divergence and uses Markov score ascent methods.
result Our method provides more accurate failure probability estimates than existing techniques.
NOFIS uses normalizing flows to estimate rare event probabilities more efficiently.
problem Accurate estimation of rare event probabilities using conventional methods is inefficient and resource-intensive.
method NOFIS learns a sequence of proposal distributions by minimizing KL divergence losses and estimates rare event probability using importance sampling.
result NOFIS outperforms baseline approaches in estimating rare event probabilities across 10 distinct test cases.
New method estimates and samples high-dimensional probability distributions avoiding optimization and approximation curse.
problem Estimating high-dimensional probability distributions from data samples.
method Hierarchic probability flow from coarse to fine scales, defined by conditional probabilities across scales.
result Sampling hierarchic models avoids critical slowing down at phase transitions and generates turbulence and dark matter images.
Improved matrix completion for non-uniformly sampled data.
problem Estimating unobserved entries in a matrix with varying sampling probabilities.
method Developed entry-specific bounds for low-rank matrix completion under structured non-uniform sampling.
result Error bounds for each entry match minimax lower bounds under certain conditions.
In this paper, we develop a general theory of truncated inverse binomial sampling. In this theory, the fixed-size sampling and inverse binomial sampling are accommodated as special cases. In particular, the classical Chernoff-Hoeffding bound is an immediate consequence of the theory. Moreover, we propose a rigorous and…
The study examines how shallow neural nets converge to training samples or manifold points during diffusion.
problem Understanding when and how shallow neural nets converge to training samples or manifold points during diffusion.
method Analysis of shallow ReLU neural network denoisers trained with minimal ℓ 2 \ell^2 ℓ 2 norm, comparing score flow and diffusion flow. result Probability flow converges to training points, sums of training points, or manifold points, depending on the diffusion time scheduler.
A nonparametric two-sample test using a parametric integral probability metric
problem Detecting distributional differences between two independent samples
method Propose a new two-sample test statistic based on a newly introduced integral probability metric (IPM)
result Establish theoretical guarantees for the associated two-sample testing procedure
Two BO methods improve reliability optimization for rare failures.
problem Maximizing reliability of designs subject to random perturbations.
method Bayesian optimization with Thompson sampling and knowledge gradient.
result Proposed methods outperform existing techniques in extreme failure probability scenarios.
This paper finds a unique partition of a sample space for estimating continuous distributions.
problem Estimating continuous probability distributions from finite samples.
method Equal-probability partition of the sample space using order statistics.
result The partition yields an entropy of log2(N+1) bits, providing a discrete entropy estimate.
Optimal testing of discrete distributions with high probability, achieving sample complexity bounds.
problem Testing discrete distributions with high probability accuracy.
method Characterizing sample complexity as a function of parameters like δ, providing sample-optimal testers.
result Optimal algorithms for closeness and independence testing, achieving within constant factors of information-theoretic lower bounds.
New research shows some distributions hard to sample via diffusions.
problem Some distributions hard to sample via diffusions.
method Learning drifts of diffusions to approximate target distributions.
result Superpolynomially close drifts can yield very far approximations.
A new algorithm approximates logistic regression probabilities efficiently.
problem Efficiently approximating probabilities in logistic regression for large datasets.
method Randomized sampling-based algorithm with leverage scores.
result Accurate approximations to estimated probabilities with smaller sample sizes.
In this paper, we consider the problem of column subset selection. We present a novel analysis of the spectral norm reconstruction for a simple randomized algorithm and establish a new bound that depends explicitly on the sampling probabilities. The sampling dependent error bound (i) allows us to better understand the …
New MCMC method samples from lattice distributions efficiently.
problem Sampling from probability distributions on lattice structures.
method Metropolis-Hastings algorithm with a pull-back measure.
result The method is uniformly ergodic under certain conditions.
New Gibbs sampling method improves MCMC efficiency.
problem Improving efficiency of Gibbs sampling.
method Non-uniform random scan with selection probability optimization.
result Non-uniform scan improves mixing time of Markov chain.
Sampling is a fundamental problem in computer science and statistics. However, for a given task and stream, it is often not possible to choose good sampling probabilities in advance. We derive a general framework for adaptively changing the sampling probabilities via a collection of thresholds.In general, adaptive samp…
Theoretical proof shows COMs are a type of contrastive divergence model with improved sampling.
problem Improving sampling quality in offline model-based optimization.
method Showed COMs are contrastive divergence models, proposed Langevin MCMC sampler, and decoupled model.
result Improved sampling quality achieved by decoupling model and using Langevin MCMC.
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
Develops asymptotic analysis for RandNLA sampling estimators in least-squares problems.
problem Lack of distributional information for RandNLA estimators in statistical inference.
method Asymptotic analysis of sampling estimators for least-squares problems in two settings.
result Sampling estimators are asymptotically normally distributed under mild conditions.
The paper studies PCA of probability measures with varying sample sizes and finds optimal convergence rates.
problem PCA of multiple probability measures with varying sample sizes.
method Double asymptotic regime analysis with convergence rates n − 1 / 2 + m − α n^{-1/2} + m^{-α} n − 1/2 + m − α for empirical covariance and PCA risk. result Optimal convergence rates for empirical covariance and PCA risk in the dense regime are proven.
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
Improves FI-PINNs by combining re-sampling and subset simulation for better failure probability estimation.
problem Estimating failure probability in physics-informed neural networks (PINNs).
method Adaptive sampling with re-sampling and subset simulation, using cosine-annealing for uniform to adaptive transition.
result Significant improvement in estimating failure probability and generating new training points in the failure region.
ES reduces high-probability regret in stochastic linear bandits.
problem High-probability regret in stochastic linear bandits.
method Linear ensemble sampling with standard Gaussian perturbations, analyzing m = Θ ( d log n ) m=Θ(d\log n) m = Θ ( d log n ) ensemble size. result ES achieves i l d e O ( d 3 / 2 n ) ilde O(d^{3/2}\sqrt n) i l d e O ( d 3/2 n ) high-probability regret, closing the gap to Thompson sampling. Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained d…
Accurate noise modelling is important for training of deep learning reconstruction algorithms. While noise models are well known for traditional imaging techniques, the noise distribution of a novel sensor may be difficult to determine a priori. Therefore, we propose learning arbitrary noise distributions. To do so, th…
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Estimating the largest community in a mixed population via sequential sampling.
problem Identifying the largest community in a mixed population with limited sampling.
method Sequential, random sampling of individuals across multiple boxes, optimizing sampling strategy and decision rule.
result Proposed algorithms achieve optimal error probability decay rates under fixed budget constraints.
Determinantal point processes (DPPs) are specific probability distributions over clouds of points that are used as models and computational tools across physics, probability, statistics, and more recently machine learning. Sampling from DPPs is a challenge and therefore we present DPPy, a Python toolbox that gathers kn…
New sampling method uses stochastic interpolants and FBSDEs.
problem Sampling from high-dimensional distributions with unnormalized densities.
method Stochastic interpolants and FBSDEs to define and solve diffusion process.
result Effective sampling from challenging distributions.
Building on the success of deep learning, two modern approaches to learn a probability model from the data are Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs). VAEs consider an explicit probability model for the data and compute a generative distribution by maximizing a variational lower-boun…
In this paper, for μ μ μ and ν ν ν two probability measures on R d \mathbb{R}^d R d with finite moments of order ρ ≥ 1 ρ\ge 1 ρ ≥ 1 , we define the respective projections for the W ρ W_ρ W ρ -Wasserstein distance of μ μ μ and ν ν ν on the sets of probability measures dominated by ν ν ν and of probability measures larger than μ μ μ in the convex order. Th…
Estimator calculates surface curvature from point cloud samples.
problem Accurately estimating curvature from limited point cloud data.
method Algorithm using probability distribution and nearby points control.
result Controlled number of points ensures accurate curvature estimation.
New methods ensure feature importance rankings are correct with high probability.
problem Stability issues in feature importance scores due to random sampling.
method Hypothesis testing-based techniques to assess and verify the stability of top-ranked features.
result Ensures the most important features are correct with high-probability guarantees.
We consider the problem of low canonical polyadic (CP) rank tensor completion. A completion is a tensor whose entries agree with the observed entries and its rank matches the given CP rank. We analyze the manifold structure corresponding to the tensors with the given rank and define a set of polynomials based on the sa…
A new test improves statistical inference in bandit algorithms without sacrificing adaptiveness.
problem Challenges in statistical inference for adaptive randomised experiments in bandits.
method An allocation probability test for Thompson Sampling without trading-off regret or requiring large sample sizes.
result Improves statistical inference in small samples, showing advantages in mental health experiments.
We develop importance sampling based efficient simulation techniques for three commonly encountered rare event probabilities associated with random walks having i.i.d. regularly varying increments; namely, 1) the large deviation probabilities, 2) the level crossing probabilities, and 3) the level crossing probabilities…
Study tests if a probability measure is near a real algebraic variety.
problem Deciding if a probability measure is near a real algebraic variety.
method Proved upper bound on sample complexity, reduced to semialgebraic decision problem, studied Hausdorff geometry of real algebraic varieties.
result Upper bound on sample complexity for testing variety hypothesis.
New error bounds for flow matching methods using deterministic sampling.
problem Improving the accuracy of flow matching methods for generating probability distributions.
method Derived error bounds for flow matching methods under deterministic sampling conditions.
result Presented error bounds for flow matching methods using L 2 L^2 L 2 loss and regularity conditions. A new method estimates rare events using tensor trains.
problem Estimating rare event probabilities in high-dimensional problems.
method Approximating optimal importance distribution via tensor-train decompositions and compositions.
result Better variance reduction and efficient computation of rare event probabilities.
Permutation invariant network learns Wasserstein metrics.
problem Understanding the space of probability measures and comparing distributions.
method Permutation invariant network mapping samples to a low-dimensional space.
result Network can generalize to compute distances between unseen densities and learn moments.
Link prediction in networks is typically accomplished by estimating or ranking the probabilities of edges for all pairs of nodes. In practice, especially for social networks, the data are often collected by egocentric sampling, which means selecting a subset of nodes and recording all of their edges. This sampling mech…
The paper defines conditions for Gaussian process sample path regularity.
problem Lack of understanding of Gaussian process sample path regularity.
method Analyzes covariance kernels to determine sample path regularity.
result Necessary and sufficient conditions for Hölder regularity are provided.
Flow Matching enables robust training of CNFs with various probability paths.
problem Training Continuous Normalizing Flows (CNFs) at large scales.
method Flow Matching (FM) is a simulation-free approach for training CNFs by regressing vector fields of conditional probability paths.
result Flow Matching with diffusion paths yields more robust and stable training compared to diffusion-based methods.