Study improves estimation of rare language model outputs.
problem Estimating probabilities of rare outputs in language models.
method Importance sampling vs. activation extrapolation for low probability estimation.
result Importance sampling outperforms activation extrapolation.
Paper develops efficient method for probability estimation.
problem Estimating probabilities with high efficiency.
method Adaptive Monte Carlo estimation using truncated inverse binomial sampling.
result Proposed method is orders of magnitude more efficient.
Adaptive sampling framework for varying probabilities.
problem Choosing good sampling probabilities for tasks and streams.
method Derives a general framework for adaptively changing sampling probabilities via thresholds.
result Shows when adaptive thresholds can be treated as independent samples.
New algorithms estimate and test collision probability with near-optimal sample complexity.
problem Estimating and testing collision probability in discrete distributions.
method Developed algorithms for ( α , β ) (α, β) ( α , β ) -local differential privacy and sequential testing. result Achieved nearly optimal sample complexity for estimating and testing collision probability.
Proposes a neural network method to combine nonprobability and probability survey samples.
problem Combining nonprobability and probability survey samples for accurate population mean estimation.
method Uses a deep neural network to estimate sampling scores from nonprobability samples and combines them with probability sample information.
result Proposed estimators improve robustness to parametric propensity-score misspecification, especially for nonlinear selection mechanisms.
DPPy offers Python tools for sampling DPPs.
problem Sampling from DPPs is challenging.
method Gathers exact and approximate sampling algorithms for finite and continuous DPPs.
result DPPy provides Python tools for DPP sampling.
New methods for estimating causal effects with limited overlap, using Stable Probability Weighting.
problem Estimating causal effects with limited overlap in multivalued treatments.
method Stable Probability Weighting (SPW) and Finite-Sample Stable Probability Weighting (FPW) methods.
result SPW and FPW provide practical solutions for estimating and inferring causal effects with limited overlap.
The paper examines how sampling data affects the performance of submodular maximization.
problem Performance loss due to probability sampling in submodular maximization.
method Examines a simple probability sampling method where each data point is selected with probability at least r.
result The sampling gap is both upper and lower bounded by 1/r for policywise submodular utility functions.
We develop a new method to estimate failure probabilities in complex systems.
problem Estimating failure probabilities in safety-critical autonomous systems is challenging due to the rarity of failures and large state spaces.
method We propose an adaptive importance sampling algorithm that minimizes forward Kullback-Leibler divergence and uses Markov score ascent methods.
result Our method provides more accurate failure probability estimates than existing techniques.
NOFIS uses normalizing flows to estimate rare event probabilities more efficiently.
problem Accurate estimation of rare event probabilities using conventional methods is inefficient and resource-intensive.
method NOFIS learns a sequence of proposal distributions by minimizing KL divergence losses and estimates rare event probability using importance sampling.
result NOFIS outperforms baseline approaches in estimating rare event probabilities across 10 distinct test cases.
New method estimates and samples high-dimensional probability distributions avoiding optimization and approximation curse.
problem Estimating high-dimensional probability distributions from data samples.
method Hierarchic probability flow from coarse to fine scales, defined by conditional probabilities across scales.
result Sampling hierarchic models avoids critical slowing down at phase transitions and generates turbulence and dark matter images.
Improved matrix completion for non-uniformly sampled data.
problem Estimating unobserved entries in a matrix with varying sampling probabilities.
method Developed entry-specific bounds for low-rank matrix completion under structured non-uniform sampling.
result Error bounds for each entry match minimax lower bounds under certain conditions.
The study examines how shallow neural nets converge to training samples or manifold points during diffusion.
problem Understanding when and how shallow neural nets converge to training samples or manifold points during diffusion.
method Analysis of shallow ReLU neural network denoisers trained with minimal ℓ 2 \ell^2 ℓ 2 norm, comparing score flow and diffusion flow. result Probability flow converges to training points, sums of training points, or manifold points, depending on the diffusion time scheduler.
A nonparametric two-sample test using a parametric integral probability metric
problem Detecting distributional differences between two independent samples
method Propose a new two-sample test statistic based on a newly introduced integral probability metric (IPM)
result Establish theoretical guarantees for the associated two-sample testing procedure
This paper finds a unique partition of a sample space for estimating continuous distributions.
problem Estimating continuous probability distributions from finite samples.
method Equal-probability partition of the sample space using order statistics.
result The partition yields an entropy of log2(N+1) bits, providing a discrete entropy estimate.
Two BO methods improve reliability optimization for rare failures.
problem Maximizing reliability of designs subject to random perturbations.
method Bayesian optimization with Thompson sampling and knowledge gradient.
result Proposed methods outperform existing techniques in extreme failure probability scenarios.
Estimating IPM is as hard as estimating under IPM, both requiring similar optimal rates.
problem Estimating Integral Probability Metrics (IPMs) between probability measures.
method Study of minimax optimal rates for IPM estimation and under IPM estimation based on samples.
result Minimax optimal rates for estimating IPM and estimating under IPM are multiplicatively equivalent.
The paper defines projections for Wasserstein distances to preserve convex order in probability measures.
problem Designing sampling techniques to preserve convex order in probability measures.
method Defining projections for Wasserstein distances and solving optimization problems.
result The projections do not depend on ρ in dimension 1 and their quantile functions are explicit.
Optimal testing of discrete distributions with high probability, achieving sample complexity bounds.
problem Testing discrete distributions with high probability accuracy.
method Characterizing sample complexity as a function of parameters like δ, providing sample-optimal testers.
result Optimal algorithms for closeness and independence testing, achieving within constant factors of information-theoretic lower bounds.
New research shows some distributions hard to sample via diffusions.
problem Some distributions hard to sample via diffusions.
method Learning drifts of diffusions to approximate target distributions.
result Superpolynomially close drifts can yield very far approximations.
A new algorithm approximates logistic regression probabilities efficiently.
problem Efficiently approximating probabilities in logistic regression for large datasets.
method Randomized sampling-based algorithm with leverage scores.
result Accurate approximations to estimated probabilities with smaller sample sizes.
In this paper, we consider the problem of column subset selection. We present a novel analysis of the spectral norm reconstruction for a simple randomized algorithm and establish a new bound that depends explicitly on the sampling probabilities. The sampling dependent error bound (i) allows us to better understand the …
Study on estimating class probabilities using empirical risk minimization.
problem Estimating class probabilities within binary classification.
method Empirical risk minimization (ERM) for class probability estimation.
result The estimator converges to true class probabilities under certain conditions.
New MCMC method samples from lattice distributions efficiently.
problem Sampling from probability distributions on lattice structures.
method Metropolis-Hastings algorithm with a pull-back measure.
result The method is uniformly ergodic under certain conditions.
New Gibbs sampling method improves MCMC efficiency.
problem Improving efficiency of Gibbs sampling.
method Non-uniform random scan with selection probability optimization.
result Non-uniform scan improves mixing time of Markov chain.
Theoretical proof shows COMs are a type of contrastive divergence model with improved sampling.
problem Improving sampling quality in offline model-based optimization.
method Showed COMs are contrastive divergence models, proposed Langevin MCMC sampler, and decoupled model.
result Improved sampling quality achieved by decoupling model and using Langevin MCMC.
This work connects GANs and VAEs by computing sample likelihoods in GANs.
problem Lack of explicit probability models in GANs limits their use in statistical inference.
method Construct an explicit probability model for GANs, proving they maximize a variational lower-bound on average sample log likelihoods.
result Wasserstein GANs with entropy regularization can compute sample likelihoods, bridging GANs and VAEs.
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
Develops asymptotic analysis for RandNLA sampling estimators in least-squares problems.
problem Lack of distributional information for RandNLA estimators in statistical inference.
method Asymptotic analysis of sampling estimators for least-squares problems in two settings.
result Sampling estimators are asymptotically normally distributed under mild conditions.
The paper studies PCA of probability measures with varying sample sizes and finds optimal convergence rates.
problem PCA of multiple probability measures with varying sample sizes.
method Double asymptotic regime analysis with convergence rates n − 1 / 2 + m − α n^{-1/2} + m^{-α} n − 1/2 + m − α for empirical covariance and PCA risk. result Optimal convergence rates for empirical covariance and PCA risk in the dense regime are proven.
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
Improves FI-PINNs by combining re-sampling and subset simulation for better failure probability estimation.
problem Estimating failure probability in physics-informed neural networks (PINNs).
method Adaptive sampling with re-sampling and subset simulation, using cosine-annealing for uniform to adaptive transition.
result Significant improvement in estimating failure probability and generating new training points in the failure region.
A new method for efficient sampling from probability measures.
problem Efficiently sampling from complex probability distributions.
method RBM-SVGD, a stochastic version of SVGD using Random Batch Method.
result Reduces computational cost, especially for long-range kernels.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.
ES reduces high-probability regret in stochastic linear bandits.
problem High-probability regret in stochastic linear bandits.
method Linear ensemble sampling with standard Gaussian perturbations, analyzing m = Θ ( d log n ) m=Θ(d\log n) m = Θ ( d log n ) ensemble size. result ES achieves i l d e O ( d 3 / 2 n ) ilde O(d^{3/2}\sqrt n) i l d e O ( d 3/2 n ) high-probability regret, closing the gap to Thompson sampling. Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained d…
Study identifies partitions of probability distributions using multi-armed bandits.
problem Identifying the correct partition of a vector of probability distributions.
method Developed sample complexity bounds and proposed algorithms for multi-armed bandit settings.
result Characterized lower bounds on mean number of samples and proposed algorithms matching these bounds.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Study shows optimal rates for estimating Wasserstein metric and measures.
problem Minimax optimal estimation of Wasserstein metric between probability measures.
method Analyzes the minimax rates for estimating the Wasserstein-1 metric and probability measures.
result Minimax optimal rates for estimating Wasserstein metric and measures are multiplicatively equivalent.
Estimating the largest community in a mixed population via sequential sampling.
problem Identifying the largest community in a mixed population with limited sampling.
method Sequential, random sampling of individuals across multiple boxes, optimizing sampling strategy and decision rule.
result Proposed algorithms achieve optimal error probability decay rates under fixed budget constraints.
Paper tackles contaminated bandits, identifying best true distribution with high probability.
problem Identifying the best true distribution in contaminated bandits.
method Developed tight sample complexity bounds for estimating robust moments from contaminated samples, adapting classical algorithms.
result Derived matching information-theoretic lower bounds for sample complexity.
New sampling method uses stochastic interpolants and FBSDEs.
problem Sampling from high-dimensional distributions with unnormalized densities.
method Stochastic interpolants and FBSDEs to define and solve diffusion process.
result Effective sampling from challenging distributions.
Estimator calculates surface curvature from point cloud samples.
problem Accurately estimating curvature from limited point cloud data.
method Algorithm using probability distribution and nearby points control.
result Controlled number of points ensures accurate curvature estimation.
We consider the problem of low canonical polyadic (CP) rank tensor completion. A completion is a tensor whose entries agree with the observed entries and its rank matches the given CP rank. We analyze the manifold structure corresponding to the tensors with the given rank and define a set of polynomials based on the sa…
New methods ensure feature importance rankings are correct with high probability.
problem Stability issues in feature importance scores due to random sampling.
method Hypothesis testing-based techniques to assess and verify the stability of top-ranked features.
result Ensures the most important features are correct with high-probability guarantees.
Paper proposes neural network for arbitrary noise modeling.
problem Training deep learning algorithms with unknown noise distributions.
method Fully connected neural network to map uniform distribution to any known probability density function, minimizing Jensen-Shannon divergence.
result Model converges to desired noise distribution, providing an efficient alternative to existing sampling methods.
A new test improves statistical inference in bandit algorithms without sacrificing adaptiveness.
problem Challenges in statistical inference for adaptive randomised experiments in bandits.
method An allocation probability test for Thompson Sampling without trading-off regret or requiring large sample sizes.
result Improves statistical inference in small samples, showing advantages in mental health experiments.
We develop importance sampling based efficient simulation techniques for three commonly encountered rare event probabilities associated with random walks having i.i.d. regularly varying increments; namely, 1) the large deviation probabilities, 2) the level crossing probabilities, and 3) the level crossing probabilities…