Unified analysis for deterministic samplers in diffusion models.
problem Challenges in analyzing deterministic samplers for diffusion models.
method Unified convergence analysis framework.
result Achieved polynomial iteration complexity for DDIM-type samplers.
Develops new bounds for deterministic samplers in diffusion models.
problem Analyzing deterministic samplers in diffusion generative models.
method Operational interpretation of deterministic sampling; restoration and degradation steps.
result First polynomial convergence bounds for DDIM-type samplers.
We convert deterministic flow models to stochastic samplers.
problem Deterministic flow models are sensitive to errors and cannot condition on intermediate states.
method Transform ODEs into SDEs with the same marginal distributions.
result Empirically outperforms deterministic samplers and controls generation diversity.
PDMP samplers improve Bayesian PDE coefficient inference.
problem Efficient Bayesian inference in non-linear inverse problems with expensive likelihoods.
method Piecewise deterministic Markov process (PDMP) with surrogate-assisted thinning.
result PDMP samplers achieve higher accuracy and efficiency than traditional methods.
New analysis improves convergence guarantees for diffusion-based samplers in Wasserstein distance.
problem Improving convergence guarantees for diffusion-based generative models.
method Simple framework to analyze discretization, initialization, and score estimation errors.
result First Wasserstein convergence bound for the Heun sampler and improved results for Euler sampler.
Paper accelerates diffusion models, improving sampling speed.
problem Low sampling speed in score-based diffusion models.
method Design of novel training-free algorithms for deterministic and stochastic samplers.
result Accelerated samplers converge faster with improved rates.
One-step diffusion samplers reduce sampling time and computational costs.
problem Efficient sampling from complex distributions.
method One-step diffusion, self-distillation, deterministic flow.
result Achieves competitive sample quality with fewer evaluations.
New PDMP samplers improve BNN inference with accelerated computation.
problem Inference on Bayesian Neural Networks violates independence and posterior assumptions.
method Piecewise Deterministic Markov Process (PDMP) with adaptive thinning for inhomogenous Poisson Process (IPPs) sampling.
result PDMP samplers accelerate inference in BNNs, improving accuracy and mixing performance.
A new method de-randomizes MCMC dynamics using the Stein operator.
problem Estimating complex target distributions in Bayesian inference.
method De-randomized kernel-based particle samplers that discretize the fiber-gradient Hamiltonian flow.
result GSVGD de-randomizes complex MCMC dynamics, maintaining high sample quality.
New PDMP samplers tackle variable selection in models.
problem Jointly explore model space and parameter space.
method Develop reversible jump PDMP samplers.
result New samplers mix better and are more efficient.
Enhanced Markov chain sampler learns network statistics faster.
problem Learning network statistics efficiently.
method Integrates graph Forman curvature into Markov chain transition probabilities and stationary distribution.
result Curved Markov chain Monte Carlo achieves faster convergence.
Improved PDMP samplers for multi-modal distributions using tempering.
problem Struggles of PDMP samplers with multi-modal or heavy-tailed distributions.
method Tempering PDMPs by interpolating between a tractable and posterior distribution.
result PDMP samplers can be improved to sample from multi-modal distributions.
FM4PDE learns PDE solutions from sparse data.
problem Reconstructing PDE solutions from limited observations.
method Flow-matching generative framework that learns PDE coefficients and solutions.
result Error guarantees for guided procedures, including deterministic and stochastic samplers.
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
LFIS uses a time-dependent velocity field to sample from complex distributions.
problem Sampling from unnormalized density functions.
method LFIS learns a time-dependent velocity field to transport samples from a simple initial distribution to a complex target distribution.
result LFIS achieves state-of-the-art performance on various benchmark problems.
This work establishes near-minimax optimal guarantees for ODE-based samplers under mild assumptions.
problem Develop rigorous statistical guarantees for ODE-based samplers in generative modeling.
method Proposes a smooth regularized score estimator and refined convergence analysis.
result Achieves minimax rate in total variation distance for ODE-based samplers under mild assumptions.
New method uses PDMPs with sub-sampling for efficient sampling from posterior distributions.
problem Efficient sampling from posterior distributions with limited data access.
method Approximate simulation of PDMPs with sub-sampling and stochastic gradient estimation.
result Stochastic-gradient PDMPs are efficient and robust compared to Langevin dynamics.
New algorithm speeds up sampling from complex distributions.
problem Efficiently sampling from non-log-concave distributions.
method Stochastic Proximal Samplers (SPS) based on SGLD and MALA.
result SPS-SGLD and SPS-MALA achieve faster sampling with reduced gradient complexity.
We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis-Hastings algorithms using random proposals and m…
Generative models using PDMPs with explicit jump rates and kernels.
problem Creating efficient generative models for complex data distributions.
method Piecewise deterministic Markov processes (PDMPs) with explicit expressions for jump rates and kernels.
result Efficient training and simulation methods for PDMP-based generative models.
New theory improves diffusion models' convergence rates.
problem Understanding and optimizing diffusion models for faster data generation.
method Developed non-asymptotic theory for diffusion models with minimal assumptions.
result Established convergence rates for two diffusion models.
The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an O(1/T) convergence rate for models with independent variables and for ful…
We study the convergence rate of stochastic optimization of exact (NP-hard) objectives, for which only biased estimates of the gradient are available. We motivate this problem in the context of learning the structure and parameters of Ising models. We first provide a convergence-rate analysis of deterministic errors fo…
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…
Generative sampler learns velocity fields for efficient posterior inference.
problem Sampling from complex posterior distributions in high dimensions.
method Generative multivariate posterior sampler via flow matching, learning a velocity field for a deterministic transport map.
result Conditional Brenier map enables fast generation of credible sets with theoretical consistency guarantees.
New method uses zeroth-order queries to approximate proximal sampling efficiently.
problem Approximating proximal sampling with zeroth-order information.
method Direct simulation of heat flow dynamics, treating intermediate distribution as Gaussian mixture.
result Inherits exponential convergence under isoperimetric conditions, avoids rejection sampling.
Develops a robust method for image reconstruction from limited data.
problem Inference of unknown images from few measurements, often ill-posed.
method Introduces DPnP, a diffusion plug-and-play method combining likelihood and score-based samplers.
result Establishes performance guarantees for DPnP, demonstrating robustness and efficiency.
Generative Bayesian Inference uses GANs for approximate posterior sampling.
problem Bayesian inference without explicit likelihoods.
method Develops Bayesian GAN (B-GAN) for posterior simulation.
result B-GAN achieves highly competitive performance in posterior sampling.
We extend probabilistic programming to handle conditioning on marginal distributions.
problem Conditioning probabilistic programs on marginal distributions of observable variables.
method We define and implement stochastic conditioning, allowing inference in probabilistic programs conditioned on marginal distributions.
result We demonstrate the effectiveness of stochastic conditioning in various real-life scenarios.
Supervised deep-embedding methods project inputs of a domain to a representational space in which same-class instances lie near one another and different-class instances lie far apart. We propose a probabilistic method that treats embeddings as random variables. Extending a state-of-the-art deterministic method, Protot…
Sparse representations have proven their efficiency in solving a wide class of inverse problems encountered in signal and image processing. Conversely, enforcing the information to be spread uniformly over representation coefficients exhibits relevant properties in various applications such as digital communications. A…
Corrected samplers reduce discretization error in discrete flow models without additional computational cost.
problem Discretization error in samplers for discrete flow models.
method Established non-asymptotic error bounds for samplers, proposed time-corrected and location-corrected samplers.
result Location-corrected sampler has lower complexity and better generation quality.
NEO combines orbits to sample and estimate complex distributions.
problem Sampling and normalizing complex distributions.
method NEO-IS and NEO-MCMC using invertible maps and orbits.
result NEO provides unbiased estimators and explores multimodal targets.
New samplers improve MCMC efficiency in high dimensions.
problem Efficient sampling in high-dimensional problems.
method Affine invariant ensemble samplers, including derivative-free and derivative-based HMC.
result Affine invariant ensemble HMC outperforms standard HMC in high dimensions.
New study shows Gaussian samplers struggle with heavy-tailed targets, while stable samplers excel.
problem The difficulty of sampling from heavy-tailed distributions using Gaussian versus stable oracles.
method Comparison of Gaussian and stable oracles for proximal samplers.
result Gaussian samplers have a fundamental barrier for high-accuracy guarantees in heavy-tailed sampling, while stable samplers excel.
SRO optimizes decisions against worst-case sampler induced by generative models.
problem Operational uncertainty shifts from explicit probability law to sampler induced by learned generators.
method SRO optimizes decisions against the worst-case sampler induced by perturbing the learned generator.
result Empirical worst-case objective provides high-probability upper certificate for true population objective.
The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Mark…
Efficient methods accelerate diffusion model sampling.
problem Slow sample generation in diffusion models.
method Conjugate Integrators and Splitting Integrators.
result Hybrid method achieves best FID scores.
Discrete diffusion samplers improve sampling from unnormalised densities.
problem Sampling from discrete unnormalised densities efficiently.
method Introduce off-policy training techniques and data-to-energy Schrödinger bridge training for discrete diffusion samplers.
result Improved performance on synthetic and new benchmarks.
To improve the efficiency of Monte Carlo estimation, practitioners are turning to biased Markov chain Monte Carlo procedures that trade off asymptotic exactness for computational speed. The reasoning is sound: a reduction in variance due to more rapid sampling can outweigh the bias introduced. However, the inexactness …
Optimization can learn Johnson-Lindenstrauss embeddings without randomization.
problem Achieving compact data representations with theoretical guarantees.
method A novel optimization-based approach over the space of random solution samplers.
result The method avoids bad stationary points and converges to a deterministic solution.
Fastens diffusion model sampling by distillation.
problem Slow sampling time of diffusion models.
method New parameterizations and progressive distillation.
result Models can be distilled to take half as many sampling steps.
This paper analyzes MaskGIT sampler and introduces a moment sampler for faster masked diffusion sampling.
problem Efficiently sampling from masked diffusion models.
method Theoretical analysis of MaskGIT sampler, introduction of moment sampler, and two innovations for improving choose-then-sample efficiency.
result The moment sampler is an asymptotically equivalent, more interpretable alternative to MaskGIT.
PTSD improves neural samplers by combining diffusion models and PT, enhancing efficiency.
problem Efficiency and correlation issues in neural samplers compared to PT.
method Sequential training of diffusion models across temperatures, combining high-temperature models for approximate lower-temperature samples.
result Significantly improved target evaluation efficiency, outperforming diffusion-based samplers.
This paper introduces a neural sampler for scalable sampling from complex distributions.
problem Efficiently sampling from high-dimensional un-normalized distributions.
method Neural implicit sampler trained with KL and Fisher divergence methods.
result The neural sampler generates large batches of samples with low computational costs.
Two parallel samplers enhance image quality in limited denoising steps.
problem Limited denoising steps in diffusion models reduce image quality.
method Two parallel samplers denoise at successive times, integrating their information.
result Two parallel samplers improve image quality compared to a single sampler.
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…
The Bouncy Particle Sampler is a novel rejection-free non-reversible sampler for differentiable probability distributions over continuous variables. We generalize the algorithm to piecewise differentiable distributions and apply it to generic binary distributions using a piecewise differentiable augmentation. We illust…