Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in W 2 W_2 W 2 distance for log-strongly-concave targets. Accelerates TPP sampling with speculative decoding for faster sequence generation.
problem Efficiently sampling from complex temporal point processes.
method Adapting speculative decoding techniques from language models to TPPs.
result Achieves significant speedup (2-6x) while maintaining distributional accuracy.
Nested Slice Sampling accelerates Nested Sampling for GPU acceleration.
problem Challenging inference for complex, multimodal targets.
method Vectorized Nested Slice Sampling using Hit-and-Run Slice Sampling.
result NSS maintains accurate evidence estimates and high-quality posterior samples, robust on multimodal problems.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
Variance reduction is a simple and effective technique that accelerates convex (or non-convex) stochastic optimization. Among existing variance reduction methods, SVRG and SAGA adopt unbiased gradient estimators and are the most popular variance reduction methods in recent years. Although various accelerated variants o…
Paper accelerates diffusion models, improving sampling speed.
problem Low sampling speed in score-based diffusion models.
method Design of novel training-free algorithms for deterministic and stochastic samplers.
result Accelerated samplers converge faster with improved rates.
Stochastic gradient decent~(SGD) and its variants, including some accelerated variants, have become popular for training in machine learning. However, in all existing SGD and its variants, the sample size in each iteration~(epoch) of training is the same as the size of the full training set. In this paper, we propose a…
A new method accelerates deep neural network training using minimal margin score.
problem Training deep neural networks is computationally expensive.
method Introduces minimal margin score (MMS) for selecting samples.
result Significant acceleration in training deep neural networks.
Accelerated RPCholesky speeds up kernel matrix approximations.
problem Efficiently approximating large kernel matrices.
method Accelerated randomly pivoted Cholesky (RPCholesky) with block matrix computations and rejection sampling.
result Approximates kernel matrices up to 40 times faster.
ES improves training efficiency by dynamically selecting data samples.
problem Efficiently selecting informative data samples for faster learning.
method Evolved Sampling (ES) dynamically selects data samples based on loss dynamics and differences.
result ES achieves significant training acceleration without compromising model performance.
LA-VDM accelerates VDM using landmarks to improve data analysis.
problem Efficiently analyzing complex datasets with nonuniform sampling densities.
method Landmark-constrained two-stage normalization to accelerate VDM.
result LA-VDM accurately recovers parallel transport and converges to the connection Laplacian.
GPU-accelerated particle methods outperform neural samplers in LFT benchmarks.
problem High-dimensional multimodal sampling problems in lattice field theory.
method GPU-accelerated particle Monte Carlo methods (Sequential Monte Carlo and nested sampling).
result These methods match or outperform neural samplers in sample quality and wall-clock time.
New method accelerates diffusion models for broader target distributions.
problem Current diffusion models have limited acceleration for certain target distributions.
method Developed a novel accelerated stochastic DDPM sampler.
result Achieved accelerated performance for three broad distribution classes.
Accelerates MCMC sampling for large-scale problems using machine learning.
problem Efficiently sampling large-scale Bayesian inference problems with high computational cost.
method Integrates low-fidelity machine learning models into a multilevel MCMC framework.
result Significantly accelerates multilevel sampling by a factor of two with similar accuracy.
New method uses birth-death process and exploration component to accelerate sampling from multimodal distributions.
problem Sampling from multimodal probability distributions efficiently.
method Combines birth-death process and exploration component to accelerate sampling.
result Proves exponential asymptotic convergence under mild assumptions.
ASVGD accelerates SVGD for efficient sampling from Gaussian targets.
problem Efficient sampling from Gaussian distributions using SVGD.
method Accelerated gradient flow in a metric space of probability densities, including momentum and Wasserstein regularization.
result ASVGD achieves optimal convergence rate for Gaussian targets, independent of covariance.
Unified method for MMD variance estimation improves accuracy and computational efficiency.
problem Variance estimation for MMD in nonparametric testing.
method Unified finite-sample characterization of MMD variance through U-statistic and Hoeffding decomposition; exact acceleration method for univariate case.
result Unified estimators improve accuracy and computational efficiency for MMD variance.
A new method for generating synthetic data using posterior distribution learning accelerates inference.
problem Generating high-quality synthetic data requires many discretization steps, which is computationally expensive.
method Learning the posterior distribution of clean data samples given noisy versions, using a scoring rule instead of regression loss.
result Consistently outperforms standard diffusion models at few discretization steps.
SUOD accelerates OD for large, diverse models.
problem Training and scoring new samples with many unsupervised, heterogeneous OD models.
method Data reduction, model approximation, and taskload optimization.
result SUOD accelerates OD for over 20 benchmark datasets and a real-world case.
Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
We present a framework for Nesterov's accelerated gradient flows in probability space to design efficient mean-field Markov chain Monte Carlo (MCMC) algorithms for Bayesian inverse problems. Here four examples of information metrics are considered, including Fisher-Rao metric, Wasserstein-2 metric, Kalman-Wasserstein m…
BONAS accelerates NAS while maintaining reliability.
problem Computational inefficiency in sample-based NAS.
method Bayesian Optimized Neural Architecture Search (BONAS) using weight-sharing.
result BONAS accelerates sample-based NAS significantly while maintaining reliability.
Accelerated coordinate descent is widely used in optimization due to its cheap per-iteration cost and scalability to large-scale problems. Up to a primal-dual transformation, it is also the same as accelerated stochastic gradient descent that is one of the central methods used in machine learning. In this paper, we imp…
New method accelerates Bayesian imaging using Langevin sampling.
problem Bayesian inference in imaging inverse problems with convex geometry.
method Stochastic relaxed proximal-point iteration targeting posterior distribution.
result Accelerated convergence for κ κ κ -strongly log-concave targets. PNDMs accelerate DDPMs by treating them as differential equations on manifolds.
problem Accelerate DDPMs while maintaining sample quality.
method Propose pseudo numerical methods (PNDMs) to solve differential equations on manifolds.
result PNDMs generate higher quality images with only 50 steps compared to 1000-step DDIMs (20x speedup).
Timewarp accelerates molecular dynamics by learning to simulate long timescales.
problem Efficiently simulating long timescales in molecular dynamics.
method Uses a normalizing flow to learn large time steps in Markov chain Monte Carlo.
result Generalizes to unseen small peptides, accelerating sampling.
New method speeds up diffusion models without requiring complex assumptions.
problem Slow sampling in diffusion models due to high computational cost.
method Training-free acceleration scheme under minimal assumptions.
result Provable acceleration within O ~ ( d 5 / 4 / ε ) \widetilde{O}(d^{5/4}/\sqrt{\varepsilon}) O ( d 5/4 / ε ) iterations. Unpaired deep learning reconstructs MRI images from accelerated data.
problem Difficulty in acquiring matched fully sampled k-space data for supervised deep learning.
method Optimal transport driven cycleGAN architecture.
result Reconstructs high resolution MR images from accelerated k-space data.
GADD accelerates uniform-rate discrete diffusion models by 2 orders of magnitude.
problem Slow sampling in uniform-rate discrete diffusion models.
method Gibbs-based corrector (GADD) that constructs Gibbs posterior likelihoods directly from the concrete score function.
result Achieves an overall sampling complexity of O ( p o l y l o g ( ε − 1 ) ) \mathcal{O}(\mathrm{polylog} (\varepsilon^{-1})) O ( polylog ( ε − 1 )) . New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanis…
Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to …
New algorithm speeds up diffusion model sampling 4-14 times.
problem Time-consuming sampling from diffusion models.
method Parallelizing autoregressive process through fixed-point iteration.
result ParaTAA reduces inference steps by 4-14 times.
New method accelerates Parallel Tempering using neural samplers.
problem Challenges in sampling from high-dimensional, multimodal distributions.
method Leverages neural samplers to reduce overlap between distributions.
result Improves sample quality and reduces computational cost.
This work accelerates constrained sampling using large deviation principles.
problem Sampling constrained probability distributions efficiently.
method Large deviation principles applied to skew-reflected non-reversible Langevin dynamics.
result The skew-symmetric matrix accelerates convergence and reduces asymptotic variance.
Faster WIND accelerates iterative BOND for LLM alignment.
problem Iterative BOND is inefficient in practice due to sample and computation inefficiency.
method Unified game-theoretic connection to self-play alignment, WIND framework with efficient algorithms.
result WIND variant achieves superior sample efficiency and faster computation.
This paper analyzes speculative decoding, a method to speed up large language model inferences.
problem Theoretical understanding of speculative decoding is lacking.
method Conceptualizes speculative decoding as a markov chain problem and studies its key properties.
result Reveals fundamental connections between LLM components and their impact on decoding efficiency.
A new gradient tree boosting framework reduces variance and accelerates performance.
problem High variance in stochastic gradient boosting.
method Combining gradient tree boosting with importance sampling and a regularizer.
result Achieves a linear convergence rate on logistic loss and 2.5x--18x acceleration on LogitBoost and LambdaMART.
Dropout is a simple but efficient regularization technique for achieving better generalization of deep neural networks (DNNs); hence it is widely used in tasks based on DNNs. During training, dropout randomly discards a portion of the neurons to avoid overfitting. This paper presents an enhanced dropout technique, whic…
DreamPropeller accelerates text-to-3D generation by 4.7x with minimal loss in quality.
problem Long generation times in text-to-3D generation algorithms degrade user experience.
method DreamPropeller uses Picard iterations generalized for non-ODE paths to accelerate parallel sampling.
result Empirically achieves up to 4.7x speedup with negligible quality loss.
We propose a novel method to accelerate Lloyd's algorithm for K-Means clustering. Unlike previous acceleration approaches that reduce computational cost per iterations or improve initialization, our approach is focused on reducing the number of iterations required for convergence. This is achieved by treating the assig…
Principal component analysis (PCA) is one of the most powerful tools in machine learning. The simplest method for PCA, the power iteration, requires O ( 1 / Δ ) \mathcal O(1/Δ) O ( 1/Δ ) full-data passes to recover the principal component of a matrix with eigen-gap Δ Δ Δ . Lanczos, a significantly more complex method, achieves an accelerated…
Efficient methods accelerate diffusion model sampling.
problem Slow sample generation in diffusion models.
method Conjugate Integrators and Splitting Integrators.
result Hybrid method achieves best FID scores.
HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.
problem Optimizing functions efficiently and accelerating convergence rates.
method Randomized Hamiltonian flow (RHF) with accelerated convergence rates.
result RHGD achieves accelerated convergence rates similar to Nesterov's AGD.
WSD uses a deterministic model to accelerate diffusion-based sampling.
problem Slow refinement process in diffusion models.
method Warm-start model that predicts an informed prior conditioned on input context.
result Significantly reduces the number of diffusion steps required for realistic samples.
This work sets lower bounds on the number of score queries needed for diffusion sampling.
problem Establishing information-theoretic limits on the number of score evaluations required for diffusion sampling.
method Proving lower bounds on the number of adaptive score queries needed for sampling.
result Any sampling algorithm requires at least \(\widetilde{\Omega}(\sqrt{d})\) adaptive score queries for \(d\)-dimensional distributions.
New study on No-U-Turn Sampler for accelerated mixing in Hamiltonian Monte Carlo.
problem Achieving accelerated convergence in Hamiltonian Monte Carlo.
method Combining concentration of measure and coupling analysis for mixing.
result Rigorous mixing guarantees for the No-U-Turn Sampler in certain Gaussian distributions.