New particle-based VI algorithm expands function class and improves scalability.
problem Limited function class in particle-based VI algorithms restricts flexibility and scalability.
method Introduces a functional regularization term to expand the function class and proposes PFG algorithm.
result Proposed PFG algorithm has larger function class, improved scalability, better adaptation to ill-conditioned distributions, and provable convergence.
New coin sampling method for Bayesian inference without learning rates.
problem Scalable Bayesian inference with learning rate tuning issues.
method Coin sampling for gradient-based Bayesian inference.
result Comparable performance to other ParVI algorithms without learning rate tuning.
PAPAL algorithm finds mixed Nash equilibria in continuous games.
problem Finding mixed Nash equilibria in non-convex, non-concave games.
method Particle-based Primal-Dual Algorithm (PAPAL) for weakly entropy-regularized min-max optimization.
result PAPAL offers non-asymptotic convergence guarantees for ε-mixed Nash equilibrium. BSVGD improves sampling for multimodal distributions using branching.
problem Sampling from multimodal distributions.
method Random branching in Stein Variational Gradient Descent (SVGD).
result Theoretical convergence guarantee and empirical validation.
Proposes PGPS for efficient Bayesian inference.
problem Efficient sampling from complex posterior distributions.
method Path-guided particle-based sampling with Log-weighted Shrinkage.
result PGPS generates samples closer to target distribution.
A new method solves high-dimensional MFGs using particle-based flow matching.
problem Solving high-dimensional Mean-Field Games (MFGs) is computationally challenging.
method Proposes a particle-based deep Flow Matching (FM) method to update particles and train a flow neural network.
result Proves convergence of the scheme to a stationary point sublinearly and linearly under convexity assumptions.
Optimal weights improve particle-based approximations of discrete distributions.
problem Improving particle-based approximations of discrete distributions.
method Proving optimality of weights and showing how to compute them efficiently.
result Optimal weights can be computed from existing particle-based methods without extra costs.
KSD Descent uses KSD to sample from a target distribution efficiently.
problem Sampling from complex target distributions efficiently.
method Wasserstein gradient flow of KSD, using L-BFGS optimization.
result KSD Descent can sample from a target distribution using a set of particles.
Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on app…
Electrostatics method samples complex distributions deterministically.
problem Sampling and inference of complex, high-dimensional distributions.
method Electrostatics-based particle system with Newton mechanics principles.
result Method achieves comparable performance to other methods in benchmark tasks.
We propose a general statistical framework for clustering multiple time series that exhibit nonlinear dynamics into an a-priori-unknown number of sub-groups. Our motivation comes from neuroscience, where an important problem is to identify, within a large assembly of neurons, subsets that respond similarly to a stimulu…
New optimization method for sampling from unknown density measures.
problem Sampling from measures with unknown normalization constants.
method Mollified Interaction Energy Descent (MIED) method.
result Gradient flow of MIE converges to chi-square divergence.
MWGraD solves multi-objective distributional optimization using particle-based gradient descent.
problem Simultaneously minimize multiple objective functionals over probability distributions.
method Iterative particle-based algorithm MWGraD, estimating and aggregating Wasserstein gradients.
result Demonstrates effectiveness on synthetic and real-world datasets.
A new EVI framework improves ParVI methods by maintaining variational structure and reducing KL-divergence.
problem Improving variational inference methods for better approximation of target distributions.
method EVI framework that minimizes the VI objective function based on an energy-dissipation law, including a new 'Approximation-then-Variation' scheme.
result The new scheme significantly decreases KL-divergence and outperforms existing ParVI methods in fidelity.
A new ParVI framework improves particle-based variational inference methods.
problem Non-trivial kernel design in particle-based variational inference methods.
method Proposes a generalized Wasserstein gradient descent (GWG) framework with broader regularizers.
result Demonstrates strong convergence guarantees and effectiveness on simulated and real data.
Paper proposes a new method for sampling from complex distributions.
problem Sampling from unnormalised density functions in complex distributions.
method Combines amortised and particle-based methods with reinforcement learning.
result Improves sampling from complex distributions compared to existing methods.
Particle-based variational inference methods (ParVIs) have gained attention in the Bayesian inference literature, for their capacity to yield flexible and accurate approximations. We explore ParVIs from the perspective of Wasserstein gradient flows, and make both theoretical and practical contributions. We unify variou…
Gaussian-SVGD dynamics converge to Gaussian distributions under certain conditions.
problem Understanding the theoretical properties of SVGD, especially for Gaussian targets.
method Detailed theoretical study of Gaussian-SVGD dynamics, considering both mean-field PDE and discrete particle systems.
result Gaussian-SVGD dynamics converge linearly to the Gaussian distribution closest to the target in KL divergence.
DriftLite improves inference quality of diffusion models without retraining.
problem Adapting pre-trained diffusion models to new target distributions without retraining.
method Lightweight, training-free particle-based approach that steers inference dynamics with optimal stability control.
result Consistently reduces variance and improves sample quality over existing methods.
New method accelerates energetic variational inference using particle dynamics.
problem Efficiently solving variational inference problems with reduced computational cost.
method Particle-based variational inference with implicit scheme, inspired by energy quadratization and operator splitting.
result Significantly reduces computational cost compared to existing methods.
Improves SVGD for high-dimensional Bayesian inference by reducing variance collapse.
problem Variance collapse in SVGD reduces accuracy and diversity of estimation.
method Augmented Message Passing SVGD (AUMP-SVGD) method, a two-stage optimization procedure.
result AUMP-SVGD achieves satisfactory accuracy and overcomes variance collapse in various benchmark problems.
New algorithm trains latent diffusion models using interacting particles.
problem Training latent diffusion models efficiently and accurately.
method Reformulate training as minimizing a free energy functional, then approximate with interacting particles.
result The new algorithm outperforms previous methods in experiments.
New algorithms for sampling in constrained domains without learning rates.
problem Sampling in constrained domains with fairness constraints and post-selection inference.
method Coin betting ideas from convex optimisation and a unifying framework for constrained sampling.
result Our algorithms achieve competitive performance without hyperparameter tuning.
Bayesian Non-negative Matrix Factorization (NMF) is a promising approach for understanding uncertainty and structure in matrix data. However, a large volume of applied work optimizes traditional non-Bayesian NMF objectives that fail to provide a principled understanding of the non-identifiability inherent in NMF-- an i…
Generative algorithms learn high-dimensional data efficiently and generate new samples.
problem Learning from scarce high-dimensional data.
method Lipschitz-regularized gradient flows and particle-based algorithms.
result Correctly transports gene expression data points with high dimensionality.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the po…
Paper proposes LSVGD to stabilize GAN training via Langevin Stein Variational Gradient Descent.
problem Mode collapse and performance deterioration in GAN training.
method Langevin Stein Variational Gradient Descent (LSVGD) incorporating noise to stabilize training.
result LSVGD improves performance and stability of various GAN models.
New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
Generative ParVI learns flexible sampling from posterior distributions.
problem Avoiding arbitrary parametric assumptions in variational inference.
method Neural sampler trained with functional gradient of KL-divergence.
result GPVI outperforms previous generative ParVI methods and is competitive with other approaches.
New method for constrained sampling using gradient flows.
problem Sampling from constrained domains.
method Introducing a boundary condition for gradient flow to confine particles within the domain.
result Provable continuous-time convergence in total variation for constrained sampling.
SIFG uses noisy particles to efficiently sample from complex distributions.
problem Efficient sampling from complex distributions using particle-based methods.
method SIFG introduces a semi-implicit functional gradient flow with Gaussian noise to improve sampling efficiency and accuracy.
result SIFG achieves strong theoretical convergence guarantees and efficient sampling.
A new algorithm for optimizing probability distributions converges linearly.
problem Optimizing functionals over families of probability distributions.
method Variational transport: particle-based algorithm approximating Wasserstein gradient descent.
result Variational transport converges linearly to the global minimum of the objective functional.
State space models (SSM) have been widely applied for the analysis and visualization of large sequential datasets. Sequential Monte Carlo (SMC) is a very popular particle-based method to sample latent states from intractable posteriors. However, SSM is significantly influenced by the choice of the proposal. Recently Ha…
NVGD uses neural networks to infer distributions without kernel choices.
problem Challenges in choosing kernel functions for SVGD.
method NVGD parameterizes the witness function of the Stein discrepancy with a neural network.
result NVGD achieves good performance on various inference problems.
Proposes optimizing neural network ensemble diversity in feature space.
problem Improving diversity in neural network ensembles while maintaining performance.
method Optimizes particles in the feature space of a specific intermediate layer.
result Significantly outperforms Deep Ensembles on various metrics.
Deep density methods improve filtering in high-dimensional systems.
problem Nonlinear filtering in high-dimensional systems.
method Two deep density methods based on Feynman-Kac formulas and neural networks.
result Logarithmic deep backward stochastic differential equation filter outperforms classical methods in high dimensions.
Stein variational neural network ensembles improve diversity and uncertainty estimation.
problem Lack of proper Bayesian justification and diversity guarantees in deep neural network ensembles.
method Particle-based inference methods, specifically Stein variational gradient descent (SVGD), operating in weight space, function space, and hybrid settings.
result SVGD methods improve diversity and uncertainty estimation, approaching the true Bayesian posterior more closely.
Approach selects variables and time intervals for comparing high-dimensional time-series data.
problem Comparing high-dimensional time-series data for significant differences.
method Data is split into subintervals, and two-sample tests are performed on each to identify distinguishing variables.
result The approach effectively identifies variables and time intervals where data significantly differs.
Approximate inference in high-dimensional, discrete probabilistic models is a central problem in computational statistics and machine learning. This paper describes discrete particle variational inference (DPVI), a new approach that combines key strengths of Monte Carlo, variational and search-based techniques. DPVI is…
New gradient flows improve high-dimensional sampling.
problem Sampling from high-dimensional target densities.
method Introducing Radon--Wasserstein gradient flows.
result Linear scaling in particles and dimensions.
Paper optimizes training data distribution for better model performance across various deployment conditions.
problem Improving model accuracy when deployed with parameters far from training data.
method Developed adaptive algorithms based on bilevel or alternating optimization in the space of probability measures.
result Optimized training distributions lead to models with improved sample complexity and robustness to distribution shift.
We make posterior sampling in FWI feasible for large surveys.
problem Uncertainty-aware subsurface models at field scale.
method Coupling diffusion-based posterior sampling with simultaneous-source FWI data.
result Lower model error and better data fit at reduced computational cost.
New algorithms learn latent variable models without tuning, outperforming existing methods.
problem Learning latent variable models without manual tuning.
method Two particle-based algorithms using free energy minimization and coin betting.
result Learning algorithms are entirely tuning-free and competitive with existing methods.
PFPN uses particle filtering to improve character control in physics-based simulations.
problem Premature commitment to suboptimal actions in high-dimensional continuous control problems for articulated characters.
method Proposes a particle-based action policy using particle filtering to dynamically explore and discretize the action space.
result Demonstrates better imitation performance and robustness to external perturbations compared to Gaussian policies.
PASOA optimizes Bayesian design by improving SMC samplers and EIG.
problem Sequential design optimization for accurate parameter inference.
method Sequential optimization using contrastive estimation, SMC samplers, and tempering.
result PASOA optimizes design and inference with improved consistency.
New method optimizes hyperparameters for randomized algorithms like random feature regression.
problem Optimizing hyperparameters in randomized algorithms is challenging due to their stochastic nature.
method Introduced a random objective function and used ensemble Kalman inversion (EKI) for gradient-free optimization.
result Demonstrated successful optimization of hyperparameters in various randomized algorithms.
Dual-space sampling tackles ill-conditioned inverse problems with Bayesian methods.
problem Bayesian inference in constrained inverse problems with ill-conditioned solutions.
method Dual-space posterior sampling using ADMM and SVGD.
result Well-calibrated uncertainty estimates and posterior contraction with increasing data.