We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines. This type of "embarrassingly parallel" procedure has recently been developed for MCMC inference al…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “VI algorithm”
Simulation-free VI closes the approximation gap in latent SDEs
problem Recovering dynamical systems from noisy observations
method Helmholtz-SDE
result Recovers dynamics more faithfully than prior methods
Extending Mean-Field Variational Inference via Entropic Regularization: Theory and Computationstat.ML
New method improves approximate inference for Bayesian models.
problem Approximate inference for high-dimensional Bayesian models.
method Entropic regularization of mean-field variational inference.
result Improved recovery of true posterior dependency.
Automates VI divergence selection for efficient few-shot learning.
problem Efficiently selecting divergence measures for VI to improve performance.
method Meta-learning algorithm to learn optimal divergence metric and variational parameter initialization.
result Meta-learning approach outperforms standard VI methods across various tasks.
This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
New algorithms solve stochastic variational inequalities without bounded variance assumption.
problem Solving stochastic variational inequalities without bounded variance assumption.
method Developed algorithms for two classes of problems: monotone and structured nonmonotone VIs.
result Oracle complexity of O(ε^-4) for solving VIs with unbounded domains and possibly unbounded variance.
A new variational inference method using Gaussian score matching.
problem Approximating posterior distributions in Bayesian statistics.
method Score matching principle applied to variational inference.
result Gaussian score matching VI (GSM-VI) is faster and requires fewer gradient evaluations.
Analyzes factors affecting flow VI performance.
problem Consistent performance of flow VI across studies.
method Step-by-step analysis of capacity, objectives, batchsize, estimators, and step-sizes.
result Specific recommendations and a flow VI recipe.
Improves MCMC and VI methods by optimising simulation bias.
problem Limitations of MCMC and VI methods in practice.
method Hybrid method combining optimisation with MCMC simulation.
result Produces low-biased samples with better balance between bias and efficiency.
New algorithm radVI improves variational inference by optimizing radial profiles.
problem Gaussian approximations often fail to capture the radial profile of complex distributions.
method Optimizes over radial profiles in variational inference, providing theoretical guarantees.
result Theoretical convergence guarantees for radVI, improving over existing VI methods.
New particle-based VI algorithm expands function class and improves scalability.
problem Limited function class in particle-based VI algorithms restricts flexibility and scalability.
method Introduces a functional regularization term to expand the function class and proposes PFG algorithm.
result Proposed PFG algorithm has larger function class, improved scalability, better adaptation to ill-conditioned distributions, and provable convergence.
Paper advances black-box VI using flows and Monte-Carlo methods.
problem Improving automatic posterior inference in black-box VI.
method Combines normalizing flows, Monte-Carlo methods, and optimization considerations.
result Significant improvement in state-of-the-art variational inference.
CAVI converges globally or locally exponentially for two-block models.
problem Convergence analysis of coordinate ascent variational inference (CAVI).
method Analysis of CAVI convergence using functional analysis and optimization.
result General conditions for certifying global or local exponential convergence of CAVI.
Boosting VI improves inference by greedily combining densities.
problem Improving tractability in Bayesian statistics.
method Adapting boosting techniques to VI, replacing a single density with a mixture of densities.
result Demonstrates convergence of boosting VI under relaxed smoothness assumptions.
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
YOASOVI improves stochastic VI for large models with fast, self-correcting sampling.
problem Efficiently performing stochastic Variational Inference on large Bayesian models.
method YOASOVI uses acceptance sampling to draw only one sample per iteration, improving convergence speed and accuracy.
result YOASOVI converges faster and more accurately than regular Monte Carlo and Quasi-Monte Carlo methods.
Variational inference simplifies Bayesian model approximations.
problem Approximating complex Bayesian posterior distributions.
method Solving optimization problems to approximate posterior distributions with simpler variational distributions.
result Variational inference has been successfully applied in various models and large-scale applications.
Paper bridges VI and GFlowNets, showing their equivalence in certain cases.
problem Modeling distributions over continuous and discrete structures.
method Demonstrates equivalence between VI and GFlowNets in specific scenarios.
result GFlowNets are more suitable for off-policy training without high gradient variance.
A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Modelsstat.ML
This paper bridges statistical and machine learning approaches to variational inference.
problem Statisticians struggle to understand variational inference from a Frequentist perspective.
method Explains VI, VAEs, and DDMs from a Frequentist viewpoint, starting with EM.
result VI emerges as a scalable solution for intractable E-steps in VAEs and DDMs.
BF-VI improves posterior approximation in complex models.
problem Inefficient posterior approximations in complex models.
method Combines normalizing flows and Bernstein polynomial transformations.
result BF-VI outperforms other VI methods in approximating complex multivariate posteriors.
One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this paper, we review variational inference (V…
Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.
problem Sparse logistic regression challenges in machine learning.
method Empirical Bayes approach with mean-field variational inference, tuning-free and scalable.
result Superior predictive performance in sparse logistic regression compared to existing methods.
New algorithm speeds up large-scale statistical inference.
problem Efficiently solving large-scale mean-field variational inference problems.
method Developed a novel primal-dual algorithm (PD-VI) and a block-preconditioned extension (PD-VI) for mean-field variational inference.
result PD-VI and PD-VI achieve faster convergence and better solution quality compared to existing methods.
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
MER algorithm speeds up VI solving with Markovian data.
problem Solving stochastic variational inequalities with Markovian data.
method MER algorithm using multi-scale sampling from a Markovian buffer.
result Achieves faster convergence without knowing Markov chain mixing time.
New algorithm minimizes inclusive KL for VI, improving accuracy.
problem Improving variational inference accuracy with KL(p||q).
method Markovian score climbing (MSC) using stochastic gradients.
result MSC converges to local optimum of inclusive KL without bias.
New algorithms reduce effort for uncertainty in deep learning.
problem Uncertainty computation in deep learning models.
method Natural-gradient algorithms within Adam optimizer for weight perturbation.
result Uncertainty estimates of comparable quality with lower effort.
Paper introduces f-divergence variational inference for broader application.
problem Variational inference limited to specific divergences.
method Generalizes variational inference to all f-divergences using f-divergence minimization.
result Unified framework for variational inference with arbitrary f-divergences.
This paper connects GANs to variational inference, providing new algorithms.
problem Understanding and applying variational inference with implicit distributions.
method Unified review of existing algorithms, introducing prior-contrastive and joint-contrastive methods.
result Unified understanding and practical inference algorithms for variational autoencoders and adversarially learned inference.
New methods improve multi-agent reinforcement learning by addressing rotational dynamics.
problem Reproducibility crisis in multi-agent reinforcement learning.
method Reframing MARL approaches using Variational Inequalities (VIs) and proposing gradient-based VI methods.
result Significant performance improvements across benchmarks, including better convergence to equilibrium strategies in zero-sum games.
This work proposes new methods for variational inference using gradient flows on Gaussian measures.
problem Developing algorithmic guarantees for variational inference.
method Proposes principled methods for variational inference using gradient flows on the Bures--Wasserstein space of Gaussian measures.
result Strong theoretical guarantees for log-concave posteriors.
RVI accelerates encoderless VI for faster convergence.
problem Slow convergence in encoderless VI methods.
method Introduces Relay Variational Inference (RVI) for faster learning.
result RVI outperforms existing methods in convergence speed and performance.
Boosting variational inference improves posterior approximations using mixtures of base distributions.
problem VI approximations do not converge to exact posterior due to constrained distribution family.
method Develops BVI algorithm that iteratively mixes current approximation with new components from base distribution family.
result BVI yields progressively more accurate posterior approximations, capturing multimodality and nonstandard shapes.
MetFlow combines MCMC and VI efficiently for better inference.
problem Combining MCMC and VI for efficient inference.
method Introduces MetFlow, a novel MCMC algorithm with Normalizing Flows, and a new method to combine it with VI.
result MetFlow produces expressive variational families with improved computational efficiency.
A-VI can approximate F-VI under certain conditions, improving inference in some models.
problem Approximate Bayesian inference using factorized distributions.
method Amortized variational inference (A-VI) learns a common inference function for latent variables.
result A-VI can close the amortization gap in simple hierarchical models.
This paper examines challenges and solutions for solving variational inequalities.
problem Stability issues in solving variational inequalities, especially in multi-objective scenarios.
method Continuous-time analysis to understand and improve stability of algorithms.
result Understanding continuous-time dynamics can help in designing more stable algorithms for variational inequalities.
Structured Nonparametric Variational Inference for Dependent Latent Modeling
problem Approximating posterior distributions with complex dependencies among latent variables
method Structured Nonparametric Variational Inference (SN-VI)
result Flexible and accurate posterior approximation with arbitrary shapes
This paper compares gradient estimators in importance-weighted VI and justifies the superiority of DREP over REP.
problem Understanding the impact of gradient estimators on importance-weighted VI algorithms.
method Unified theoretical comparison of reparameterized and doubly-reparameterized gradient estimators tied to IWAE, VR, and VR-IWAE bounds.
result Formally justifies the superiority of doubly-reparameterized gradient estimators over reparameterized ones in importance-weighted VI.
Improved L-BFGS algorithm with faster convergence and practical acceleration strategies.
problem Optimization of large-scale machine learning problems.
method New convergence analysis framework and practical acceleration strategies.
result Significant improvements in empirical performance on large-scale logistic and ridge regression problems.
Paper develops algorithms for solving non-convex non-concave problems with applications in GAN training.
problem Solving non-convex non-concave min-max saddle-point problems.
method Inexact proximal point method with strongly monotone mappings.
result First-order convergence to a nearly stationary solution of the original min-max problem.
CYCLEGAN models are shown to be a special case of approximate Bayesian inference.
problem Learning correspondences between domains without paired data.
method Formalized as Bayesian inference in an LVM, developed a VI algorithm based on KL divergence minimization.
result CYCLEGAN models can be derived within the proposed VI framework.
Partitioned Variational Inference: A unified framework encompassing federated and continual learningstat.ML
Unified framework for variational inference tackles federated and continual learning.
problem Fragmented literature in variational inference.
method Partitioned Variational Inference (PVI) framework.
result New methods outperform state-of-the-art in federated and continual learning.
New framework for improved inference algorithms.
problem Challenges in improving VI and MCMC methods.
method Ergodic Inference framework based on ergodic transformations.
result Established theoretical foundation for practical algorithms.
New findings on Malgrange-Galois groupoid for Painlevé VI equation parameters.
problem Understanding transformations preserving specific forms for Painlevé VI equation.
method Computed Malgrange-Galois groupoid for Painlevé VI family with all parameters.
result Solutions of Painlevé VI do not satisfy new partial differential equations.
New algorithms accelerate value function convergence in MDPs.
problem Accelerating convergence of value functions in Markov Decision Processes (MDPs).
method Operator Splitting Value Iteration (OS-VI) and OS-Dyna.
result Achieves much faster convergence rate with accurate models.
A new method interpolates between sampling and variational inference using stochastic mixtures.
problem Combining the strengths of sampling and variational inference methods.
method Develops a framework using stochastic mixtures of simple component distributions to interpolate between sampling and variational inference.
result Improves on both sampling and variational inference methods by reducing bias and variance.
Unified framework speeds up SMF algorithms via variance reduction.
problem Improving convergence speed and accuracy in stochastic matrix factorization.
method Unified framework using variance reduction for SMF.
result Consistently faster convergence and more accurate output.
Novel approach simplifies VI problems with faster performance.
problem Black-box VI optimization problems.
method Sample Average Approximation (SAA) combined with quasi-Newton methods and line search.
result Achieves faster performance than existing methods.