New variational principle found for non-variational differential equations.
problem Non-variational differential equations without variational multipliers.
method Connecting functional forms with antiexact differential forms to identify obstructions.
result Formulation of variational problem for non-variational equations.
Paper improves variational inference convergence using many control variates.
problem High variance in gradient estimates hinders variational inference convergence.
method Develops a Bayesian risk minimization framework to combine many control variates.
result Combining many control variates significantly improves inference convergence.
Derives Lagrangian for minimal surfaces, proving tangential variations vanish.
problem Variational calculus for minimal surfaces.
method Lagrangian formulation, pullback covariant derivative, geometric argument.
result Tangential variations vanish for minimal surfaces.
Improved Bayesian uncertainty quantification using variational bagging.
problem Inefficient and underestimating uncertainty in mean-field variational Bayes.
method Integrates bagging with variational Bayes for improved inference.
result Bagged variational posterior provides proper uncertainty quantification.
This research explores using Alpha-Divergences in variational dropout for better inference.
problem Improving variational inference methods using alternative divergences.
method Extending the Stochastic Gradient Variational Bayes (SGVB) framework with Alpha-Divergences.
result The α-divergence with αightarrow1 yields the lowest training error and optimizes the ELBO. A new variational method with statistical guarantees for Bayesian inference.
problem Improving variational inference with provable statistical guarantees.
method Introducing α-Variational Inference (α-VB) with statistical guarantees. result The α-VB method provides optimal convergence rates for parameter estimates. A new natural gradient accounts for correlated variational parameters in variational inference.
problem Traditional natural gradients fail to correct for correlations in variational inference.
method Construct a new natural gradient called the Variational Predictive Natural Gradient (VPNG).
result VPNG accounts for the relationship between model parameters and variational parameters.
This work improves variational inference by reducing gradient variance.
problem Hard optimization of flexible variational distributions.
method Control variate based on quadratic approximation of the model's mean and covariance.
result Significant improvement in gradient variance and optimization convergence.
A new framework enriches variational family with auxiliary variables.
problem Challenges in maximizing ELBO with complex variational distributions.
method Proposes a novel framework using auxiliary variables to enrich variational family.
result Flexible inference model built from probabilistic mixture of simple variational posteriors.
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
A new EVI framework improves ParVI methods by maintaining variational structure and reducing KL-divergence.
problem Improving variational inference methods for better approximation of target distributions.
method EVI framework that minimizes the VI objective function based on an energy-dissipation law, including a new 'Approximation-then-Variation' scheme.
result The new scheme significantly decreases KL-divergence and outperforms existing ParVI methods in fidelity.
Develops unbiased variational inference method for better model performance.
problem Improving variational inference methods for better model performance.
method Defines an expressive variational family using a simple reparameterizable distribution and deep neural networks, directly optimizing the ELBO.
result Achieves tighter ELBO and better predictive performance than existing approaches at similar computational cost.
Automatic variational inference simplifies Bayesian model fitting.
problem Manual variational inference is tedious and model-specific.
method Automatic Differentiation Variational Inference (ADVI) in Stan.
result ADVI automatically optimizes variational objective and selects family.
OPVI uses operators to optimize variational objectives, improving scalability and approximation quality.
problem Statistical properties of classical variational inference can be undesirable.
method OPVI redefines variational inference using operators to optimize variational objectives.
result OPVI enables data subsampling and variational programs, improving scalability and approximation quality.
New method finds global optima in variational inference.
problem Uncertainty in finding global optima in variational inference.
method Deterministic optimization algorithm for variational inference.
result Always converges to globally optimal variational lower bound.
Semi-Implicit Variational Inference broadens variational distributions.
problem Expanding the variational family to include implicit distributions.
method Mixing variational parameters with a flexible distribution.
result SIVI provides an asymptotically exact surrogate ELBO for optimization.
New method improves variational inference for hierarchical models.
problem Limited expressivity of variational distributions in Bayesian models.
method Importance weighted hierarchical variational inference.
result Superior performance in experiments compared to existing methods.
Improves VAE training by refining variational parameters with BSVI.
problem Amortized inference in VAEs leads to suboptimal variational parameters and the amortization gap.
method Proposes BSVI, a refinement procedure using SVI's importance weights.
result Training VAEs with BSVI yields improved performance compared to SVI.
Variational autoencoders learn deep latent models.
problem Learning deep latent-variable models.
method Principled framework using variational inference.
result Introduction to variational autoencoders and extensions.
New examples of variational bivectors found that are not Poissonian.
problem Identifying variational bivectors that are not Poissonian.
method Constructing examples of variational bivectors.
result Found examples of variational bivectors that are not Poissonian.
Incremental variational inference speeds up LDA processing.
problem Efficiently processing large document collections in LDA.
method Inspired by incremental EM, introduces incremental variational inference for LDA.
result Incremental LDA converges faster and monotonically improves variational bound.
Improved VAE estimation from incomplete data using variational mixtures.
problem Estimating VAEs from incomplete data increases posterior complexity.
method Introducing variational mixtures based on finite and imputation distributions.
result Variational mixtures improve VAE estimation accuracy from incomplete data.
Variational Prediction simplifies Bayesian inference without test time costs.
problem Bayesian inference's computational costs and posterior predictive distribution marginalization.
method Variational Prediction learns a variational approximation to the posterior predictive distribution using a variational bound.
result Directly learns a variational approximation to the posterior predictive distribution without test time marginalization costs.
In the paper "On Truncated Variation of Brownian Motion with Drift" (Bull. Pol. Acad. Sci. Math. 56 (2008), no.4, 267 - 281) we defined truncated variation of Brownian motion with drift, Wt=Bt+μt,t≥0, where (Bt) is a standard Brownian motion. Truncated variation differs from regular variation by neglect…
Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.
problem Intractable densities in variational distributions hinder SIVI training.
method SIVI-SM uses score matching to handle intractable densities in a minimax formulation.
result SIVI-SM outperforms ELBO-based SIVI methods in Bayesian inference tasks.
This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.
problem Pathwise gradient estimators in variational inference have high variance, leading to inefficient optimization.
method Apply zero-variance control variates to pathwise gradient estimators.
result Zero-variance control variates can significantly reduce the variance of pathwise gradient estimators without requiring complex assumptions.
AVO improves variational inference by encouraging exploration in latent space.
problem Biasing the true posterior to be unimodal limits the density learned in variational inference.
method Inspired by Annealed Importance Sampling, AVO incorporates energy tempering into the optimization objective.
result AVO facilitates learning by encouraging exploration in latent space, improving robustness and benefits.
The paper studies variational functionals for submanifolds using the Lepage form.
problem Variational functionals for submanifolds in Grassmann fibrations.
method Introduces the fundamental Lepage form and uses it to study variations of submanifolds.
result Proves the first infinitesimal variation formula and Euler-Lagrange equations.
New classification of hypersurfaces with conformal variations.
problem Classifying hypersurfaces with conformal infinitesimal variations.
method Analyzing hypersurfaces in conformal geometry, extending previous work by Cartan and Sbrana.
result The class of hypersurfaces with conformal infinitesimal variations is larger than previously known.
The paper classifies and studies conformal variations of submanifolds.
problem Classifying and understanding conformal variations of submanifolds.
method Develops a Fundamental theorem and a rigidity theorem for Euclidean submanifolds.
result Fundamental theorem and rigidity theorem for Euclidean submanifolds.
This tutorial derives the VAE loss function under Gaussian assumptions.
problem Computational intractability of posterior distributions in Bayesian machine learning.
method Derives the variational lower bound loss function of a standard VAE.
result The Kullback-Leibler divergence has a closed form solution under Gaussian assumptions.
A new variational inference method using optimal transport.
problem Approximating complex posterior distributions with flexible particle-based methods.
method Introducing a new particle-based variational inference method based on semi-discrete optimal transport.
result The method provides a particle approximation and optimal transportation densities.
Variational boosting refines posterior approximations through iterative optimization.
problem Approximating intractable distributions with rich approximations.
method Iteratively solves optimization problems to refine variational approximations.
result Posterior inferences using variational boosting are more accurate and efficient.
Wasserstein variational inference uses optimal transport for stable likelihood-free training.
problem Approximate Bayesian inference with stability and flexibility for implicit distributions.
method Optimal transport theory, Sinkhorn iterations, and backpropagation.
result Stable likelihood-free training method for autoencoders and probabilistic programs.
Variational approach to basic manifold structures.
problem Understanding basic differential geometric structures.
method Variational description of geometric structures.
result Variational formulation of manifold structures.
The paper introduces structured variational families to improve scalability in black-box variational inference.
problem Scalability issues in black-box variational inference, especially for large datasets and hierarchical models.
method Developed structured variational families that achieve better iteration complexity of O(N) compared to full-rank families.
result Structured variational families can achieve better scaling with respect to dataset size N, improving iteration complexity from O(N^2) to O(N).
New method reduces inference variance for faster optimization.
problem High variance in black-box variational inference.
method Joint control variate addressing both data subsampling and Monte Carlo noise.
result Significantly reduced gradient variance, leading to faster optimization.
Paper variates Navier-Stokes-Fourier system for thermodynamic consistency.
problem Modeling compressible fluid dynamics with thermodynamic constraints.
method Variational discretization with discrete exterior calculus.
result Derives a nonholonomic variational integrator for NSF system.
TVO tightens variational inference bounds for deep models.
problem Improving variational inference bounds for deep models.
method Introduces thermodynamic variational objective (TVO) connecting variational inference and thermodynamic integration.
result TVO provides tighter lower bound to log marginal likelihood than ELBO.
We derive a formula for the first variation of horizontal perimeter measure for C2 hypersurfaces of completely general sub-Riemannian manifolds, allowing for the existence of characteristic points. For C2 hypersurfaces in vertically rigid sub-Riemannian manifolds we also produce a second variation formula for var…
Study convergence rates of variational posterior distributions for inference.
problem Characterize convergence rates of variational posterior distributions for nonparametric and high-dimensional inference.
method Formulate general conditions on prior, likelihood, and variational class to characterize convergence rates. Propose novel prior mass conditions for specific prior distributions.
result The convergence rate of variational posterior distributions is the sum of the convergence rate of the true posterior and the variational approximation error.
Study variational submanifolds in Euclidean spaces from dynamical systems.
problem Formulate and solve conditions for variationality of induced systems on submanifolds.
method Employ variational sequence theory on sheaves of differential forms to analyze local and global variationality.
result Solve the problem of existence of variational submanifolds for second-order systems.
Formulas for mean curvature in various spaces.
problem Understanding mean curvature in different geometric settings.
method First and second variation formulas for arbitrary variations in Riemannian and sub-Riemannian manifolds.
result Formulas derived for mean curvature in Heisenberg group via approximation.
The paper interprets VQ-VAE loss as a form of information bottleneck.
problem Understanding the VQ-VAE loss function.
method Interpreted VQ-VAE loss as variational deterministic information bottleneck (VDIB) and variational information bottleneck (VIB).
result VQ-VAE loss can be derived from VDIB and approximated by VIB.
New method for variational inference tackles noisy estimation and high-dimensional latent variables.
problem Challenges in estimating and applying implicit variational posteriors to high-dimensional models.
method Kernel Implicit Variational Inference (KIVI) approach.
result Successfully applied to Bayesian neural networks, showing promising results.
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.
Combines Laplace approximation and variational inference for better posterior correlations.
problem Lack of posterior correlations in variational inference.
method Combines Laplace approximation and variational inference, explicitly minimising KL divergence.
result Improves over Laplace approximation and variational inference with factorised Gaussian posteriors.
New method finds global Lagrangians for variational systems.
problem Constructing global variational principles for variational systems.
method Analyzing Lepage 2-forms and finding global Lagrangians for systems defined by homogeneous functions of degree \(c
eq 0, 1\).
result Locally variational systems defined by homogeneous functions of degree \(c
eq 0, 1\) are globally variational.