Rotates MFVI for better Gaussian approximations.
problem Improving variational approximations for complex distributions.
method Rotated coordinate system, PCA-based rotation, iterative Gaussianization.
result Significantly more accurate approximations with lower computational cost.
MFVI can overestimate predictive variance compared to the exact posterior
problem MFVI underestimates posterior variance
method Analyzing conjugate Bayesian Linear Regression
result MFVI can overestimate predictive variance compared to the exact posterior
MFVI mode collapse explained; RoVI proposed to mitigate.
problem Mode collapse in MFVI for mixture distributions.
method Introducing ε-separateness, deriving bounds, proposing RoVI.
result MFVI optimizers collapse to a single component when components are ε-separated.
The paper studies stability of mean-field variational inference for log-concave distributions.
problem Stability of mean-field variational inference for log-concave distributions.
method Novel approach via linearized optimal transport, lifting non-convex problem to convex optimization over transport maps.
result Dimension-free Lipschitz continuity of the MFVI optimizer with respect to the target distribution, measured in 2-Wasserstein distance.
Paper improves variational inference for complex models.
problem Improving statistical accuracy of variational inference in high-dimensional models.
method Developed a general framework for MFVI and proposed a partially grouped VI algorithm.
result Proposed algorithm works and outperforms vanilla MFVI in mixed membership stochastic blockmodel.
Analog BNNs perform similarly regardless of noise distribution shape.
problem Difficulty in precisely controlling noise distribution shape in analog devices.
method Used real device noise as the variational distribution in MFVI training.
result Predictive distributions converge to the same distribution regardless of noise shape.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Develops a new framework for analyzing MFVI algorithms.
problem Analyzes mean field variational inference (MFVI) formulations.
method Inspired by variational Bayesian formulations, represents MFVI problem in three ways: gradient flow, Fokker-Planck-like equations, and diffusion process.
result Establishes rigorous guarantees for convergence of time-discretized coordinate ascent variational inference algorithms.
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. Th…
We propose Radial Bayesian Neural Networks (BNNs): a variational approximate posterior for BNNs which scales well to large models while maintaining a distribution over weight-space with full support. Other scalable Bayesian deep learning methods, like MC dropout or deep ensembles, have discrete support-they assign zero…
Leaky ReLU activations improve the calibration of Bayesian neural networks.
problem Bayesian neural networks struggle with mean-field variational inference for ReLU activations.
method Investigated the effect of activation functions on the calibration of Bayesian neural networks.
result Leaky ReLU activations lead to more Gaussian-like weight posteriors and lower expected calibration error.
This work connects SAM to variational inference and evaluates its performance.
problem Improving generalization of gradient-based learning by finding flat minima.
method Establishes connections between SAM and Mean-Field Variational Inference (MFVI), and evaluates variational algorithms combining or interpolating between SAM and MFVI.
result SAM-like updates can be used as a drop-in replacement for the reparametrisation trick.
A novel stepwise VI method using vine copulas for complex latent dependence.
problem Modeling complex latent dependence structures in probabilistic models.
method Stepwise estimation of vine copula parameters using Rényi divergence and a stopping criterion.
result Our method outperforms mean-field VI and is more parsimonious in complex applications.
New algorithm speeds up large-scale statistical inference.
problem Efficiently solving large-scale mean-field variational inference problems.
method Developed a novel primal-dual algorithm (PD-VI) and a block-preconditioned extension (P2D-VI) for mean-field variational inference. result PD-VI and P2D-VI achieve faster convergence and better solution quality compared to existing methods. Bayesian models provide a framework for probabilistic modelling of complex datasets. However, many of such models are computationally demanding especially in the presence of large datasets. On the other hand, in sensor network applications, statistical (Bayesian) parameter estimation usually needs distributed algorithm…
Researchers estimate optimal PAC-Bayes bounds using Hamiltonian Monte Carlo.
problem Estimating tight PAC-Bayes bounds with restricted posterior families.
method Sampling from optimal Gibbs posterior using Hamiltonian Monte Carlo, estimating KL divergence, and proposing high-probability bounds.
result Significant tightness gaps in PAC-Bayes bounds, up to 5-6% in some cases.
The Koopman operator has emerged as a powerful tool for the analysis of nonlinear dynamical systems as it provides coordinate transformations to globally linearize the dynamics. While recent deep learning approaches have been useful in extracting the Koopman operator from a data-driven perspective, several challenges r…