A new natural gradient accounts for correlated variational parameters in variational inference.
problem Traditional natural gradients fail to correct for correlations in variational inference.
method Construct a new natural gradient called the Variational Predictive Natural Gradient (VPNG).
result VPNG accounts for the relationship between model parameters and variational parameters.
Paper introduces sampling-based speech synthesis with natural variation.
problem Synthetic speech lacks natural inter-utterance variation.
method Moment-matching networks trained to match moments of generated speech parameters to natural speech parameters.
result Sampling-based generation does not degrade synthetic speech quality.
TM-VI uses flexible transformation models to approximate complex posteriors in Bayesian models.
problem Approximating complex posteriors in Bayesian models with limited flexibility.
method Transformation models for variational inference (TM-VI).
result TM-VI allows accurate approximation of complex posteriors in models with one parameter and works in a mean-field fashion for multi-parameter models.
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
Inference for SDEs using variational methods and neural networks.
problem Parameter inference for stochastic differential equations is challenging due to latent diffusion processes.
method Variational inference with a mean-field approximation for parameters and a recurrent neural network for diffusion paths.
result Accurate parameter estimates for SDE systems, demonstrated on Lotka-Volterra and epidemic models.
We find the maximum regularization parameter for total-variation denoising.
problem Finding the maximum regularization parameter for anisotropic total-variation denoising.
method Established a closed form expression for the one-dimensional case and an upper-bound for the two-dimensional case using the pseudo-inverse of the divergence.
result The maximum regularization parameter is crucial for optimal parameter tuning and can be computed efficiently.
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
Improves VAE training by refining variational parameters with BSVI.
problem Amortized inference in VAEs leads to suboptimal variational parameters and the amortization gap.
method Proposes BSVI, a refinement procedure using SVI's importance weights.
result Training VAEs with BSVI yields improved performance compared to SVI.
This work uses variational inference to estimate parameters of opinion dynamics models.
problem Challenges in parameter estimation for ABMs of social phenomena.
method Transformed ABM parameter estimation into an optimization problem using variational inference.
result Estimates parameters more accurately than simulation-based and MCMC methods.
Proposes a new DQN framework using variational inference.
problem Efficient exploration and large-scale MDPs.
method Variational inference for value function parameters.
result Achieves efficient exploration on large-scale MDPs.
Online method for state estimation and parameter learning in SSMs.
problem State estimation and parameter learning in state-space models.
method Stochastic gradient optimization of variational lower bound, using backward decompositions and Bellman recursions.
result Ability to operate online without revisiting historic observations.
Combines Laplace approximation and variational inference for better posterior correlations.
problem Lack of posterior correlations in variational inference.
method Combines Laplace approximation and variational inference, explicitly minimising KL divergence.
result Improves over Laplace approximation and variational inference with factorised Gaussian posteriors.
Improved DP-SGD for variational inference reduces noise and variance.
problem Poor convergence and high variance in variational parameter outputs due to gradient noise in DP-SGD.
method Introduced aligned gradients and iterate averaging to reduce DP-induced noise, and noise-aware posteriors.
result Less noisy gradient estimator and improved parameter estimates for variational inference.
Solves wealth maximization problem using variational analysis.
problem Maximizing expected utility of terminal wealth.
method Variational analysis, forward-backward stochastic differential equation (FBSDE).
result Characterization and solutions for various utility functions.
Paper introduces a new method to model epidemic dynamics with varying parameters.
problem Capturing discontinuous variations in epidemic model parameters.
method Total variation regularization with Iterated Nelder--Mead optimization.
result The method accurately models epidemic dynamics with instant changes.
We apply variational inference to learn vehicle trajectory parameters from noisy data.
problem Learning parameters for vehicle trajectory estimation from noisy measurements.
method Gaussian variational inference with parameter learning in a motion and sensor model context.
result High-quality state estimates achieved even with outliers and false loop closures.
Variational Dropout sparsifies deep neural networks by reducing parameter count.
problem Overfitting and high parameter count in deep neural networks.
method Extending Variational Dropout to unbounded dropout rates, reducing gradient variance, and applying individual dropout rates per weight.
result Significant reduction in parameter count (up to 280 times) with minimal accuracy loss.
A new method reduces complexity and uncertainty in neural networks.
problem Uncertainty quantification in complex neural networks.
method Condensed Stein Variational Gradient Descent (cSVGD) method.
result Condensed SVGD provides uncertainty quantification on parameters.
Method for initializing Gaussian mixtures for variational inference with multi-modal distributions.
problem Challenges in variational inference with Gaussian mixtures due to multimodality and nonconvex loss functions.
method Optimization to find local maxima, local Gaussian approximations, and constrained least squares regression.
result Robust initialization improves variational inference performance and scalability.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
The paper develops scalable variational inference for Bayesian neural networks under model and parameter uncertainty.
problem Combining structural and parameter uncertainties in scalable Bayesian neural networks.
method Adapted variational inference with reparametrization for model space constraints.
result Comparable accuracy with sparse inference compared to ordinary BNNs.
Proposes a new method for continual learning in neural networks.
problem Challenges in applying sequential Bayesian inference to neural networks.
method Sequential function-space variational inference.
result Neural networks trained with the proposed method achieve better predictive accuracy.
Variational Laplace improves Bayesian neural networks performance.
problem Improving Bayesian neural networks performance.
method Develops variational Laplace for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms other inference methods.
Improved inference for heterogeneous multi-output Gaussian processes using natural gradient optimization.
problem Challenges in adaptive gradient optimization for multi-output Gaussian processes.
method Introducing a fully natural gradient scheme to overcome optimization issues.
result Better local optima solutions and higher test performance rates compared to adaptive gradient methods.
ESVI speeds up variational inference for large datasets.
problem Inefficient scaling of SVI for large datasets and high parameter counts.
method Asynchronous, lock-free algorithm for distributed and asynchronous variational inference.
result ESVI outperforms SVI and VI in wallclock-time and quality of solution.
New method reduces parameter overhead for Bayesian neural networks.
problem High parameter overhead and difficulty of implementation in variational Bayesian neural networks.
method Constructs a general variational family for ensemble-based Bayesian neural networks that works well with batch normalization layers.
result Improves predictive accuracy and achieves almost perfect calibration on a ResNet-18 trained with ImageNet.
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Bayesian state and parameter estimation for nonlinear models using variational methods.
problem Estimating states and parameters for nonlinear state-space models.
method Variational approach to approximate the intractable Bayesian distribution, resulting in an optimisation problem.
result The proposed method efficiently computes Bayesian estimates for nonlinear models, outperforming Hamiltonian Monte Carlo in numerical examples.
Variational Laplace improves Bayesian neural network performance without sampling.
problem Improving Bayesian neural network performance and calibration.
method Develops a new variational Laplace method for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms standard VI methods in test performance and calibration.
A new VSMC family improves variational inference efficiency and accuracy.
problem Efficient and accurate Bayesian inference for complex models.
method Integrates variational inference and sequential Monte Carlo for flexible posterior approximation.
result VSMC family can approximate posterior arbitrarily well and optimize parameters efficiently.
Semi-amortized approach improves generative models without posterior-collapse.
problem Posterior-collapse in VAEs for tasks like text generation.
method Hybrid approach combining AVI for initialization and SVI for refinement.
result Semi-amortized approach outperforms autoregressive and variational baselines.
Proposes an amortized variational framework for Deep Q Networks.
problem Efficient exploration in deep reinforcement learning.
method Amortized variational inference for action value function approximation.
result Significantly less learning parameters and better performance.
VNCE improves parameter estimation and posterior inference for unnormalised models.
problem Learning parameters of unnormalised latent variable models is intractable.
method Variational noise-contrastive estimation (VNCE) using a variational lower bound to the NCE objective function.
result VNCE can be used for both parameter estimation and posterior inference of latent variables.
Bayesian surrogate models reduce uncertainty in high-dimensional design optimisation problems.
problem Uncertainty in high-dimensional inputs for complex computational models.
method Variational Bayesian inference for constructing statistical surrogates with Gaussian process priors and KL divergence for approximation.
result The RDVGP surrogate provides accurate and versatile approximations for robust structural optimisation.
A scalable method for accurate inference of low-dimensional parameters in high-dimensional linear regression.
problem Statistical inference for low-dimensional parameters in high-dimensional linear regression models.
method Mean-field variational Bayes approach, focusing on nuisance parameters and conditional distributions.
result Competitive numerical performance and theoretical guarantees for estimation and uncertainty quantification.
We introduce local expectation gradients which is a general purpose stochastic variational inference algorithm for constructing stochastic gradients through sampling from the variational distribution. This algorithm divides the problem of estimating the stochastic gradients over multiple variational parameters into sma…
Variational autoencoders struggle with inference quality due to recognition network limitations.
problem Inference suboptimality in variational autoencoders.
method Examined approximate inference in terms of variational distribution capacity and recognition network quality.
result Inference quality is more influenced by recognition network limitations than variational distribution complexity.
Paper uses variational inference to estimate nonlinear models.
problem Parameter estimation for nonlinear state-space models.
method Variational inference approach for nonlinear state-space models.
result The method provides robust parameter estimates and outperforms alternatives.
Novel variational EM approach using truncated distributions.
problem Efficiently modeling latent variables and mixtures.
method Truncated posterior distributions as variational parameters.
result Efficient and concise variational lower bounds.
PAVI speeds up VI for large-scale studies by sharing parameterization across i.i.d. variables.
problem Challenges in Bayesian inference for large population studies with many latent parameters.
method Designing plate-amortized variational inference (PAVI) to share parameterization across i.i.d. variables.
result Significant speedup in training large-scale hierarchical variational distributions.
HiP-MDPs help personalize HIV treatment across patient variations.
problem Physiological variation leads to different responses to treatments.
method Embed tasks in a low-dimensional space, updating HiP-MDP framework.
result Robust personalized medicine strategies developed for HIV treatment.
The paper proves variational Bayes methods are statistically optimal under certain conditions.
problem Justification of variational Bayes methods for parameter estimation.
method General conditions for optimal risk bounds in mean-field variational Bayesian inference.
result Optimal risk bounds for variational Bayes estimates are achievable under specific conditions.
This paper extends VB to manifolds for more flexible parameter spaces.
problem Limited applicability of VB due to Euclidean parameter spaces.
method Developed manifold-based VB algorithms exploiting geometric and information geometry.
result Proven convergence and improved performance compared to existing methods.
Improves SVGP methods for faster and more accurate Gaussian process inference.
problem Efficient non-conjugate Gaussian process inference.
method Dual parameterization of SVGP methods using site parameters.
result Faster and more accurate inference with tighter evidence lower bound.
The study uncovers latent parameters for phase transitions in 2D and 3D models using PCA and VAE.
problem Identifying latent parameters for phase transitions in complex systems.
method Employed unsupervised learning techniques including PCA and VAE.
result The latent parameters correspond to known order parameters and can identify phases without prior knowledge.
Efficient Bayesian inference via Gaussian approximations.
problem Parameter estimation and model selection in Bayesian statistics.
method Variational-Laplace approach using Gaussian approximations.
result Novel theoretical results on asymptotic convergence of VL schemes.
Bidirectional VAE reduces parameters and improves image tasks.
problem Improving image reconstruction, classification, interpolation, and generation.
method Uses a single neural network for both encoding and decoding in both forward and backward directions.
result Bidirectional VAEs reduce parameters by almost 50% and slightly outperform unidirectional VAEs.
Proposes a method to emulate sparse priors using L1 regularization without complex transformations.
problem Sparse priors in under-determined estimation problems.
method Parameter transform to emulate sparse priors under L2 regularization.
result L1 regularization can be achieved with a remapping of parameters under normal priors.