Empirical Bayes rates via variational approximations and prior decomposition.
problem Nonparametric and high-dimensional inference convergence rates.
method Variational perspective and prior decomposition.
result Empirical Bayes posterior rates derived from variational Bayes.
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
VBphenoR uses variational Bayes for EHR-based patient phenotyping.
problem Phenotyping patients from EHR data for targeted treatments.
method Variational Bayes Gaussian Mixture Model (GMM) and logistic regression.
result Closed-form inference for efficient patient phenotype determination.
Simplified Variational Bayes for easier inference.
problem Complex derivation of Variational Bayes.
method 3-step recipe to identify posterior form and directly write updates.
result Easier, faster, shorter derivation of Variational Bayes.
A new method combines Laplace and Variational Bayes for scalable inference.
problem Complex models and large datasets make exact inference infeasible.
method Low-Rank Variational Bayes Correction (VBC) using Laplace method and Variational Bayes correction in a lower dimension.
result The method ensures scalability in both model complexity and data size.
Unified empirical and variational Bayes for unnormalized densities.
problem Approximating unnormalized densities using latent variable models.
method Formulate a latent variable model for Y=X+N(0,σ2Id), use ELBO as parametrization of Y's energy function, and estimate X with empirical Bayes least-squares. result UVB has higher capacity to approximate energy functions than MLPs in DEEN.
A practical guide to Variational Bayes methods.
problem Bayesian inference for data analysis practitioners.
method Explains and demonstrates commonly used Variational Bayes methods.
result Quick derivation and implementation of VB algorithms.
This paper presents studies on a deterministic annealing algorithm based on quantum annealing for variational Bayes (QAVB) inference, which can be seen as an extension of the simulated annealing for variational Bayes (SAVB) inference. QAVB is as easy as SAVB to implement. Experiments revealed QAVB finds a better local …
New clustering algorithm for time series data using RNN and variational Bayes.
problem Lack of generative model-based clustering methods for time series data.
method Recurrent Neural Network (RNN) with variational Bayes method.
result Robustness against phase shift, amplitude, and signal length variations.
A framework uses variational Bayes for solving inverse problems efficiently.
problem Solving inverse problems in various dimensions with flexibility and accuracy.
method Variational Bayes approximations with message passing and factor graph approach.
result Efficient algorithm updates for higher dimensions and computational advantage over MCMC.
Develops a fast variational approximation for high-dimensional empirical Bayes posteriors.
problem Optimal posterior computation in high-dimensional settings with prior tails effect.
method Variational approximation of empirical Bayes posterior with data-driven centers and thin-tailed conjugate priors.
result Retains optimal concentration rate properties and superior performance compared to existing methods.
Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…
Bayes-CATSI uses variational Bayesian deep learning for medical time series data imputation.
problem Missing values in medical time series data.
method Bayes-CATSI integrates variational inference for uncertainty quantification and context-aware imputation.
result Bayes-CATSI outperforms CATSI by 9.57% in imputation performance.
We introduce Deep Variational Bayes Filters (DVBF), a new method for unsupervised learning and identification of latent Markovian state space models. Leveraging recent advances in Stochastic Gradient Variational Bayes, DVBF can overcome intractable inference distributions via variational inference. Thus, it can handle …
The paper improves Gaussian process regression by optimizing hyperparameters.
problem Hyperparameter tuning for Gaussian process regression models.
method Adaptive sparse variational approximations using variational Bayes.
result Minimax optimal rates of convergence for variational posterior.
Efficiently identifies important variables in binary outcomes using variational Bayes.
problem Bayesian variable selection for binary outcomes with computational challenges.
method Mean-field variational Bayes approximation with closed-form updates and efficient inference algorithm.
result Successfully identifies important variables and is orders of magnitude faster than MCMC.
Bayesian supertrees method uses variational Bayes for overlapping taxon subsets.
problem Inference of phylogenetic tree topologies for overlapping taxon sets.
method Variational Bayesian approach.
result Demonstrates effectiveness of variational Bayes for Bayesian supertrees.
Quantum annealing improves VB inference, avoiding local minima.
problem Variational Bayes inference stuck in local minima.
method Quantum annealing approach to VB inference.
result Quantum annealing variational Bayes (QAVB) outperforms classical VB.
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.
Variational Bayes (VB) inference is one of the most important algorithms in machine learning and widely used in engineering and industry. However, VB is known to suffer from the problem of local optima. In this Letter, we generalize VB by using quantum mechanics, and propose a new algorithm, which we call quantum annea…
VB uses natural gradients in information geometry.
problem Estimating or computing natural gradients in VB.
method Natural-gradient descent algorithm and Bayesian Learning Rule.
result Simplification of Bayes' rule and generalization of quadratic surrogates.
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
Fast variational Bayes methods improve geospatial data analysis speed and accuracy.
problem Inaccurate and slow variational Bayes methods for large geospatial data.
method Combination of calculus of variations, closed-form gradient updates, and linear response corrections.
result Comparable accuracy to spNNGP with reduced computational costs and faster speed.
New method for high-dimensional linear regression using empirical Bayes.
problem Estimating prior in high-dimensional linear regression.
method Variational empirical Bayes approach with NPMLE and mean field approximation.
result Established asymptotic consistency and computational efficiency of the method.
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) …
A scalable method for accurate inference of low-dimensional parameters in high-dimensional linear regression.
problem Statistical inference for low-dimensional parameters in high-dimensional linear regression models.
method Mean-field variational Bayes approach, focusing on nuisance parameters and conditional distributions.
result Competitive numerical performance and theoretical guarantees for estimation and uncertainty quantification.
Improved Bayesian uncertainty quantification using variational bagging.
problem Inefficient and underestimating uncertainty in mean-field variational Bayes.
method Integrates bagging with variational Bayes for improved inference.
result Bagged variational posterior provides proper uncertainty quantification.
A new method improves likelihood-free Bayesian inference by transforming summary statistics and using efficient Variational Bayes.
problem Incorrectly assuming normally distributed summary statistics in likelihood-free Bayesian inference.
method Wasserstein Gaussianization transformation combined with robust BSL and efficient Variational Bayes.
result Highly efficient and reliable approximate Bayesian inference for likelihood-free problems.
In this document we are going to derive the equations needed to implement a Variational Bayes i-vector extractor. This can be used to extract longer i-vectors reducing the risk of overfittig or to adapt an i-vector extractor from a database to another with scarce development data. This work is based on Patrick Kenny's …
RQMC improves optimization in variational Bayes problems.
problem Optimizing variational Bayes problems with noisy objective functions.
method Use of randomized quasi-Monte Carlo (RQMC) sampling with stochastic L-BFGS.
result RQMC can significantly speed up optimization and find better parameter values.
The paper analyzes high-dimensional linear regression using parametric empirical Bayes methods.
problem Estimation of i.i.d. priors in high-dimensional Bayesian linear regression with random design.
method Parametric empirical Bayes estimation, variational lower bound maximization, phase transition analysis.
result The vEB estimator is information theoretically optimal up to p=o(n2/3) but sub-optimal in higher dimensions. Quantum method speeds up VB estimation in machine learning.
problem Prohibitively expensive natural gradient in high dimensions.
method Regression-based natural gradient estimation with quantum matrix inversion.
result Quantum method enables efficient VB estimation.
New scalable variational Bayes methods for Hawkes processes.
problem Computational intractability of Bayesian estimation for generalised nonlinear Hawkes processes.
method Unified variational Bayes framework, adaptive mean-field approximation, sparsity-inducing procedure.
result Adaptive mean-field variational algorithm for sigmoid Hawkes processes is scalable and robust.
Variational Gaussian Processes solve linear inverse problems efficiently.
problem Solving inverse problems where indirect observations are corrupted by noise.
method Variational Bayesian methods with Gaussian process priors and inducing variables.
result Posterior contraction rates can be attained by correctly tuned variational procedures.
Deep neural networks have achieved impressive results on a wide variety of tasks. However, quantifying uncertainty in the network's output is a challenging task. Bayesian models offer a mathematical framework to reason about model uncertainty. Variational methods have been used for approximating intractable integrals t…
Meta-learning framework improves model performance on few-shot classification tasks.
problem Improving model performance on few-shot classification tasks.
method Empirical Bayes formulation with synthetic gradients for transductive meta-learning.
result Meta-learning framework outperforms previous state-of-the-art methods on benchmarks.
Bayesian hierarchical models are increasing popular in economics. When using hierarchical models, it is useful not only to calculate posterior expectations, but also to measure the robustness of these expectations to reasonable alternative prior choices. We use variational Bayes and linear response methods to provide f…
Gradient-based optimization improves variational empirical Bayes regression.
problem Sparse, large-scale multiple regression models.
method Gradient-based optimization (GradVI) for variational empirical Bayes (VEB) regression.
result GradVI produces similar predictive performance to CAVI but converges faster and is faster in certain settings.
This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making con…
We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two…
Efficiently selects important variables in high-dimensional logistic regression.
problem Variable selection in high-dimensional logistic regression with binary responses.
method Developed a variational empirical Bayes approach for efficient model space marginal distribution.
result The variational approximation inherits strong selection consistency from the posterior distribution.
The cold posterior effect is explored through PAC-Bayes bounds for small sample sizes.
problem The cold posterior effect in approximate Bayesian inference for small datasets.
method Investigation through PAC-Bayes generalization bounds, focusing on temperature parameter λ.
result The temperature parameter λ in PAC-Bayes bounds captures the cold posterior effect.
New approach to meta-learning with variational Bayes for unlabeled data.
problem Improving machine learning systems' adaptability to small data.
method Generative meta-learning using AEVB and mean-field variational Bayes.
result Relevant VB updates do not depend on generative neural networks for certain models.
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
problem Sparse high-dimensional logistic regression model selection.
method Spike and slab variational Bayes approximation.
result Optimal convergence rates in ℓ2 and prediction loss for sparse truths. Mean Field Variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, it is well known that a major failing of MFVB is its (sometimes severe) underestimates of the uncertainty of model variables and lack of information about model variable covariance.…
Robust VB framework for large datasets with outliers.
problem Handling outliers and contamination in large datasets.
method Divide and conquer approach with geometric median aggregation.
result VM-Posterior distribution preserves contraction properties.
Recently, a number of mostly ℓ1-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a Bayesian perspective, this task is equivalent to maximum a posteriori probability estimation under a …
Bayesian Empirical Bayes extends EB to complex structures using probabilistic symmetry.
problem Improving simultaneous inference in complex settings like arrays and graphs.
method Generalized empirical Bayes approach based on probabilistic symmetry.
result BEB outperforms existing methods in denoising arrays and spatial data.