We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “variational-Bayesian”
The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of acting as a tutorial for the derivation of variational Bayesian inference for simple models, as well a…
BayesPy is an open-source Python software package for performing variational Bayesian inference. It is based on the variational message passing framework and supports conjugate exponential family models. By removing the tedious task of implementing the variational Bayesian update equations, the user can construct model…
Improved uncertainty estimation in neural networks with VBLL.
problem Improving uncertainty estimation in neural networks.
method Deterministic variational formulation for training Bayesian last layer neural networks.
result Improves predictive accuracy, calibration, and out-of-distribution detection.
Unified approach to DP problems using Gumbel distribution and variational Bayesian inference.
problem Solving classical optimal path problems in a probabilistic framework.
method Gumbel distribution and variational Bayesian inference for latent optimal paths.
result Unified approach transforms DP problems into directed acyclic graphs with Gibbs distribution.
Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defi…
We present a fast variational Bayesian algorithm for performing non-negative matrix factorisation and tri-factorisation. We show that our approach achieves faster convergence per iteration and timestep (wall-clock) than Gibbs sampling and non-probabilistic approaches, and do not require additional samples to estimate t…
Improved phylogenetic inference with normalizing flows.
problem Limitations of current diagonal Lognormal branch length approximation in VBPI.
method Proposes VBPI-NF using normalizing flows to handle non-Euclidean branch length space.
result Significantly improves phylogenetic posterior estimation on real data.
Variational Bayesian Inference For A Scale Mixture Of Normal Distributions Handling Missing Datastat.ML
In this paper, a scale mixture of Normal distributions model is developed for classification and clustering of data having outliers and missing values. The classification method, based on a mixture model, focuses on the introduction of latent variables that gives us the possibility to handle sensitivity of model to out…
Mean Field Variational Bayesian Inference and Statistical Mechanics of Gaussian Mixture Modelmath.DG
This paper analyzes MFVBI for GMM using statistical mechanics.
problem Approximate fast computation of Gaussian Mixture Model.
method Statistical mechanics and MFVBI applied to GMM.
result Rigorous analysis and mathematical foundation for MFVBI applied to GMM.
VB approach for dynamic network models improves efficiency and accuracy.
problem Estimating dynamic network models in large-scale systems.
method Variational Bayesian inference for network autoregression.
result VB approach detects proper active structures and achieves similar or better accuracy.
Improved phylogenetic inference using VBPI-Mixtures for tree topology and branch length.
problem Multimodality of tree-topology posterior distributions in phylogenetic inference.
method VBPI-Mixtures algorithm that uses mixture learning within the BBVI framework.
result VBPI-Mixtures captures tree-topology distributions better than VBPI.
A software library for constructing and learning probabilistic models is presented. The library offers a set of building blocks from which a large variety of static and dynamic models can be built. These include hierarchical models for variances of other variables and many nonlinear models. The underlying variational B…
New method improves cause-effect identification using neural networks.
problem Identifying cause and effect from observational data.
method Variational Bayesian learning of neural networks.
result Improves model fitness and codelengths succinctness.
Scalable Bayesian LASSO using variational inference for large p and n.
problem Large-scale regression with sparsity constraints.
method Variational Bayesian inference with scale mixtures of Normal distributions.
result The method achieves comparable performance to Bayesian LASSO but at a lower computational cost.
Complex network reconstruction is a hot topic in many fields. Currently, the most popular data-driven reconstruction framework is based on lasso. However, it is found that, in the presence of noise, lasso loses efficiency for weighted networks. This paper builds a new framework to cope with this problem. The key idea i…
Unified theory linking Bayesian and ensemble methods in deep learning.
problem Uncertainty quantification in deep learning.
method Reformulating optimisation as convex optimisation in probability measures, studying Wasserstein gradient flows.
result Unified theory explaining success of deep ensembles over variational inference.
Variational Bayesian inference and (collapsed) Gibbs sampling are the two important classes of inference algorithms for Bayesian networks. Both have their advantages and disadvantages: collapsed Gibbs sampling is unbiased but is also inefficient for large count values and requires averaging over many samples to reduce …
VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.
problem Error accumulation in continual test-time adaptation.
method VCoTTA employs variational Bayesian techniques to update a Bayesian Neural Network (BNN) during testing, combining priors from source and teacher models.
result VCoTTA effectively mitigates error accumulation in CTTA, as shown by experimental results on three datasets.
Clarifies EM algorithm and variational Bayesian inference concepts.
problem Gaps in AI literature understanding of EM and variational concepts.
method Tutorial presentation of EM algorithm, variational Bayesian inference, and autoencoded variational Bayes.
result Establishes clear links between EM and variational methods.
Object tracking is an ubiquitous problem that appears in many applications such as remote sensing, audio processing, computer vision, human-machine interfaces, human-robot interaction, etc. Although thoroughly investigated in computer vision, tracking a time-varying number of persons remains a challenging open problem.…
Two novel distributed VB algorithms improve Bayesian inference in sensor networks.
problem Efficient inference in Bayesian frameworks for sensor networks.
method Two novel distributed VB algorithms for general Bayesian inference, using stochastic natural gradient and ADMM.
result Distributed algorithms perform nearly as well as centralized ones, demonstrating excellent performance.
Bayes-CATSI: A variational Bayesian deep learning framework for medical time series data imputationcs.LG
Bayes-CATSI uses variational Bayesian deep learning for medical time series data imputation.
problem Missing values in medical time series data.
method Bayes-CATSI integrates variational inference for uncertainty quantification and context-aware imputation.
result Bayes-CATSI outperforms CATSI by 9.57% in imputation performance.
Improved phylogenetic tree reconstruction using flexible branch length distributions.
problem Inefficient Markov chain Monte Carlo methods for large sequence datasets.
method Variational Bayesian phylogenetic inference with semi-implicit branch length distributions.
result Proposed method improves marginal likelihood estimation and branch length posterior approximation.
Estimating hidden processes from non-linear noisy observations is particularly difficult when the parameters of these processes are not known. This paper adopts a machine learning approach to devise variational Bayesian inference for such scenarios. In particular, a random process generated by the autoregressive moving…
S-VBMC improves VBMC's exploration of complex posterior distributions.
problem Efficient inference for computationally expensive models with complex posterior distributions.
method Stacking multiple independent VBMC runs to create a robust global posterior approximation.
result Significant improvements in posterior approximation quality across various applications.
Bayesian method improves online NARMAX model identification.
problem Online identification of nonlinear systems with small sample sizes and low noise.
method Variational Bayesian inference using message passing algorithm for polynomial NARMAX models.
result Variational Bayesian estimator outperforms recursive and offline least-squares methods.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
Bayesian approach generalizes ADMM for federated learning.
problem Improving federated learning efficiency and accuracy.
method Integrates Bayesian duality with ADMM for optimization.
result New extensions of ADMM for various distributions.
This note is concerned with an accurate and computationally efficient variational bayesian treatment of mixed-effects modelling. We focus on group studies, i.e. empirical studies that report multiple measurements acquired in multiple subjects. When approached from a bayesian perspective, such mixed-effects models typic…
The problem of low rank matrix completion is considered in this paper. To exploit the underlying low-rank structure of the data matrix, we propose a hierarchical Gaussian prior model, where columns of the low-rank matrix are assumed to follow a Gaussian distribution with zero mean and a common precision matrix, and a W…
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
Stochastic LWTA networks resist adversarial attacks while maintaining accuracy.
problem Adversarial robustness of neural networks.
method Replaced ReLU with stochastic LWTA activations, trained with Variational Bayesian and PGD.
result Stochastic LWTA networks achieve state-of-the-art robustness against adversarial attacks.
Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translationstat.ML
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
Two main obstacles preventing the widespread adoption of variational Bayesian neural networks are the high parameter overhead that makes them infeasible on large networks, and the difficulty of implementation, which can be thought of as "programming overhead." MC dropout [Gal and Ghahramani, 2016] is popular because it…
Nonnegative matrix factorization (NMF) with group sparsity constraints is formulated as a probabilistic graphical model and, assuming some observed data have been generated by the model, a feasible variational Bayesian algorithm is derived for learning model parameters. When used in a supervised learning scenario, NMF …
VALC provides concept-level interpretations of FLMs, overcoming word-level limitations.
problem Lack of higher-level structure interpretation in FLMs' attention weights.
method Formal definition of conceptual interpretation, variational Bayesian framework (VALC).
result VALC finds optimal language concepts for FLM predictions, providing concept-level interpretations.
Advances variational Bayesian neural networks using singular learning theory.
problem Discrepancies between predictive performance and variational objective in BNNs.
method Corrected asymptotic form of singular posterior distributions to inform variational family design.
result Improvements in variational free energy and generalization error with proposed normalizing flow.
A new method clusters data from multiple sources using a mixture of multilayer SBMs.
problem Aggregating multiple clustering results from different data sources.
method Uses a mixture of multilayer Stochastic Block Models (SBM) to group co-membership matrices.
result Identifies and clusters observations based on their specificities within components.
GeoPhy uses geometric gradients to efficiently infer phylogenetic trees from molecular data.
problem Challenges in accurately inferring species relationships from molecular data due to combinatorially vast tree topologies.
method Introduces a novel, fully differentiable formulation of phylogenetic inference using geometric spaces and variational Bayesian methods.
result Significantly outperforms other approximate Bayesian methods in inferring phylogenetic trees.
Stochastic VB improves nonlinear model inference speed and accuracy.
problem Bayesian inference of nonlinear models from noisy data.
method Stochastic Variational Bayesian (VB) inference for nonlinear models.
result Stochastic VB achieves comparable parameter recovery to analytical solution but is faster.
In this paper, a methodology is investigated for signal recovery in the presence of non-Gaussian noise. In contrast with regularized minimization approaches often adopted in the literature, in our algorithm the regularization parameter is reliably estimated from the observations. As the posterior density of the unknown…
Generative dynamic texture models (GDTMs) are widely used for dynamic texture (DT) segmentation in the video sequences. GDTMs represent DTs as a set of linear dynamical systems (LDSs). A major limitation of these models concerns the automatic selection of a proper number of DTs. Dirichlet process mixture (DPM) models w…
In this work we perform outlier detection using ensembles of neural networks obtained by variational approximation of the posterior in a Bayesian neural network setting. The variational parameters are obtained by sampling from the true posterior by gradient descent. We show our outlier detection results are comparable …
This paper presents an efficient Bayesian framework for solving nonlinear, high-dimensional model calibration problems. It is based on a Variational Bayesian formulation that aims at approximating the exact posterior by means of solving an optimization problem over an appropriately selected family of distributions. The…
Many probabilistic models of interest in scientific computing and machine learning have expensive, black-box likelihoods that prevent the application of standard techniques for Bayesian inference, such as MCMC, which would require access to the gradient or a large number of likelihood evaluations. We introduce here a n…
Predicting Cellular Responses with Variational Causal Inference and Refined Relational Informationcs.LG
Novel framework predicts cell responses to perturbations using GRNs.
problem Predicting cellular responses to perturbations for drug discovery and personalized therapeutics.
method Graph variational Bayesian causal inference framework with refined GRNs and robust estimator.
result Enhanced model performance and robust estimation of perturbation effects.
VBMC+VIQR outperforms noisy models in Bayesian inference.
problem Bayesian inference with noisy likelihoods in complex models.
method Gaussian process surrogates, expected information gain, variational interquantile range.
result VBMC+VIQR achieves state-of-the-art performance in noisy inference benchmarks.