We introduce Deep Variational Bayes Filters (DVBF), a new method for unsupervised learning and identification of latent Markovian state space models. Leveraging recent advances in Stochastic Gradient Variational Bayes, DVBF can overcome intractable inference distributions via variational inference. Thus, it can handle …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “Stochastic Gradient Variational Bayes”
This research explores using Alpha-Divergences in variational dropout for better inference.
problem Improving variational inference methods using alternative divergences.
method Extending the Stochastic Gradient Variational Bayes (SGVB) framework with Alpha-Divergences.
result The -divergence with yields the lowest training error and optimizes the ELBO.
RQMC improves optimization in variational Bayes problems.
problem Optimizing variational Bayes problems with noisy objective functions.
method Use of randomized quasi-Monte Carlo (RQMC) sampling with stochastic L-BFGS.
result RQMC can significantly speed up optimization and find better parameter values.
We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stocha…
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
Paper proposes MMD-Bayes for robust Bayesian estimation in misspecified models.
problem Inconsistent estimates in Bayesian statistics due to model misspecification.
method Uses Maximum Mean Discrepancy (MMD) to create a robust pseudo-likelihood.
result MMD-Bayes posterior is consistent and robust to model misspecification.
While stochastic variational inference is relatively well known for scaling inference in Bayesian probabilistic models, related methods also offer ways to circumnavigate the approximation of analytically intractable expectations. The key challenge in either setting is controlling the variance of gradient estimates: rec…
Backprop-Q extends standard backpropagation for stochastic computation graphs.
problem Applying standard backpropagation to stochastic computation graphs is challenging.
method Construct Q-functions for each stochastic node and use them to train the SCG with standard backpropagation.
result Generalized backpropagation for stochastic computation graphs is feasible and extends learning signals beyond gradients.
VB uses natural gradients in information geometry.
problem Estimating or computing natural gradients in VB.
method Natural-gradient descent algorithm and Bayesian Learning Rule.
result Simplification of Bayes' rule and generalization of quadratic surrogates.
Neural GARCH models financial time series with time-varying coefficients.
problem Modeling conditional heteroskedasticity in financial time series.
method Neural network adaptation of GARCH and BEKK models with time-varying coefficients parameterized by a recurrent neural network.
result Neural Students t model consistently outperforms other models on financial time series.
New method for better initializations in variational Bayes for deep models.
problem Effective initializations for stochastic variational inference in deep models.
method Layer-wise initialization strategy based on Bayesian linear models.
result Faster and better convergence compared to alternatives.
Quantum method speeds up VB estimation in machine learning.
problem Prohibitively expensive natural gradient in high dimensions.
method Regression-based natural gradient estimation with quantum matrix inversion.
result Quantum method enables efficient VB estimation.
Meta-learning framework improves model performance on few-shot classification tasks.
problem Improving model performance on few-shot classification tasks.
method Empirical Bayes formulation with synthetic gradients for transductive meta-learning.
result Meta-learning framework outperforms previous state-of-the-art methods on benchmarks.
Proposes a new model for directed graphs combining deep learning and latent variable models.
problem Graph representation learning for directed graphs.
method Deep Latent Space Model (DLSM) integrating GCN encoder and stochastic decoder with hierarchical variational auto-encoder architecture.
result Achieves state-of-the-art performance on link prediction and community detection tasks.
Novel framework discovers SPDEs from limited data.
problem Discovering SPDEs from limited data.
method Combines stochastic calculus, variational Bayes, and sparse learning.
result Accurately identifies SPDEs from limited data.
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.
Gradient-based optimization improves variational empirical Bayes regression.
problem Sparse, large-scale multiple regression models.
method Gradient-based optimization (GradVI) for variational empirical Bayes (VEB) regression.
result GradVI produces similar predictive performance to CAVI but converges faster and is faster in certain settings.
Draft proposes adapting neural networks to match naive Bayes classifiers.
problem Bridge between neural networks and naive Bayes classifiers.
method Class-conditional compression and disentanglement using variational bounds.
result Latent representations enable naive Bayes classifier performance.
Bayesian method approximates intractable stochastic programs with chance constraints.
problem Designing systems with stochastic constraints and chance constraints.
method Variational Bayesian approach to approximate posterior predictive integral.
result The solution set converges to the true solution set as the number of observations increases.
A neural model samples arbitrary features conditioned on some inputs.
problem Sampling arbitrary features conditioned on some inputs.
method Variational autoencoder with stochastic variational Bayes training.
result Effectiveness and diversity of generated samples on synthetic data and feature problems.
New algorithm optimizes PAC-Bayes bound without surrogate loss.
problem Mismatch between optimisation objective and generalisation bound in stochastic neural networks.
method Proposes a novel training algorithm that optimizes the PAC-Bayesian bound directly.
result Empirical results show improved performance over existing PAC-Bayesian training methods.
New method makes variational inference robust for Bayesian neural networks.
problem Fragility of variational Bayes in neural networks.
method Deterministic approximation of moments and hierarchical prior selection.
result Good predictive performance in heteroscedastic regression.
Wasserstein Gaussianization and Efficient Variational Bayes for Robust Bayesian Synthetic Likelihoodstat.CO
A new method improves likelihood-free Bayesian inference by transforming summary statistics and using efficient Variational Bayes.
problem Incorrectly assuming normally distributed summary statistics in likelihood-free Bayesian inference.
method Wasserstein Gaussianization transformation combined with robust BSL and efficient Variational Bayes.
result Highly efficient and reliable approximate Bayesian inference for likelihood-free problems.
How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild dif…
Unified empirical and variational Bayes for unnormalized densities.
problem Approximating unnormalized densities using latent variable models.
method Formulate a latent variable model for , use ELBO as parametrization of 's energy function, and estimate with empirical Bayes least-squares.
result UVB has higher capacity to approximate energy functions than MLPs in DEEN.
Streaming Variational Bayesstat.ML
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) …
Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.
problem Sparse logistic regression challenges in machine learning.
method Empirical Bayes approach with mean-field variational inference, tuning-free and scalable.
result Superior predictive performance in sparse logistic regression compared to existing methods.
Deep model predicts robot trajectories in real time.
problem Real-time robot trajectory prediction with low latency.
method Deep conditional generative model trained with SGDVB.
result More accurate long-term predictions with lower latency.
Lower bounds show many sampling algorithms need many gradient queries.
problem Sampling from strongly log-concave densities in high dimensions.
method Information theory and stochastic gradient methods.
result Lower bound on number of gradient queries needed.
WHAI combines autoencoding and MCMC for scalable topic modeling.
problem Training scalable deep topic models for big corpora.
method Develops a hybrid autoencoding inference network for deep latent Dirichlet allocation.
result WHAI achieves both scalability and speed in topic inference.
Bayesian surrogate models reduce uncertainty in high-dimensional design optimisation problems.
problem Uncertainty in high-dimensional inputs for complex computational models.
method Variational Bayesian inference for constructing statistical surrogates with Gaussian process priors and KL divergence for approximation.
result The RDVGP surrogate provides accurate and versatile approximations for robust structural optimisation.
SMI uses mixture models to improve SVGD's performance in Bayesian inference.
problem Variance collapse in SVGD for Bayesian inference, especially with small models.
method Generalizes SVGD to Stein mixture models, optimizing an ELBO lower bound.
result SMI avoids variance collapse and accurately estimates uncertainty for small BNNs.
New research shows existing information-theoretic methods can't establish minimax rates for gradient descent in stochastic convex optimization.
problem Establishing minimax rates for gradient descent in stochastic convex optimization using information-theoretic methods.
method Examined several information-theoretic frameworks including input-output mutual information bounds, conditional mutual information bounds, PAC-Bayes bounds, and their variants.
result Proved that none of the examined information-theoretic frameworks can establish minimax rates for gradient descent in stochastic convex optimization.
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
Advanced Dropout improves DNN performance without requiring model-specific dropout techniques.
problem Overfitting in deep neural networks due to lack of data.
method Model-free, end-to-end training with adaptive dropout rate optimization.
result Advanced Dropout outperforms other techniques on various datasets.
Fast variational Bayes methods improve geospatial data analysis speed and accuracy.
problem Inaccurate and slow variational Bayes methods for large geospatial data.
method Combination of calculus of variations, closed-form gradient updates, and linear response corrections.
result Comparable accuracy to spNNGP with reduced computational costs and faster speed.
Proposes MOPED method for choosing priors in Bayesian DNNs.
problem Challenges in specifying meaningful priors for deep neural networks.
method Two-stage hierarchical modeling with empirical Bayes.
result MOPED enables scalable variational inference and reliable uncertainty quantification.
BONG optimizes Bayesian inference online with natural gradient descent.
problem Sequential Bayesian inference in online settings.
method Bayesian online natural gradient (BONG) approach based on variational Bayes.
result BONG outperforms other online VB methods in non-conjugate settings.
A new model predicts multivariate regression using similarities to data points.
problem Complex, high-dimensional input-output relationships.
method Bayesian mixture-of-experts with conditional Gaussian mixtures and variational Bayes.
result Outperforms competitors in high-dimensional settings.
Black-box alpha (BB-) is a new approximate inference method based on the minimization of -divergences. BB- scales to large datasets because it can be implemented using stochastic gradient descent. BB- can be applied to complex probabilistic models with little effort since it only requires as input the likel…
SVIGL speeds up convergence in stochastic variational inference.
problem Optimizing log-posterior in random field models is difficult.
method Stochastic variational inference with gradient linearization.
result SVIGL improves convergence speed with comparable KL divergence.
Unified framework connects stochastic optimization to Bayesian inference.
problem Stochastic optimization algorithms and their theoretical underpinnings.
method Latent variational problem and Forward Backward Stochastic Differential Equations (FBSDE).
result Recovery of various adaptive stochastic gradient descent methods.
Proposes VSGD optimizer combining probabilistic and gradient-based methods.
problem Uncertainty modeling in deep neural networks.
method Combines probabilistic and gradient-based approaches using SVI.
result VSGD outperforms Adam and SGD on image classification tasks.
Flexible online learning framework for neural dynamics.
problem Learning latent neural state and dynamics from complex neural recordings.
method Stochastic gradient variational Bayes approach for joint optimization.
result Framework can optimize nonlinear dynamical system, observation model, and recognition model.
New method improves training stochastic neural networks with tighter guarantees.
problem Training stochastic neural networks with provable guarantees.
method Developed partially-aggregated estimators and reformulated PAC-Bayesian bounds.
result Derives a differentiable objective leading to tighter generalisation guarantees.
Empirical Bayes rates via variational approximations and prior decomposition.
problem Nonparametric and high-dimensional inference convergence rates.
method Variational perspective and prior decomposition.
result Empirical Bayes posterior rates derived from variational Bayes.
Variational inference approximates the posterior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solutions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approximation for the gradients. This enables variatio…
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.