Sparse variational posteriors improve clustering and topic modeling speed.
problem Inefficient storage and runtime costs for standard variational posteriors.
method Sparse variational distributions with tunable threshold L. result Moderate values of L>1 provide superior performance in clustering and topic modeling. Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
Post-process Bayesian inference speeds up posterior approximation.
problem Leveraging pre-existing model evaluations for quick posterior approximation.
method Variational Sparse Bayesian Quadrature (VSBQ) using sparse Gaussian process (GP) surrogate model.
result VSBQ builds high-quality posterior approximations from existing optimization traces.
New method for faster, scalable inference in coupled Gaussian Processes.
problem Coupled Gaussian Processes require scalable inference methods for posterior uncertainty.
method Structured variational inference for multi-Gaussian Processes.
result Fast and scalable inference capturing posterior dependencies.
Improved sparse Gaussian processes using structured scaling matrices and Power-EP framework.
problem Scaling Gaussian processes for large datasets.
method Structured diagonal scaling matrix and Power-EP framework.
result Structured approximations improve performance without increasing computational cost.
Paper tightens variational GP approximations for large datasets.
problem Scaling Gaussian processes to large datasets.
method Relaxing the standard assumption about inducing points' posterior matching the prior, leading to a tighter variational approximation.
result The proposed approximation consistently matches or outperforms standard sparse variational GPs while maintaining computational cost.
Paper proposes a method to improve variational inference for sparse networks.
problem Variational inference struggles with sparse networks, leading to inaccurate community detection.
method The method involves hard thresholding the posterior of community assignment after each iteration.
result The proposed method accurately recovers true community labels in sparse networks.
A new method for efficient Gaussian process inference using sparse approximations.
problem Scalable and accurate inference for latent Gaussian processes.
method Variational approximation with sparse inverse Cholesky factors and double Kullback-Leibler minimization.
result The proposed method can achieve highly accurate approximations with polylogarithmic time complexity.
New method DDVI improves posterior inference for deep Gaussian processes.
problem Inference of inducing points in DGPs is challenging and biased.
method DDVI uses denoising diffusion SDE and score matching for posterior approximation.
result Empirically shows DDVI outperforms baseline methods in inducing point inference.
Many problems of low-level computer vision and image processing, such as denoising, deconvolution, tomographic reconstruction or super-resolution, can be addressed by maximizing the posterior distribution of a sparse linear model (SLM). We show how higher-order Bayesian decision-making problems, such as optimizing imag…
Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.
problem Sparse logistic regression challenges in machine learning.
method Empirical Bayes approach with mean-field variational inference, tuning-free and scalable.
result Superior predictive performance in sparse logistic regression compared to existing methods.
Proposes a Bayesian approach for automatic node selection in sparse neural networks.
problem Reduces structural complexity and computational speedup in large-scale predictive models.
method Uses spike-and-slab Gaussian priors and variational Bayes approach for node selection.
result Establishes variational posterior consistency and optimal contraction rates for sparse networks.
A new knot selection method speeds up sparse Gaussian process approximations.
problem Efficiently selecting knots for sparse Gaussian processes.
method One-at-a-time Bayesian optimization for knot selection.
result Competitive performance with reduced computational cost.
Elliptical processes extend Gaussian models with heavier tails.
problem Regression and classification with non-Gaussian likelihoods or heavy tails.
method Spline normalizing flow for variational inference of elliptical distributions.
result Elliptical processes outperform Gaussian processes in non-Gaussian settings.
New model learns deep, sparse hierarchies of features from images.
problem Learning deep, sparse hierarchies of features from natural images.
method Structured variational auto-encoder with rectified Gaussian units.
result Joint training of deep models with many layers of latent variables.
New method improves GP regression by relaxing variational assumption.
problem Improving variational Gaussian processes for better predictive performance.
method Relaxing the variational assumption to a more general distribution for optimization.
result New tighter evidence lower bound for GP regression.
Paper develops efficient variational inference for sparse deep learning with theoretical guarantees.
problem Sparse deep learning's challenge of huge storage consumption and sparse structure recovery.
method Bayesian treatment with spike-and-slab priors and continuous relaxation of Bernoulli distribution for computationally efficient variational inferences.
result Provides variational posterior contraction rate, justifying consistency of the proposed method.
A new method for Bayesian neural networks using probabilistic backpropagation.
problem Approximating posterior distributions in Bayesian neural networks.
method Variational Expectation Propagation (VEP) with probabilistic backpropagation.
result Efficient algorithm for approximate integration over posterior distributions.
Novel method for SDE calibration from sparse data using neural flows.
problem Calibrating SDEs from sparse, noisy observations.
method Characterization of posterior SDE using neural networks trained to solve a PDE with multiplicative updates.
result Significant improvement in scalability and accuracy compared to classical methods.
Paper proposes VAE-BPTF for better tensor factorization of sparse, imbalanced count data.
problem Inference of Bayesian Poisson-Gamma models for sparse and imbalanced count data is challenging.
method Variational auto-encoder framework with multi-layer perceptron networks for complex update information sharing and reweighting.
result VAE-BPTF outperforms current models in reconstruction errors and latent factor coherence across real-world datasets.
Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate co…
Empirical Bayes rates via variational approximations and prior decomposition.
problem Nonparametric and high-dimensional inference convergence rates.
method Variational perspective and prior decomposition.
result Empirical Bayes posterior rates derived from variational Bayes.
Efficiently infers Poisson process intensity using Gaussian process with sigmoid link.
problem Estimating intensity of inhomogeneous Poisson processes efficiently.
method Variational free-form mean field optimization and sparse Laplace's method.
result Method is one order of magnitude faster than exact inference and competitive with quadratic link function models.
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
We study inference and learning based on a sparse coding model with `spike-and-slab' prior. As in standard sparse coding, the model used assumes independent latent sources that linearly combine to generate data points. However, instead of using a standard sparse prior such as a Laplace distribution, we study the applic…
New variational family approximates non-Gaussian posteriors efficiently.
problem Approximating non-Gaussian posteriors in Bayesian models.
method Copula-like variational distributions with efficient sampling and normalizing flows.
result The proposed method performs comparably to state-of-the-art approximations.
Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…
Paper shows variational inference works well for sparse deep learning models.
problem Generalization of sparse deep learning models using variational inference.
method Theoretical analysis linking variational inference to Bayesian and nonparametric regression.
result Near-minimax rates of convergence for Hölder smooth functions in sparse deep learning.
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
problem Sparse high-dimensional logistic regression model selection.
method Spike and slab variational Bayes approximation.
result Optimal convergence rates in ℓ2 and prediction loss for sparse truths. Sparse Gaussian Processes simplify GP inference for large datasets.
problem Efficiently handling large datasets in Gaussian Process models.
method Sparse Gaussian Processes combined with variational inference.
result Sparse GPs enable approximate inference with reduced memory and computational requirements.
Sparse precision matrices in Gaussian variational approximations for high-dimensional models.
problem Learning posterior distributions with high-dimensional parameters and conditional independence structures.
method Sparse precision matrix parameterization and efficient stochastic gradient optimization methods.
result Flexibility and parsimony in Gaussian variational distributions achieved through sparsity in precision matrices.
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
New entropy-based objective for sparse coding improves learning.
problem Sparse coding with probabilistic priors and non-Gaussian observables.
method Derive a solely entropy-based learning objective for sparse coding parameters.
result Fully analytical ELBO objective for sparse coding with non-trivial posterior approximations.
Bayesian ODEs with Gaussian processes infer unknown dynamics from data.
problem Estimating unknown continuous-time system dynamics from data.
method Bayesian nonparametric model using Gaussian processes, sparse variational inference, probabilistic shooting.
result Posterior predictive uncertainty scores outperform alternative methods on multiple ODE learning tasks.
Develops an online learning framework for Bayesian joint filtering.
problem Streaming inference of nonlinear state-space models.
method Variational inference and sequential Monte Carlo.
result Efficient approximation of filtering posterior for a wide class of models.
New algorithm selects multiple kernels for better GP regression predictions.
problem Improving Gaussian process regression accuracy with multiple kernels.
method Variational Bayesian kernel selection (VBKS) for sparse Gaussian process regression (SGPR).
result VBKS learns uncertainty in kernel selection for better predictions.
Extends GP models for sequential data, scalable and robust.
problem Handling sequential input-output observations in multi-task settings.
method Variational inference with sparse approximations and recursive GP priors.
result Tractable continual learning with KL divergences and recursive reconstruction.
Improved Thompson Sampling for high-dimensional sparse bandits.
problem Stochastic linear contextual bandits with high-dimensional features.
method Thompson Sampling with spike-and-slab priors and variational inference.
result Nearly optimal upper bound on expected cumulative regret.
Sparse Gaussian Processes improve scalability by learning inducing points from data.
problem Scaling issues in Gaussian Processes due to cubic computational cost.
method Amortized learning of inducing points and variational posterior parameters using neural networks.
result Significant reduction in the number of inducing points, improving scalability.
TADDAA improves accuracy diagnostics for variational approximations.
problem Challenges in evaluating the accuracy of variational approximations.
method Uses many short parallel MCMC chains to obtain lower bounds on the error of each posterior functional of interest.
result Validates the practical utility and computational efficiency of TADDAA on various models.
Recently, a number of mostly ℓ1-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a Bayesian perspective, this task is equivalent to maximum a posteriori probability estimation under a …
The paper tackles approximate unlearning from a subset of training data using variational inference.
problem Unlearning from a small subset of erased training data while maintaining the posterior belief from the full data.
method Formulates unlearning as minimizing KL divergence, equivalent to minimizing an evidence upper bound. Uses variational inference to approximate posterior beliefs and proposes two tricks to handle challenges.
result Demonstrates the effectiveness of the proposed unlearning methods on various Bayesian models.
New method for fast, accurate Gaussian process regression with data guarantees.
problem Slow and unreliable GP inference methods for nonparametric regression.
method Developed a novel objective function (preconditioned Fisher divergence) for scalable approximate GP regression with finite-data guarantees.
result Minimizing the pF divergence provides pointwise mean and variance estimates with tight 2-Wasserstein distance bounds and comparable empirical performance to variational sparse GPs.
Sparse representations have proven their efficiency in solving a wide class of inverse problems encountered in signal and image processing. Conversely, enforcing the information to be spread uniformly over representation coefficients exhibits relevant properties in various applications such as digital communications. A…
Efficiently infers Gaussian process density models with Gibbs sampling and variational methods.
problem Density estimation for complex, nonparametric models.
method Augmented likelihood with latent variables, Gibbs sampling, and variational mean field approximations.
result Efficient inference for Gaussian process density models with up to thousands of data points.
Scalable multi-task regression via sparse Gaussian process priors.
problem Efficiently modeling and predicting multiple related tasks.
method Direct Cholesky factorization for sparse parameterization of Gaussian process priors.
result Sparse parameterization improves scalability and accuracy in multi-task regression.
We improve DGP models by using importance-weighted variational inference for better accuracy.
problem Accurate modeling of non-Gaussian marginals in deep Gaussian processes.
method Introduced noisy latent covariates and an importance-weighted objective for variational inference.
result The importance-weighted objective consistently outperforms classical variational inference, especially for deeper models.
Improved Gaussian process approximations reduce computational cost.
problem Efficiently approximating Gaussian process posteriors for large datasets.
method Characterized KL divergence behavior and derived a rule for increasing inducing variables.
result For regression with normally distributed inputs, M=O(logDN) is sufficient to ensure small KL divergence.