Deep Bayesian neural nets can use simpler weight approximations without sacrificing performance.
problem The need for complex weight posterior approximations in deep Bayesian neural networks.
method Theoretical and empirical analysis of mean-field variational inference in deep networks.
result Mean-field variational weight posteriors in deep networks can induce similar function-space distributions as complex approximations in shallower networks.
A new algorithm optimizes Gaussian process posterior mean functions efficiently.
problem Optimizing Gaussian process posterior mean functions over hyperrectangles is challenging due to nonlinearity and nonconvexity.
method PALM-Mean, a piecewise-analytic lower-bounding framework embedded in reduced-space spatial branch-and-bound.
result PALM-Mean improves scalability for large datasets compared to general-purpose solvers.
We consider the problem of sequential learning from categorical observations bounded in [0,1]. We establish an ordering between the Dirichlet posterior over categorical outcomes and a Gaussian posterior under observations with N(0,1) noise. We establish that, conditioned upon identical data with at least two observatio…
A new ensemble filter uses transport maps and MMD optimization for high-dimensional data assimilation.
problem High-dimensional data assimilation challenges in ensemble filtering.
method Optimized Maximum Mean Discrepancy (MMD) for transport map construction.
result Significant improvement in robustness and posterior approximation.
Study shows TAP free energy minimization provides better posterior inference in high-dimensional linear models.
problem Deviation from true posterior mean and underestimation of posterior uncertainty in variational inference.
method Minimization of TAP free energy in a high-dimensional asymptotic framework, showing geometric and statistical properties.
result Local minimizer of TAP free energy provides consistent estimate of posterior marginals and correctly calibrated posterior inference.
Bayesian neural networks ignore data in infinite units limit.
problem Pathological behavior of posterior in over-parameterized networks.
method Mean-field variational inference in infinite hidden units limit.
result Posterior mean converges to zero, ignoring data.
The paper analyzes mean-field variational Bayes for complex models and proposes new uncertainty quantification methods.
problem Approximating posterior distributions in complex Bayesian models with latent variables.
method Non-asymptotic analysis on mean-field variational inference, showing that a normal distribution with the MLE center approximates the posterior well.
result The mean-field approximation matches the MLE up to higher-order terms and is essentially efficient for regular parametric models.
New method for faster, scalable inference in coupled Gaussian Processes.
problem Coupled Gaussian Processes require scalable inference methods for posterior uncertainty.
method Structured variational inference for multi-Gaussian Processes.
result Fast and scalable inference capturing posterior dependencies.
Sparse matrices simplify computation of GP variances and likelihoods.
problem Efficient computation of posterior variance and log-likelihood for additive Matérn GPs.
method Represented posterior mean, variance, log-likelihood, and gradient using sparse matrices.
result Efficient computation of posterior mean, variance, log-likelihood, and gradient in O(nlogn) time. Bayesian inference for inverse problems using mean-shift interacting particles
problem Bayesian inference for inverse problems
method Amortized mean-shift interacting particles
result Improves accuracy of Bayesian inference by reducing the number of samples needed
Geometric framework analyzes bias in variational inference for posterior functionals.
problem Analyzing the bias of posterior functionals under variational approximations.
method Developed a geometric framework to evaluate the bias of posterior functionals using the variational tangent space.
result The leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space.
MFVI can overestimate predictive variance compared to the exact posterior
problem MFVI underestimates posterior variance
method Analyzing conjugate Bayesian Linear Regression
result MFVI can overestimate predictive variance compared to the exact posterior
Robust VB framework handles contamination using min-max median aggregation.
problem Handling contamination and outliers in datasets.
method Partition data into subsets, formulate robust optimization problem, use min-max median KL divergence.
result Min-max median formulation improves robustness and statistical rates.
Compact parameterization improves Bayesian neural network performance.
problem Improving performance of Bayesian neural networks using variational methods.
method Restricting variational distribution to a k-tied Normal distribution with low-rank factorization.
result Compact parameterization improves signal-to-noise ratio and convergence speed.
VB approximates posterior mean perfectly in linear Gaussian VAR models.
problem Unknown approximation error of VB in VAR models.
method Derive approximation error in terms of mean, mode, variance, predictive density, and KL divergence.
result VB approximates posterior mean perfectly.
Paper addresses the disparity between sampled and mean representations in disentangled learning.
problem Disparity between sampled and mean representations in disentangled learning.
method Proposes a method to eliminate the disparity by proving and utilizing the relationship between total correlation of sampled and mean representations for multivariate normal distributions.
result Demonstrates that a factorized mean representation can have lower total correlation than the sampled representation.
Advocates for a new posterior that predicts better than classical and generalised Bayes.
problem Combining parameter inference and density estimation for better predictive models.
method Predictively Oriented (PrO) posterior using mean field Langevin dynamics.
result PrO posteriors converge to the predictively optimal model average, adapting to model misspecification.
Proposes a new method for posterior sampling using MMD with negative distance kernel.
problem Posterior sampling and conditional generative modeling.
method Approximates joint distribution using discrete Wasserstein gradient flows of MMD with negative distance kernel.
result Establishes an error bound for posterior distributions and proves the method is a Wasserstein gradient flow.
Paper proposes MMD-Bayes for robust Bayesian estimation in misspecified models.
problem Inconsistent estimates in Bayesian statistics due to model misspecification.
method Uses Maximum Mean Discrepancy (MMD) to create a robust pseudo-likelihood.
result MMD-Bayes posterior is consistent and robust to model misspecification.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Wide BNNs with odd activations fail to approximate data under mean-field inference.
problem Theoretical limitations of mean-field variational inference in wide, deep Bayesian neural networks.
method Analysis of mean-field variational inference in fully-connected BNNs with odd activation functions and Gaussian likelihood.
result The optimal mean-field variational posterior predictive distribution converges to the prior predictive distribution as network width increases.
KELFI improves inference accuracy in likelihood-free settings with limited simulations.
problem Intractable likelihood evaluations in likelihood-free inference.
method Kernel embedding likelihood-free inference (KELFI) learns model hyperparameters to balance accuracy and efficiency.
result Improved accuracy and efficiency on challenging inference problems in ecology.
FastMuyGPs speeds up GP predictions for large datasets.
problem High cost of Gaussian process predictions for large data.
method Combines cross-validation, batching, nearest neighbors sparsification, and precomputation.
result Superior accuracy and competitive runtime compared to other methods.
Max-Entropy approach improves variational inference for complex posterior distributions.
problem Efficient inference with simple families vs. accuracy in variational inference.
method Greedy approximation of the posterior distribution with Max-Entropy approach.
result Demonstrated ability to capture complex multimodal posterior distributions.
Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
Fuses posterior distributions from different datasets using KL divergence.
problem Combining information from multiple datasets with uncertainty.
method Mean field assumption, KL divergence, assign-and-average approach.
result Efficient non-parametric algorithm for fused model computation.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
New method quantifies uncertainty in denoising models.
problem Uncertainty quantification in denoising models.
method Derives a relation between posterior moments and derivatives, uses it for efficient uncertainty quantification.
result Efficient computation of principal components and full marginal distributions of the posterior.
Deep clustering models are shown to be equivalent to K-means under certain conditions.
problem The challenge of unsupervised deep learning and clustering.
method The study proves the equivalence of recent discriminative models and K-means under specific conditions and parameters.
result Maximizing the L2 regularized mutual information is equivalent to a soft and regularized K-means loss.
New algorithm for Gaussian process classification using posterior linearisation.
problem Improving Gaussian process classification performance.
method Posterior linearisation to approximate posterior density iteratively, accounting for linearisation error.
result PL has better performance than EP in experimental data.
New algorithm improves Bayesian inference for complex models.
problem Poor performance of existing Bayesian approaches for simulators.
method Posterior bootstrap and maximum mean discrepancy estimators.
result Strong robustness and parallelizability of the new algorithm.
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given i…
Kernel methods are one of the mainstays of machine learning, but the problem of kernel learning remains challenging, with only a few heuristics and very little theory. This is of particular importance in methods based on estimation of kernel mean embeddings of probability measures. For characteristic kernels, which inc…
Mean field Gaussian inference limits mutual information to regularize neural networks.
problem Understanding and quantifying the regularization effect of mean field Gaussian inference.
method Empirically observed and theoretically quantified mutual information limitation through noise.
result Bounding mutual information between parameters and data effectively regularizes neural networks.
Study shows mean-field approximation fails to improve PAC-Bayes bounds for neural networks.
problem Understanding why overparametrized neural networks achieve low risk and zero empirical risk.
method Optimized PAC-Bayes bounds using variational inference (VI), investigating mean-field approximation.
result Mean-field approximation does not provide significant improvements in PAC-Bayes bounds for neural networks.
DADVI improves ADVI by using deterministic approximation for faster, more accurate posterior estimation.
problem Intractable posterior uncertainty estimates and lack of clear convergence criteria in ADVI.
method Replaces stochastic MFVB objective with deterministic Monte Carlo approximation (SAA) and uses second-order optimization.
result DADVI provides faster and more accurate posterior estimates with default settings.
Bayesian method improves portfolio selection under uncertain parameters.
problem Optimal portfolio choice with unknown asset return parameters.
method Bayesian posterior predictive distribution for optimization.
result Bayesian approach yields better portfolio predictions and returns.
Unified framework combines views and optimization for better portfolio management.
problem Optimizing portfolio weights with dynamic adjustment based on volatility.
method Dynamic sliding window adjusting horizon, factor estimates, BL posterior returns, and weights over time.
result Outperforms dynamic mean-variance optimization without BL views, providing stronger downside risk control.
A new method eliminates miscalibration in Gaussian process models for dynamical systems.
problem Miscalibration and overestimation of transition function parameters in Gaussian process models.
method Explicitly models the dependence between state trajectories and Gaussian process posterior, eliminating factorization.
result Better predictive performance and more calibrated estimates of the transition function.
A new data-adaptive prior stabilizes kernel learning in operators.
problem Learning kernels in operators from data is ill-posed due to nonlocal dependence.
method Introduces a data-adaptive prior to stabilize the Bayesian posterior mean.
result The data-adaptive prior achieves a stable posterior with small noise limits.
Paper tackles model vulnerabilities by reconstructing training data.
problem Reconstructing training data from model parameters poses a security risk.
method Developed a mathematical framework and score matching method for both Bayesian and non-Bayesian models.
result First score matching framework for reconstructing data in Bayesian models.
We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace U and its estimate U^ may not be adequate as the MSE is not the natural metric in the Gra…
TM-VI uses flexible transformation models to approximate complex posteriors in Bayesian models.
problem Approximating complex posteriors in Bayesian models with limited flexibility.
method Transformation models for variational inference (TM-VI).
result TM-VI allows accurate approximation of complex posteriors in models with one parameter and works in a mean-field fashion for multi-parameter models.
Paper proposes a method to break symmetries in Bayesian matrix factorization.
problem Symmetries in posterior distribution reduce MCMC sampling efficiency.
method Modification to Gaussian prior mean and covariance to break symmetries.
result Breaking symmetries leads to lower autocorrelation and reconstruction errors.
Develops bounds on Bayesian posterior approximations using Fisher distance.
problem Lack of finite-sample theory for scalable inference methods.
method Bounding Wasserstein distance via generalized Fisher distance.
result Derives bounds on Wasserstein error for various approximations.
We study the relationship between online Gaussian process (GP) regression and kernel least mean squares (KLMS) algorithms. While the latter have no capacity of storing the entire posterior distribution during online learning, we discover that their operation corresponds to the assumption of a fixed posterior covariance…
Dynamic trees are mixtures of tree structured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorized di…
Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually…