A new method combines Laplace and Variational Bayes for scalable inference.
problem Complex models and large datasets make exact inference infeasible.
method Low-Rank Variational Bayes Correction (VBC) using Laplace method and Variational Bayes correction in a lower dimension.
result The method ensures scalability in both model complexity and data size.
Variational Laplace improves Bayesian neural networks performance.
problem Improving Bayesian neural networks performance.
method Develops variational Laplace for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms other inference methods.
Variational Laplace improves Bayesian neural network performance without sampling.
problem Improving Bayesian neural network performance and calibration.
method Develops a new variational Laplace method for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms standard VI methods in test performance and calibration.
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Derives a formula for the second variation of the Laplace eigenvalue functional on manifolds.
problem Calculating the second variation of the Laplace eigenvalue functional on closed manifolds.
method Derives a scale-invariant second variation formula for the Laplace eigenvalue functional.
result Proves that the canonical flat metric on a torus is not a maximal point of the functional in its conformal class.
The Laplace approximation has been one of the workhorses of Bayesian inference. It often delivers good approximations in practice despite the fact that it does not strictly take into account where the volume of posterior density lies. Variational approaches avoid this issue by explicitly minimising the Kullback-Leibler…
This thesis disentangles Gauss-Newton and variational approximations in Bayesian deep learning.
problem Understanding the interplay between the Gauss-Newton method and variational approximations in Bayesian deep learning.
method Analysis of the Gauss-Newton method and Laplace/Gaussian variational approximations for neural networks.
result The combination of the Gauss-Newton method with approximate inference can be cast as inference in a linear or Gaussian process model.
The paper extends graph-based semi-supervised learning to infinite-dimensional Wasserstein space.
problem Graph-based semi-supervised learning in high-dimensional data.
method Laplace Learning in the Wasserstein space, proving variational convergence and characterizing the Laplace-Beltrami operator.
result Consistent classification performance in high-dimensional settings.
Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.
problem Improving meta-reinforcement learning by providing full posterior distributions.
method Augmenting point estimates with Laplace approximation for full posterior distributions.
result Our method performs similarly to variational baselines with fewer parameters.
We study the existence and properties of metrics maximising the first Laplace eigenvalue among conformal metrics of unit volume on Riemannian surfaces. We describe a general approach to this problem and its higher eigenvalue versions via the direct method of calculus of variations. The principal results include the gen…
Variational approaches to approximate Bayesian inference provide very efficient means of performing parameter estimation and model selection. Among these, so-called variational-Laplace or VL schemes rely on Gaussian approximations to posterior densities on model parameters. In this note, we review the main variants of …
Several numerical approximation strategies for the expectation-propagation algorithm are studied in the context of large-scale learning: the Laplace method, a faster variant of it, Gaussian quadrature, and a deterministic version of variational sampling (i.e., combining quadrature with variational approximation). Exper…
Revisits online Laplace methods for neural networks, showing they are sound under certain conditions.
problem Online Laplace methods violate the Laplace approximation's critical assumption.
method Re-derives online Laplace methods, showing they target a variational bound on a mode-corrected variant of the Laplace evidence.
result Online Laplace and its mode-corrected counterpart share stationary points that satisfy the Laplace method's assumption.
We compare the eigenvalues of the Dirac and Laplace operator on a two-dimensional torus with respect to the trivial spin structure. In particular, we compute their variation up to order 4 upon deformation of the flat metric, study the corresponding Hamiltonian and discuss several families of examples.
Method for initializing Gaussian mixtures for variational inference with multi-modal distributions.
problem Challenges in variational inference with Gaussian mixtures due to multimodality and nonconvex loss functions.
method Optimization to find local maxima, local Gaussian approximations, and constrained least squares regression.
result Robust initialization improves variational inference performance and scalability.
We derive the explicit formula for the joint Laplace transform of the Wishart process and its time integral which extends the original approach of Bru. We compare our methodology with the alternative results given by the variation of constants method, the linearization of the Matrix Riccati ODE's and the Runge-Kutta al…
We establish the existence of analytic curves of eigenvalues for the Laplace-Neumann operator through an analytic variation of the metric of a compact Riemannian manifold M with boundary by means of a new approach rather than Kato's method for unbounded operators. We obtain an expression for the derivative of the cur…
New VAE model improves data fitting without sacrificing computational efficiency.
problem Limitation of Gaussian assumption in VAE for continuous variable fitting.
method Infinite mixture of asymmetric Laplace distribution in decoder, nonparametric M-estimator for quantile estimation.
result Model demonstrates superior data privacy adjustment and better distribution fitting.
New CRM models for sparse networks with linear edge growth.
problem Modeling extremely sparse networks with tractable properties.
method Introduced a new class of CRMs with index of variation α∈(0,1] based on mixtures of stable or generalized gamma processes.
result Models produce networks with near-linear edge growth, aligning with empirical evidence.
So-called sparse estimators arise in the context of model fitting, when one a priori assumes that only a few (unknown) model parameters deviate from zero. Sparsity constraints can be useful when the estimation problem is under-determined, i.e. when number of model parameters is much higher than the number of data point…
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
We use the time real analyticity of Ricci flow proved by Kotschwar to extend a result in ~\cite{B}, namely, we prove that the Laplace spectra of negatively curved compact surfaces having same genus γ≥2, same area and same curvature bounds vary in a "controlled way", of which we give a quantitative estimate (Theor…
Unified method to compute Laplace spectra on homogeneous principal bundles.
problem Computing the Laplace-Beltrami spectrum on homogeneous principal bundles.
method Unified representation-theoretic approach using generalized canonical variations and spectral branching criterion.
result Explicit formulas for the full spectra of several geometric families.
Variational method for eigenvalues on manifolds.
problem Optimizing functionals involving eigenvalues of Riemannian manifolds.
method New Palais-Smale sequences and min-max methods for locally-Lipschitz functionals.
result Convergence of Palais-Smale sequences in Laplace and Steklov eigenvalues.
Laplace approximation improves deep learning efficiency without sacrificing performance.
problem Bayesian deep learning's practical implementation and efficiency.
method Review and implementation of Laplace approximation (LA) in PyTorch.
result Laplace approximation is competitive with popular alternatives in performance but significantly more efficient.
Fast approximate inference for non-Gaussian data.
problem Efficient inference for non-Gaussian data.
method Laplace Matching for fast approximate inference in latent Gaussian models.
result Achieves high approximation quality with low computational cost.
Researchers prove spectral rigidity of Liouville tori under specific conditions.
problem Spectral rigidity of Liouville tori under generic conformal classes.
method Noncancellation of wave trace and analysis of second order variational formula for energy.
result Laplace isospectral deformations of Liouville metrics on torus are trivial.
Bayesian online meta-learning framework tackles catastrophic forgetting in few-shot classification.
problem Catastrophic forgetting in few-shot classification problems.
method Bayesian online learning, meta-learning, Laplace approximation, variational inference.
result Framework effectively achieves goal of overcoming catastrophic forgetting in few-shot classification.
Bayesian unlearning uses Bayes' rule to remove data from a model, but faces challenges in obtaining the exact posterior.
problem Removing data from a trained model while maintaining model accuracy.
method Uses Laplace approximation and Variational Inference to approximate the updated posterior.
result Insights on the applicability of Bayesian unlearning in practical scenarios for neural networks.
Study eigenvalues and shapes, proving sharp inequalities for Steklov eigenvalues.
problem Eigenvalue continuity and shape optimization for Laplace and Steklov problems.
method Variational eigenvalue analysis, Sobolev space convergence, shape optimization techniques.
result Sharp isoperimetric inequalities for Steklov eigenvalues, upper bound 8πk for k-th perimeter-normalized eigenvalue. Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
Bayesian PINNs optimize loss weights for PDEs and data.
problem Optimizing loss weights in physics-informed neural networks.
method Laplace approximation for efficient model evidence computation.
result Unified Bayesian setting for PDEs and noisy measurements.
The Hurwitz space is the moduli space of pairs (X,f) where X is a compact Riemann surface and f is a meromorphic function on X. We study the Laplace operator Δ∣df∣2 of the flat singular Riemannian manifold (X,∣df∣2). We define a regularized determinant for Δ∣df∣2 and study it as a functional on t…
The paper develops a new approach to solve vector-valued PDEs on manifolds with minimal regularity.
problem Well-posedness and Lp-based Sobolev regularity of vector-valued PDEs on compact manifolds. method Develops a parametrization-free variational approach using classical results in reflexive Banach spaces.
result Establishes higher-order Wm,p regularity for vector-valued PDEs on manifolds of minimal regularity. For the dual operator sg′∗ of the linearization sg′ of the scalar curvature function, it is well-known that if kersg′∗=0, then sg is a non-negative constant. In particular, if the Ricci curvature is not flat, then sg/(n−1) is an eigenvalue of the Laplacian of the metric g. In this work, some…
Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily …
A new method for efficient uncertainty estimation in deep learning models.
problem High computational costs in uncertainty estimation for deep neural networks.
method Variational sparse Gaussian Process approximation of the Linearized Laplace Approximation.
result Sub-linear training time and improved performance compared to existing methods.
Formula derived for Laplace-Beltrami spectrum on homogeneous spaces.
problem Calculating the spectrum of the Laplace-Beltrami operator on homogeneous spaces.
method Formula derivation based on eigenvalues of a generalized Casimir operator and spherical representations.
result First detailed computation and investigation of the spectrum for a family of metrics on the Aloff-Wallach manifold.
New algorithm radVI improves variational inference by optimizing radial profiles.
problem Gaussian approximations often fail to capture the radial profile of complex distributions.
method Optimizes over radial profiles in variational inference, providing theoretical guarantees.
result Theoretical convergence guarantees for radVI, improving over existing VI methods.
The paper presents an evolutionary economic model for the price evolution of stocks. Treating a stock market as a self-organized system governed by a fast purchase process and slow variations of demand and supply the model suggests that the short term price distribution has the form a logistic (Laplace) distribution. T…
In the paper "On Truncated Variation of Brownian Motion with Drift" (Bull. Pol. Acad. Sci. Math. 56 (2008), no.4, 267 - 281) we defined truncated variation of Brownian motion with drift, Wt=Bt+μt,t≥0, where (Bt) is a standard Brownian motion. Truncated variation differs from regular variation by neglect…
Post-hoc uncertainty quantification improves on pre-trained neural networks without underfitting.
problem Uncertainty quantification in neural networks is underfitting or computationally demanding.
method Gaussian Process Activation function (GAPA) for neuron-level uncertainty, with two methods: GAPA-Free and GAPA-Variational.
result GAPA-Variational outperforms Laplace approximation on most datasets in uncertainty quantification metrics.
MCMC complexity matches optimization for large n and d.
problem Lack of theoretical understanding of MCMC complexity for large n and d. method Comparison of MCMC, LA, and VI complexities for linear, logistic, and Poisson regression.
result MCMC complexity matches optimization complexity for n≳d. Paper confirms Yau's conjecture about sphere eigenvalues.
problem Yau's conjecture on eigenvalues of minimal hypersurfaces.
method Constructing a minimizing sequence in Sobolev space, using variational principle.
result First non-zero eigenvalue equals hypersurface dimension.
In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice of penalty weight which is often found from empirical experiments. We propose an algorithm called t…
New method for hyperparameter tuning in sparse matrix factorization.
problem Hyperparameter tuning in sparse matrix factorization.
method Numerical method based on evaluating the zero point of normalization factor in sparse matrix prior.
result Our method outperforms existing algorithms in ground-truth sparse matrix reconstruction.
Bayesian framework for sphere regression using Gaussian fields.
problem Nonparametric regression on the sphere with Gaussian priors.
method Isotropic Gaussian field priors, harmonic structure, exact posterior distributions, optimal spectral truncation, posterior contraction rates.
result Sharp posterior contraction rates for Gaussian priors with polynomially decaying angular power spectra.
Let x:M→Em be an isometric immersion of a Riemannian manifold M into a Euclidean m-space. Denote by Δ the Laplace operator of M. Then Δ gives rise to a differentiable map L:M→Em, called the Laplace map, defined by L(p)=(Δx)(p), p∈M. We call L(M) the Laplace image, and the transformat…