The paper analyzes how Gaussian kernel parameters affect posterior covariance in Gaussian processes.
problem Understanding the influence of Gaussian kernel parameters on posterior covariance in Gaussian processes.
method Geometric analysis and a posteriori error estimation techniques from adaptive finite element methods.
result The bandwidth parameter and spatial distribution of observations significantly influence posterior covariance and its matrix.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
Bayesian neural networks approximate Gaussian, this method adapts to non-Gaussian posteriors.
problem Bayesian neural networks struggle with non-Gaussian posteriors, leading to poor performance.
method Proposes a Riemannian Laplace approximation to adapt to the shape of the true posterior.
result Consistently improves over conventional Laplace approximation across tasks.
We consider the problem of sequential learning from categorical observations bounded in [0,1]. We establish an ordering between the Dirichlet posterior over categorical outcomes and a Gaussian posterior under observations with N(0,1) noise. We establish that, conditioned upon identical data with at least two observatio…
New method efficiently samples Gaussian process posteriors without cubic scaling.
problem Intractable posterior distributions in Gaussian processes.
method Decomposes Gaussian processes into prior and data components for scalable sampling.
result Fast posterior sampling at a fraction of the usual cost.
Proposes sampling from reverse diffusion posteriors for contextual bandits.
problem Complex distributions in contextual bandits.
method Approximate posterior sampling with a diffusion model prior using Laplace approximation.
result Empirically consistent and efficient approximations for contextual bandits.
Bayesian framework for sphere regression using Gaussian fields.
problem Nonparametric regression on the sphere with Gaussian priors.
method Isotropic Gaussian field priors, harmonic structure, exact posterior distributions, optimal spectral truncation, posterior contraction rates.
result Sharp posterior contraction rates for Gaussian priors with polynomially decaying angular power spectra.
New method for Bayesian neural networks with unbounded weights.
problem Posterior inference for Bayesian neural networks with unbounded weights.
method Conditionally Gaussian representation for efficient posterior inference.
result Interpretable and computationally efficient procedure for posterior inference.
Enhanced Gaussian process models accelerate optimization and posterior approximation.
problem Improving the accuracy and speed of Gaussian process models for optimization and inference.
method Introduces a random exploration step to classical GP-UCB algorithms, facilitating faster convergence.
result New algorithms achieve nearly optimal convergence rates and provide bounds for Hellinger distance.
Bayesian ODEs with Gaussian processes infer unknown dynamics from data.
problem Estimating unknown continuous-time system dynamics from data.
method Bayesian nonparametric model using Gaussian processes, sparse variational inference, probabilistic shooting.
result Posterior predictive uncertainty scores outperform alternative methods on multiple ODE learning tasks.
Gaussian process regression helps approximate Bayesian inverse problems efficiently.
problem Computational intractability of Bayesian posterior distributions in inverse problems.
method Gaussian process regression to build a surrogate model for the likelihood.
result Error between true and approximate posterior can be bounded by weighted L2-norm error between true and approximate likelihood. This paper addresses error bounds and posterior variance for Gaussian process regression.
problem Deriving performance guarantees for Gaussian process regression without prior knowledge.
method Lipschitz continuity and analysis of posterior variance function.
result Uniform error bounds for Gaussian process regression are derived.
We utilize copulas to constitute a unified framework for constructing and optimizing variational proposals in hierarchical Bayesian models. For models with continuous and non-Gaussian hidden variables, we propose a semiparametric and automated variational Gaussian copula approach, in which the parametric Gaussian copul…
This paper proposes a new algorithm for Gaussian process classification based on posterior linearisation (PL). In PL, a Gaussian approximation to the posterior density is obtained iteratively using the best possible linearisation of the conditional mean of the labels and accounting for the linearisation error. PL has s…
Markov chain Monte Carlo (MCMC) algorithms have become powerful tools for Bayesian inference. However, they do not scale well to large-data problems. Divide-and-conquer strategies, which split the data into batches and, for each batch, run independent MCMC algorithms targeting the corresponding subposterior, can spread…
New Bayesian method improves Pareto front estimation in multitask finetuning.
problem Efficiently estimating Pareto fronts for multitask finetuning.
method Variational Model Merging using non-Gaussian posteriors.
result More flexible posteriors lead to better Pareto front estimates.
Variational Bayesian neural networks combine the flexibility of deep learning with Bayesian uncertainty estimation. However, inference procedures for flexible variational posteriors are computationally expensive. A recently proposed method, noisy natural gradient, is a surprisingly simple method to fit expressive poste…
A scalable method for efficient inference in Gaussian process regression networks.
problem Intractable inference in Gaussian process regression networks (GPRN).
method Tensorization of output space, tensor/matrix-normal variational posteriors, joint optimization, and exploiting Kronecker product structure.
result Captures posterior dependencies and improves inference quality for large number of outputs.
Unified analysis of Gaussian Process Thompson Sampling without discretization.
problem Sequential decision-making over continuous action spaces.
method Frequentist regret analysis based on fractional Gaussian process posteriors.
result Unified discretization-free regret bound for various kernel classes.
Gradient boosting can be seen as Gaussian process inference.
problem Improving uncertainty estimates in out-of-domain detection.
method Gradient boosting reformulated as a kernel method converging to Gaussian process inference.
result Gradient boosting can provide better uncertainty estimates through Monte-Carlo estimation of posterior variance.
Bayesian method learns network structure from Gaussian process priors.
problem Computational infeasibility of Bayesian structure learning in GPNs.
method Monte Carlo and MCMC methods for sampling network structures.
result Method outperforms state-of-the-art algorithms in recovering network structure.
Elliptical slice sampling converges geometrically, providing reliable sampling for Bayesian learning.
problem Sampling from posterior distributions in Bayesian learning.
method Elliptical slice sampling, geometric ergodicity.
result Elliptical slice sampling yields geometric convergence guarantees under weak regularity assumptions.
Meta-learning reformulated as Bayesian risk minimization.
problem Learning models to quickly adapt to new tasks from small datasets.
method Formalized meta-learning as Bayesian risk minimization, using a probabilistic framework to compute predictive distributions from posterior distributions of latent variables conditioned on contextual datasets.
result A novel Gaussian approximation for the posterior distribution that converges to maximum likelihood estimates and outperforms Neural Process on benchmark datasets.
Learning in Gaussian Process models occurs through the adaptation of hyperparameters of the mean and the covariance function. The classical approach entails maximizing the marginal likelihood yielding fixed point estimates (an approach called \textit{Type II maximum likelihood} or ML-II). An alternative learning proced…
GPry accelerates Bayesian inference for non-Gaussian posteriors.
problem Efficient Bayesian inference for complex models with moderate parameters.
method Generative Gaussian Process surrogate model with SVM classifier, active learning.
result Significant reduction in inference time and computational cost.
New method DDVI improves posterior inference for deep Gaussian processes.
problem Inference of inducing points in DGPs is challenging and biased.
method DDVI uses denoising diffusion SDE and score matching for posterior approximation.
result Empirically shows DDVI outperforms baseline methods in inducing point inference.
Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate co…
Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. One of the biggest challenges with these models is that exact inference is intractable. The current state-of-the-art inference method, …
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on random matrices. Specifically, we employ a matrix variate Gaussian \cite{gupta1999matrix} parameter posterior distribution where we explicitly model the covariance among the input and output dimensions…
Exact posterior score estimation for solving linear inverse problems
problem Solving linear inverse problems
method Derive the exact posterior score and use it as a denoising training objective
result EPS outperforms training-free and training-based baselines on various metrics
The posterior variance of Gaussian processes is a valuable measure of the learning error which is exploited in various applications such as safe reinforcement learning and control design. However, suitable analysis of the posterior variance which captures its behavior for finite and infinite number of training data is …
Improves Bayesian optimization using Gaussian process Thompson sampling.
problem Global optimization of Gaussian process posterior samples.
method Carefully selects starting points for gradient-based multi-start optimizers, identifies all local optima via univariate global rootfinding, and optimizes the posterior sample.
result Dramatic improvements in overall performance of Bayesian optimization.
Sparse variational approximations allow for principled and scalable inference in Gaussian Process (GP) models. In settings where several GPs are part of the generative model, theses GPs are a posteriori coupled. For many applications such as regression where predictive accuracy is the quantity of interest, this couplin…
GP-PCA reduces infinite-dimensional GP posteriors to a finite space for meta-learning.
problem How to define a structure for a set of Gaussian process posteriors.
method Information geometric framework and variational inference.
result GP-PCA improves meta-learning performance through reduced GP posteriors.
Gaussian processes struggle with compositional functions, but deep Gaussian processes can outperform.
problem Gaussian process regression struggles with compositional functions.
method We study information-theoretic lower bounds for posterior contraction rates in Gaussian process regression for a continuous regression model.
result Posterior based on any mean-zero Gaussian process can only recover the truth at a rate strictly slower than the minimax rate for generalized additive functions.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
TM-VI uses flexible transformation models to approximate complex posteriors in Bayesian models.
problem Approximating complex posteriors in Bayesian models with limited flexibility.
method Transformation models for variational inference (TM-VI).
result TM-VI allows accurate approximation of complex posteriors in models with one parameter and works in a mean-field fashion for multi-parameter models.
Develops intrinsic Gaussian process regression for manifold-valued data.
problem Lack of intrinsic Gaussian process methods for manifold-valued response variables.
method Proposes an intrinsic covariance structure and a novel intrinsic Gaussian process regression model.
result Establishes asymptotic properties and shows posterior consistency.
Efficient Reinforcement Learning usually takes advantage of demonstration or good exploration strategy. By applying posterior sampling in model-free RL under the hypothesis of GP, we propose Gaussian Process Posterior Sampling Reinforcement Learning(GPPSTD) algorithm in continuous state space, giving theoretical justif…
FTIP uses normalizing flows to improve posterior inference in function space.
problem Challenges in posterior inference with implicit-process priors.
method FTIP uses normalizing flows to define a richer variational distribution over combination weights.
result FTIP captures asymmetric and multimodal posterior structure better than Gaussian coefficient approximations.
Flow Annealing Posterior Sampling unifies stochastic-process regression and PDE inverse problems.
problem Function-space posterior sampling for stochastic processes and inverse problems.
method Flow Annealing Posterior Sampling (FAPS) using pretrained function-space flow-matching priors.
result Coherent posterior samples with accurate uncertainty quantification.
Stein variational gradient descent improves inference in Gaussian process models.
problem Inference in Gaussian process models with non-Gaussian likelihoods and large data volumes is computationally intensive and inaccurate with traditional methods.
method Stein variational gradient descent (SVGD) for non-parametric inference.
result SVGD monotonically decreases the Kullback-Leibler divergence from the sampling distribution to the true posterior.
Posterior refinement improves sample efficiency in Bayesian neural networks.
problem Bayesian neural networks suffer from poor predictive performance due to inaccurate posterior approximations.
method Propose refining Gaussian approximate posteriors with normalizing flows to improve predictive distributions.
result Posterior refinement yields competitive predictive performance with minimal computational overhead.
Variational language models seek to estimate the posterior of latent variables with an approximated variational posterior. The model often assumes the variational posterior to be factorized even when the true posterior is not. The learned variational posterior under this assumption does not capture the dependency relat…
We prove exact BNN posterior convergence to GP limit and provide sampling methods.
problem Theoretical and empirical challenges in obtaining exact posterior distributions of wide BNNs.
method Theoretical proof and rejection sampling for generating exact samples.
result Exact BNN posterior converges to GP limit as width increases.
A new algorithm optimizes Gaussian process posterior mean functions efficiently.
problem Optimizing Gaussian process posterior mean functions over hyperrectangles is challenging due to nonlinearity and nonconvexity.
method PALM-Mean, a piecewise-analytic lower-bounding framework embedded in reduced-space spatial branch-and-bound.
result PALM-Mean improves scalability for large datasets compared to general-purpose solvers.
New theory for BNNs with Gaussian priors achieves optimal posterior concentration rates.
problem Lack of theoretical results for BNNs with Gaussian priors.
method New approximation theory for non-sparse DNNs with bounded parameters.
result BNNs with non-sparse general priors can achieve near-minimax optimal posterior concentration rates.