Two algorithms improve GP bandits by selecting priors and minimizing regret.
problem Selecting appropriate GP priors for unknown functions.
method Developed two algorithms: Prior-Elimination GP-TS and HyperPrior GP-TS.
result Established sublinear regret bound for HyperPrior GP-TS.
We use diffusion models to sample from complex GP priors in climate data.
problem Sampling from non-stationary Gaussian process priors is computationally hard.
method Replace GP prior with a diffusion model surrogate and use training-free guidance algorithms.
result Generated distributions are close to GP priors and can be fine-tuned.
New method improves GP uncertainty quantification for misspecified priors.
problem Uncertainty quantification for GPs under incorrect priors.
method Constructs a confidence sequence using martingale techniques.
result Empirically outperforms standard GP methods in robustness and utility for Bayesian Optimization.
New method uses trainable activations to make BNNs behave like GPs.
problem Making Bayesian Neural Networks (BNNs) behave like Gaussian Processes (GPs).
method Introduced trainable activations and periodic activations to map GP priors to BNNs. Used 2-Wasserstein distance for optimization.
result Method consistently outperforms existing approaches or matches heuristic methods.
The study optimizes Gaussian process approximations for finite-rank models.
problem Posterior behavior of finite-rank approximations differs from parent GP priors.
method Locally supported basis expansions with dependent Gaussian coefficients.
result Finite-rank expansions inherit the same posterior contraction rate as parent GP priors.
DeepRV accelerates spatiotemporal inference using neural priors.
problem Intractable scaling of Gaussian Processes for large datasets.
method Neural-network surrogate replacing GP prior sampling with O(N2) complexity. result DeepRV achieves highest fidelity to exact GPs while significantly speeding up inference.
Empirical Gaussian Processes learn flexible priors from data.
problem Limited effectiveness of standard Gaussian process kernels.
method Estimate mean and covariance functions empirically from data.
result Empirical GPs converge to closest GP to real data generating process.
New approach to neural networks by incorporating observation noise and arbitrary prior means.
problem Misspecification on noisy data and limitations of NTK-GP equivalence.
method Introducing a regularizer for observation noise and proposing a shifted network for arbitrary prior means.
result Removes key obstacles to practical Gaussian process modeling in neural networks.
Proposes DAK model for improved GP computations.
problem Challenges in high-dimensional GP layers in DKL.
method Additive structure and induced prior approximation for GP units.
result Outperforms state-of-the-art DKL methods in regression and classification.
Gaussian process (GP) audio source separation is a time-domain approach that circumvents the inherent phase approximation issue of spectrogram based methods. Furthermore, through its kernel, GPs elegantly incorporate prior knowledge about the sources into the separation model. Despite these compelling advantages, the c…
KITT uses transformers to quickly recommend kernels for GP models.
problem Kernel selection for high-dimensional GP regression models.
method Transformer-based architecture for generating kernel recommendations.
result KITT selects kernels that perform well on various regression benchmarks.
ET-GP-UCB optimizes time-varying functions without knowing change rates.
problem Sequentially optimizing a time-varying objective function with unknown change rates.
method Event-triggered Bayesian optimization with adaptive resets based on probabilistic uniform error bounds.
result ET-GP-UCB outperforms other GP-UCB algorithms in synthetic and real-world data.
PriorVAE uses VAEs to efficiently encode spatial priors for small-area estimation.
problem Efficiently encoding spatial priors for small-area estimation using Gaussian processes.
method Approximating Gaussian process priors with a variational autoencoder (VAE).
result Efficient spatial inference through a low-dimensional latent Gaussian space representation.
The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a well-known non-parametric and interpretable Bayesian model, which suffers from cu…
This work extends Gaussian process priors to neural operators for function space mappings.
problem Improving uncertainty quantification in deep neural networks.
method Extending Gaussian process priors to neural operators with conditions for convergence and computation of covariance functions.
result Arbitrary-depth neural operators with Gaussian kernels converge to function-valued GPs, enabling posterior computation in regression scenarios.
Develops a new MCMC-based Wishart prior for Gaussian Process covariance matrix.
problem Difficult inference for multivariate Gaussian Processes with multiple lengthscale parameters.
method Introduces a self-assembled Wishart prior and uses MCMC for Bayesian inference on kernel hyperparameters.
result Demonstrates the effectiveness of the new prior in GP-based learning with empirical results.
GP model for time series forecasting with priors.
problem Automatic selection of optimal kernels and reliable estimation of hyperparameters.
method Fixed composition of kernels, automatic relevance determination (ARD), empirical Bayes priors.
result GP model is more accurate than state-of-the-art models.
Paper tightens variational GP approximations for large datasets.
problem Scaling Gaussian processes to large datasets.
method Relaxing the standard assumption about inducing points' posterior matching the prior, leading to a tighter variational approximation.
result The proposed approximation consistently matches or outperforms standard sparse variational GPs while maintaining computational cost.
GPs' decisions can vary significantly with different kernels, even if kernels are qualitatively similar.
problem Robustness of GP decisions to kernel choice.
method Solved a constrained optimization problem over a finite-dimensional space to identify changes in GP decisions.
result Decisions made with a GP can be non-robust to kernel choice, even with qualitatively similar kernels.
It is well-known that the distribution over functions induced through a zero-mean iid prior distribution over the parameters of a multi-layer perceptron (MLP) converges to a Gaussian process (GP), under mild conditions. We extend this result firstly to independent priors with general zero or non-zero means, and secondl…
This paper discovers classification models from sequential data without prior knowledge.
problem Lack of prior knowledge in defining kernels for online classification.
method Adapts GP-based time-series structure discovery with SMC to learn new features from sequential data.
result Improves classification accuracy by 10% on real-world data.
We address the problem of continual learning in multi-task Gaussian process (GP) models for handling sequential input-output observations. Our approach extends the existing prior-posterior recursion of online Bayesian inference, i.e.\ past posterior discoveries become future prior beliefs, to the infinite functional sp…
New GP-based method improves uncertainty quantification for causal functions.
problem Challenges in quantifying uncertainty for causal effects, especially for entire functions.
method GP-based approach using inner-product of observational functions in RKHS, with tractable posterior moments and calibration.
result Improves uncertainty quantification while maintaining causal effect estimation performance.
GP-BART improves BART's predictive performance by incorporating Gaussian process priors.
problem Lack of smoothness and explicit covariance structure in BART.
method GP-BART extends BART with Gaussian process priors for tree predictions.
result GP-BART outperforms traditional models in various applications.
This technical report presents pseudo-code for a Riemannian manifold Hamiltonian Monte Carlo (RMHMC) method to efficiently simulate samples from N-dimensional posterior distributions p(x∣y), where x∈RN is drawn from a Gaussian Process (GP) prior, and observations yn are independent given xn. Sufficient…
Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost and it is difficult to design nonstationary GP priors in practice. In this paper, we propose a sparse Gaussian process model, EigenGP, based on the Karhunen-Loeve (KL) expan…
Method learns SDEs from one trajectory using GP priors and randomized cross-validation.
problem Learning SDEs from a single trajectory.
method Combining CGC and data-adapted kernels learned via randomized cross-validation.
result Efficacy, robustness, and scope of the method demonstrated in numerical experiments.
The paper proposes a method to improve Bayesian inference for periodic data using data-driven priors.
problem Efficiency in approximating posterior distribution in models with periodicity.
method Construct a prior distribution from data using a Gaussian process with a periodic kernel, approximated using adaptive importance sampling.
result The proposed method improves the marginal posterior distribution of the period parameter.
GP-PCA reduces infinite-dimensional GP posteriors to a finite space for meta-learning.
problem How to define a structure for a set of Gaussian process posteriors.
method Information geometric framework and variational inference.
result GP-PCA improves meta-learning performance through reduced GP posteriors.
PriorCVAE uses deep generative models to infer hyperparameters in MCMC.
problem Losing hyperparameter information in GP prior inference.
method Conditioning VAE on hyperparameters to encode and estimate them during inference.
result PriorCVAE enables efficient and distinct inference of hyperparameters.
Central to robot exploration and mapping is the task of persistent localization in environmental fields characterized by spatially correlated measurements. This paper presents a Gaussian process localization (GP-Localize) algorithm that, in contrast to existing works, can exploit the spatially correlated field measurem…
Proposes PE-GP-UCB for time-varying Bayesian optimisation.
problem Time-varying Gaussian process bandits with unknown prior.
method PE-GP-UCB algorithm, relying on consistency of function values with priors.
result Regret bound provided for the proposed algorithm.
PFNs4BO uses neural processes for flexible Bayesian Optimization.
problem Efficient surrogate modeling for Bayesian Optimization.
method In-context learning of PFNs to approximate posterior predictive distribution.
result PFNs outperform traditional GP, BNN in BO tasks.
New sparse GP model learns compositional kernels efficiently.
problem Learning accurate Gaussian Process models with complex kernel structures.
method MultiSVGP model with Horseshoe prior for kernel selection.
result Our model provides better fit and faster computation for large-scale data.
Tabular FMs struggle with reliable uncertainty quantification.
problem Uncertainty quantification in tabular foundation models.
method Compared TabPFN and Gaussian processes (GPs) across various regression tasks.
result GP outperforms TabPFN in data-scarce settings and when kernels are good priors.
Gaussian process (GP) priors are non-parametric generative models with appealing modelling properties for Bayesian inference: they can model non-linear relationships through noisy observations, have closed-form expressions for training and inference, and are governed by interpretable hyperparameters. However, GP models…
Generative Bayesian Computation improves surrogates for expensive simulations.
problem Limitations of Gaussian process surrogates in handling complex, non-stationary data.
method Generative Bayesian Computation via Implicit Quantile Networks (IQNs).
result Generative Bayesian Computation outperforms traditional Gaussian process methods across various benchmarks.
Gaussian processes (GP) are powerful tools for probabilistic modeling purposes. They can be used to define prior distributions over latent functions in hierarchical Bayesian models. The prior over functions is defined implicitly by the mean and covariance function, which determine the smoothness and variability of the …
A scalable GPVAE method using local adjacencies to approximate GP inference.
problem Scalability issues in exact GP inference for large-scale GPVAEs.
method Neighbour-driven approximation strategy that confines computations to nearest neighbours.
result Outperforms other GPVAE variants in predictive performance and computational efficiency.
Paper analyzes convergence rate of noisy Bayesian Optimization with Expected Improvement.
problem Theoretical convergence behaviors and rates of Expected Improvement (EI) in Bayesian optimization.
method Analyzes Expected Improvement (EI) under Gaussian process (GP) prior assumption, considering noisy observations.
result Established asymptotic error bound and rate for GP-EI with noisy observations.
Bayesian neural networks with Mercer priors for interpretable uncertainty quantification.
problem Uncertainty quantification in neural networks, especially for complex input-to-output mappings.
method Introducing Mercer priors for BNNs, which approximate a specified GP and are scalable.
result BNNs with Mercer priors can approximate the uncertainty of a specified GP, making them interpretable and scalable.
Enhances neural networks with prior knowledge through a composite kernel.
problem Lack of effective methods to incorporate prior knowledge into neural networks.
method Integrates a composite kernel combining a neural network kernel and a GP kernel for modeling known properties.
result Demonstrates superior performance and flexibility of the Implicit Composite Kernel (ICK) on synthetic and real-world data.
In this paper we introduce a novel framework for making exact nonparametric Bayesian inference on latent functions, that is particularly suitable for Big Data tasks. Firstly, we introduce a class of stochastic processes we refer to as string Gaussian processes (string GPs), which are not to be mistaken for Gaussian pro…
Enhances deep kernel learning with stochastic latent variables for better model regularization.
problem Weak model regularization in deep kernel learning, especially on small datasets.
method Introduces DLVKL model with stochastic latent variables, NSDE for expressive posterior, and hybrid prior.
result DLVKL-NSDE outperforms existing deep GPs on large datasets.
New framework uses dynamics to justify Gaussian process for turbulent flows.
problem Lack of rigorous justification for Gaussian process priors in turbulent flows.
method Introduces a dynamics-informed Gaussian process framework based on quasi-Gaussianity.
result Provides a principled, long-time dynamical justified GP prior for turbulent flows.
Develops SGP-VAE for efficient sparse GP inference in multi-dimensional datasets.
problem Sparse GP approximations and missing data in multi-dimensional spatio-temporal datasets.
method Leverages partial inference networks for sparse GP approximations and amortized variational inference.
result Outperforms multi-output GPs and structured VAEs in various experiments.
New GP-VAE model improves scalability and performance.
problem Inability of conventional VAEs to model correlations between data points.
method Principled sparse inference approaches to improve scalability of GP-VAEs.
result New model outperforms existing approaches in runtime and memory usage.
EPGP priors solve linear PDEs from data.
problem Modeling physical systems with PDEs.
method EPGP priors based on Ehrenpreis-Palamodov principle.
result EPGP priors improve computation time and precision.