Deep Gaussian Processes with polynomial kernels can collapse rapidly without proper hyperparameter tuning.
problem The collapse of Deep Gaussian Processes with polynomial kernels without careful hyperparameter tuning.
method Analysis using the Berry-Esseen Theorem and observation of prior behavior.
result The prior of a Deep Gaussian Process collapses rapidly towards zero or places negligible mass on low norm functions without proper hyperparameter tuning.
Extends Gaussian Process regression for handling multiple prior distributions.
problem Handling multiple prior distributions in Bayesian Machine Learning models.
method Mixtures of Gaussian Processes with analytical and Sparse Variational approaches.
result Effective in accounting for prior misspecification in functional regression problems.
TSFlow uses Gaussian processes to match priors for better time series forecasting.
problem Difficulties in aligning generative models' priors with time series data.
method Conditional flow matching (CFM) with Gaussian processes, optimal transport, and data-dependent priors.
result TSFlow produces high-quality unconditional samples and competitive forecasting results.
The paper analyzes uncertainty quantification in sparse Gaussian process regression with a Brownian motion prior.
problem Analyzing uncertainty in sparse Gaussian process regression with a Brownian motion prior.
method Theoretical guarantees and limitations for pointwise credible sets are derived for a rescaled Brownian motion prior with a sparse variational Gaussian process method.
result Theoretical characterization of asymptotic frequentist coverage for credible sets, distinguishing conservative and overconfident cases.
Bayesian neural networks use ridgelet prior for uncertainty quantification.
problem Combining strong predictive performance with uncertainty quantification in Bayesian neural networks.
method Proposes a ridgelet prior that approximates a Gaussian process covariance function in the output space of the network.
result Establishes universality property allowing Bayesian neural networks to approximate any Gaussian process.
Gaussian processes are ubiquitous in nature and engineering. A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes. Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors. The methodo…
Gaussian processes struggle with compositional functions, but deep Gaussian processes can outperform.
problem Gaussian process regression struggles with compositional functions.
method We study information-theoretic lower bounds for posterior contraction rates in Gaussian process regression for a continuous regression model.
result Posterior based on any mean-zero Gaussian process can only recover the truth at a rate strictly slower than the minimax rate for generalized additive functions.
Bayesian inverse problems solved with Gaussian models for PDEs.
problem Solving inverse problems with limited data for PDEs.
method Constructing PDE-informed Gaussian priors for Bayesian inversion.
result PDE-informed Gaussian priors outperform traditional priors.
This paper improves Gaussian process predictions by integrating prior knowledge.
problem Gaussian processes lack predictive power when prior information is ignored.
method Derive mean and covariance functions from previous data using weighted sums of basis functions.
result Integrating prior knowledge significantly increases look-ahead time and accuracy.
One goal in Bayesian machine learning is to encode prior knowledge into prior distributions, to model data efficiently. We consider prior knowledge from systems of linear partial differential equations together with their boundary conditions. We construct multi-output Gaussian process priors with realizations in the so…
Proposes scale mixture of NNGPs for more flexible stochastic processes.
problem Limited focus on broadening the class of stochastic processes from NNGPs.
method Scale mixture of NNGPs with scale priors on last-layer parameters.
result Turns neural networks into a richer class of stochastic processes.
Proposes Gaussian process priors on graph sets with geometric structure.
problem Defining Gaussian process priors on sets of graphs with geometric structure.
method Defines priors respecting graph geometric structure, analogous to Euclidean isotropic processes.
result Efficient computational technique for evaluating priors' kernels, making them usable in toolboxes.
tvGP-VAE models tensor-valued latent variables with Gaussian processes for better data structure representation.
problem Agnostic latent variables in VAEs ignore data structure correlations.
method Proposes tensor-variate Gaussian process prior for variational autoencoder.
result Explicitly modeling correlation structures improves model performance in reconstruction.
The paper proposes a semi-parametric Bayesian network model using Gaussian Processes and Horseshoe priors.
problem Learning semi-parametric relationships in Expert Bayesian Networks with minimal nonlinear components.
method Uses Gaussian Processes and Horseshoe priors to model relationships, prioritizes modifying expert graphs, and generates diverse graphs.
result Models outperform state-of-the-art semi-parametric Bayesian Network models in synthetic and real-world datasets.
New framework uses dynamics to justify Gaussian process for turbulent flows.
problem Lack of rigorous justification for Gaussian process priors in turbulent flows.
method Introduces a dynamics-informed Gaussian process framework based on quasi-Gaussianity.
result Provides a principled, long-time dynamical justified GP prior for turbulent flows.
Ens-CGP synthesizes ensemble-based inference with Gaussian processes.
problem Ensemble-based inference and Gaussian process modeling.
method Formulates Ens-CGP as a conditional Gaussian process for ensemble moments.
result Ens-CGP provides a unified probabilistic foundation for Kalman-type methods.
New method uses Gaussian processes for solving linear PDEs with boundary conditions.
problem Solving linear PDEs with boundary conditions.
method Boundary Ehrenpreis--Palamodov Gaussian Processes (B-EPGPs).
result Significant accuracy and resource improvements over existing methods.
We study learning problems in which the conditional distribution of the output given the input varies as a function of additional task variables. In varying-coefficient models with Gaussian process priors, a Gaussian process generates the functional relationship between the task variables and the parameters of this con…
Improves transparency and incorporates prior knowledge in Gaussian Process models.
problem Challenges in understanding and expressing prior assumptions in complex Bayesian models.
method Introduces self-explaining variational posterior distributions for Gaussian Processes.
result Allows incorporation of both general and feature-specific prior knowledge.
Review of priors in Bayesian deep learning models.
problem The importance of prior choices in Bayesian deep learning models.
method Overview of different priors and methods of learning priors from data.
result Motivate practitioners to think carefully about prior specification.
Develops a new MCMC-based Wishart prior for Gaussian Process covariance matrix.
problem Difficult inference for multivariate Gaussian Processes with multiple lengthscale parameters.
method Introduces a self-assembled Wishart prior and uses MCMC for Bayesian inference on kernel hyperparameters.
result Demonstrates the effectiveness of the new prior in GP-based learning with empirical results.
We consider a Gaussian process formulation of the multiple kernel learning problem. The goal is to select the convex combination of kernel matrices that best explains the data and by doing so improve the generalisation on unseen data. Sparsity in the kernel weights is obtained by adopting a hierarchical Bayesian approa…
This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.
problem Understanding the relationship between Tensor Networks and Gaussian Processes.
method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.
result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.
The study optimizes Gaussian process approximations for finite-rank models.
problem Posterior behavior of finite-rank approximations differs from parent GP priors.
method Locally supported basis expansions with dependent Gaussian coefficients.
result Finite-rank expansions inherit the same posterior contraction rate as parent GP priors.
Deep Gaussian processes can have non-degenerate and non-Gaussian limits.
problem Understanding the behavior of deep Gaussian processes as depth grows.
method Studying the limit of compositional Gaussian processes where each layer is a Gaussian process.
result Identified a sharp bandwidth threshold above which the limit is degenerate, and proved that for bandwidths below this threshold, the limit is a non-degenerate and non-Gaussian distribution.
Develops Gaussian processes on non-Euclidean spaces with symmetries.
problem Invariance to symmetries in non-Euclidean spaces.
method Constructive techniques for stationary Gaussian processes on compact and non-compact spaces.
result Makes non-Euclidean Gaussian processes compatible with standard software.
Develops Gaussian processes on non-compact Lie groups.
problem Invariance to symmetries in non-Euclidean spaces.
method Constructive techniques for stationary Gaussian processes.
result Makes non-Euclidean Gaussian processes compatible with standard software.
We algorithmically construct multi-output Gaussian process priors which satisfy linear differential equations. Our approach attempts to parametrize all solutions of the equations using Gröbner bases. If successful, a push forward Gaussian process along the paramerization is the desired prior. We consider several exampl…
This work approximates finite neural networks with Gaussian processes, providing error bounds and applications in prior selection.
problem Approximating finite neural networks with Gaussian processes for error bounds and uncertainty quantification.
method Iterative approximation of neural network layers as mixtures of Gaussian processes, using optimal transport and Gaussian processes.
result The ability to return a mixture of Gaussian processes that is ε-close to the neural network at a finite set of input points.
Empirical Gaussian Processes learn flexible priors from data.
problem Limited effectiveness of standard Gaussian process kernels.
method Estimate mean and covariance functions empirically from data.
result Empirical GPs converge to closest GP to real data generating process.
We present a non-parametric Bayesian latent variable model capable of learning dependency structures across dimensions in a multivariate setting. Our approach is based on flexible Gaussian process priors for the generative mappings and interchangeable Dirichlet process priors to learn the structure. The introduction of…
We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a G…
Two algorithms improve GP bandits by selecting priors and minimizing regret.
problem Selecting appropriate GP priors for unknown functions.
method Developed two algorithms: Prior-Elimination GP-TS and HyperPrior GP-TS.
result Established sublinear regret bound for HyperPrior GP-TS.
Efficiently solves inverse PDE problems with Gaussian processes.
problem Solving inverse problems in linear PDEs with noisy data.
method Gaussian process regression with algebraic priors.
result High accuracy and computational efficiency achieved.
New approach to neural networks by incorporating observation noise and arbitrary prior means.
problem Misspecification on noisy data and limitations of NTK-GP equivalence.
method Introducing a regularizer for observation noise and proposing a shifted network for arbitrary prior means.
result Removes key obstacles to practical Gaussian process modeling in neural networks.
When fitting Bayesian machine learning models on scarce data, the main challenge is to obtain suitable prior knowledge and encode it into the model. Recent advances in meta-learning offer powerful methods for extracting such prior knowledge from data acquired in related tasks. When it comes to meta-learning in Gaussian…
Develops a new method for functional regression that works with non-Gaussian data.
problem Limited models for regression in function spaces with Gaussian process priors.
method Introduces Neural Operator Flows (OpFlow) for non-Gaussian function spaces.
result OpFlow enables robust and accurate uncertainty quantification for functional regression.
WS diffusion models handle anisotropic Gaussian noise better than conventional methods.
problem Handling anisotropic Gaussian noise in imaging inverse problems.
method Whitened Score (WS) diffusion models based on stochastic differential equations.
result WS DMs outperform conventional DMs on anisotropic Gaussian noise.
Hybrid Bayesian neural networks use function uncertainty for probabilistic inference.
problem Uncertainty in neural network weights is hard to specify and interpret.
method Integrates probabilistic layers with standard deterministic layers for function uncertainty.
result Improves probabilistic inference by encoding function uncertainty.
GP-BART improves BART's predictive performance by incorporating Gaussian process priors.
problem Lack of smoothness and explicit covariance structure in BART.
method GP-BART extends BART with Gaussian process priors for tree predictions.
result GP-BART outperforms traditional models in various applications.
The data association problem is concerned with separating data coming from different generating processes, for example when data come from different data sources, contain significant noise, or exhibit multimodality. We present a fully Bayesian approach to this problem. Our model is capable of simultaneously solving the…
New method tunes prior IP to data for flexible predictive distributions.
problem Challenges in approximate inference for large models with high parameter dependencies.
method Inducing-point representation of prior IP to approximate posterior process.
result Scalable method that tunes prior IP to data and provides accurate non-Gaussian predictive distributions.
Meta-learn sparse Gaussian process inference for faster predictions.
problem Cubic computational cost of exact Gaussian process inference for many observations.
method Meta-learn sparse Gaussian process inference.
result Rapid prediction on new tasks with sparse Gaussian processes.
The paper introduces a method to probabilistically select inducing points in sparse Gaussian processes.
problem The challenge is selecting the optimal number of inducing points in sparse Gaussian processes.
method A point process prior is applied to the inducing points, and the posterior is approximated using stochastic variational inference.
result The model learns which and how many inducing points to use, leading to fewer inducing points being preferred as they become less informative.
Proposes a Bayesian Autoencoder with sparse Gaussian process priors to capture data correlations.
problem Autoencoders' i.i.d. assumption of latent representations fails to capture data correlations.
method Imposes fully Bayesian sparse Gaussian Process priors on the latent space of a Bayesian Autoencoder and uses stochastic gradient Hamiltonian Monte Carlo for posterior estimation.
result Consistently outperforms alternatives relying on Variational Autoencoders on various tasks.
The Brownian bridge serves as a physics-informed prior for solving the Poisson equation.
problem Reconstructing physical fields from limited and noisy data with known governing equations.
method Formalizing inverse problems via Bayesian inference in function spaces using a Brownian bridge Gaussian process.
result The Brownian bridge Gaussian process can be viewed as a physics-constrained prior for the Poisson equation, allowing for a fully Bayesian framework.
We tackle the problem of multi-task learning with copula process. Multivariable prediction in spatial and spatial-temporal processes such as natural resource estimation and pollution monitoring have been typically addressed using techniques based on Gaussian processes and co-Kriging. While the Gaussian prior assumption…
We use diffusion models to sample from complex GP priors in climate data.
problem Sampling from non-stationary Gaussian process priors is computationally hard.
method Replace GP prior with a diffusion model surrogate and use training-free guidance algorithms.
result Generated distributions are close to GP priors and can be fine-tuned.