Gaussian processes struggle with compositional functions, but deep Gaussian processes can outperform.
problem Gaussian process regression struggles with compositional functions.
method We study information-theoretic lower bounds for posterior contraction rates in Gaussian process regression for a continuous regression model.
result Posterior based on any mean-zero Gaussian process can only recover the truth at a rate strictly slower than the minimax rate for generalized additive functions.
New sparse Gaussian process method tackles unconstrained regression problems.
problem Dealing with physical systems that satisfy inequality constraints.
method Extends constrained Gaussian process by redefining hat basis functions.
result Reduces computational complexity from O(n3) to O(nm2). Develops a new method for functional regression that works with non-Gaussian data.
problem Limited models for regression in function spaces with Gaussian process priors.
method Introduces Neural Operator Flows (OpFlow) for non-Gaussian function spaces.
result OpFlow enables robust and accurate uncertainty quantification for functional regression.
The paper introduces a non-linear version of the process convolution formalism for building covariance functions for multi-output Gaussian processes. The non-linearity is introduced via Volterra series, one series per each output. We provide closed-form expressions for the mean function and the covariance function of t…
This paper proposes a method to approximate non-Gaussian likelihoods in Gaussian Processes.
problem Approximating non-Gaussian likelihoods in Gaussian Processes.
method Proposes a piece-wise constant approximation for the inverse-link function.
result Yields a closed form solution for the SVGP lower bound.
New method optimizes processes under constraints using bivariate Gaussian models.
problem Optimizing processes with constraints using traditional methods.
method Developed a constrained expected improvement acquisition function using bivariate Gaussian process models.
result Demonstrated improved performance in a manufacturing cure process optimization.
Novel Hilbert space Gaussian process improves sequential design accuracy and efficiency.
problem Efficiently implementing Gaussian process acquisition functions for expensive simulations.
method Proposed a truncated eigenbasis representation for closed-form evaluation of IMSE acquisition function.
result Significantly lower prediction error and reduced computation time compared to benchmarks.
Graph Gaussian processes use Matérn models for better function learning.
problem Lack of Gaussian process models for graph input spaces.
method Stochastic partial differential equation characterization of Matérn Gaussian processes.
result Graph Matérn Gaussian processes inherit properties of Euclidean and Riemannian models and can be trained efficiently.
Improved Gaussian process models for interpretable predictions.
problem Complex responses require high-dimensional interaction terms in additive Gaussian processes.
method Orthogonal additive kernel (OAK) with orthogonality constraint on additive functions.
result OAK models achieve similar or better predictive performance with fewer terms, retaining interpretability.
Projection pursuit model improves Gaussian process regression for high-dimensional data.
problem Scalability issues with traditional Gaussian process models in high dimensions.
method Additive Gaussian process regression with dimension expansion and gradient descent.
result The proposed method approximates more complex functions and outperforms traditional models.
This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.
problem Understanding the relationship between Tensor Networks and Gaussian Processes.
method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.
result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.
Extends Gaussian process theory to Banach spaces.
problem Extending Gaussian process theory to Banach spaces.
method Investigates the connection between Gaussian processes and Gaussian random elements in reproducing kernel Banach spaces.
result Characterizes positive definite functions that arise from covariance operators in Banach space setting.
Deep neural networks and Gaussian processes are shown to be equivalent through activation functions.
problem Understanding the relationship between neural networks and Gaussian processes.
method Developing an equivalence theory based on activation functions and kernels.
result Models can be seen as neural networks with improved uncertainty prediction or deep Gaussian processes with increased accuracy.
Advanced kernels improve Gaussian process accuracy by incorporating domain knowledge.
problem Improving function approximation accuracy in Gaussian processes.
method Advanced kernel designs that enforce specific function properties (symmetry, periodicity) and non-stationarity.
result Advanced kernels significantly enhance function approximation accuracy and relevance.
Bayesian approach for inhomogeneous Poisson process intensity estimation.
problem Intractable integral in likelihood of Gaussian Cox process.
method Joint modeling of intensity and cumulative intensity as transformed Gaussian process; exact MCMC sampler.
result Exact posterior inference without approximations.
Introduces tunable basis functions for Gaussian processes.
problem Reduces computational complexity in Gaussian process approximations.
method Introduces tunable, local, and bounded basis functions for kernel approximation.
result Demonstrates superior performance compared to state-of-the-art methods, especially with poorly chosen kernel functions.
The paper studies deep neural networks with Gaussian weights and finds their asymptotic behavior.
problem Understanding the behavior of deep neural networks with large width.
method Function-space perspective, Gaussian process analysis, weak convergence in large-width limit.
result Deep neural networks with large width converge to a continuous Gaussian process.
Wide neural networks can be closely approximated by Gaussian processes, with rates depending on the activation function's properties.
problem Approximating the behavior of wide neural networks using Gaussian processes.
method Established convergence rates for the central limit theorem in an infinite-dimensional functional space, using a transportation distance metric.
result Explicit convergence rates for neural networks approximated by Gaussian processes, varying based on the activation function's properties.
New analysis explains pathology of deep Gaussian processes.
problem Pathology of deep Gaussian processes reduces learning capacities with increased layers.
method Study nonlinear dynamic systems corresponding to DGPs, derive recurrence relations.
result Provide tighter bounds and rate of convergence for dynamic systems.
A scalable Gaussian process clustering method for large datasets.
problem Infeasibility of Gaussian process clustering on large grids.
method Embedding Vecchia approximation in EM algorithm for scalability.
result Efficient Gaussian process clustering for large environmental applications.
Bayesian Optimization using Gaussian Processes is a popular approach to deal with the optimization of expensive black-box functions. However, because of the a priori on the stationarity of the covariance matrix of classic Gaussian Processes, this method may not be adapted for non-stationary functions involved in the op…
New method for spatiotemporal data regression using Gaussian processes.
problem Regression in spatiotemporal random fields.
method Empirical Bayes approach, tight Gaussian measures, truncation scheme.
result Effective dimension reduction through time-varying angular spectra.
This paper improves Gaussian process predictions by integrating prior knowledge.
problem Gaussian processes lack predictive power when prior information is ignored.
method Derive mean and covariance functions from previous data using weighted sums of basis functions.
result Integrating prior knowledge significantly increases look-ahead time and accuracy.
Bayesian approach uses Gaussian process for reinforcement learning.
problem Robotic locomotion environments
method Bayesian actor-critic, model-free reinforcement learning with Gaussian process for exploration and policy optimization.
result Gaussian process method outperforms current algorithms in robotic locomotion environments.
Gaussian process priors are commonly used in aerospace design for performing Bayesian optimization. Nonetheless, Gaussian processes suffer two significant drawbacks: outliers are a priori assumed unlikely, and the posterior variance conditioned on observed data depends only on the locations of those data, not the assoc…
We present a novel extension of multi-output Gaussian processes for handling heterogeneous outputs. We assume that each output has its own likelihood function and use a vector-valued Gaussian process prior to jointly model the parameters in all likelihoods as latent functions. Our multi-output Gaussian process uses a c…
New method for global optimization of Gaussian processes reduces computational time.
problem Nonconvex optimization problems with Gaussian processes trained on few data points.
method Reduced-space formulation with branch-and-bound solver and McCormick relaxations.
result Significantly reduced computational time compared to state-of-the-art methods.
GNet uses Gaussian processes for scalable, flexible neural networks.
problem Large-scale predictive modeling with high computational and storage costs.
method GNet employs Gaussian processes with nonparametric activation functions and a fast algorithm for training and predictions.
result GNet achieves competitive performance across various test problems, including nonlinear function prediction and real-world data regression.
GNet uses Gaussian processes for scalable, flexible neural networks.
problem Large-scale predictive modeling with high computational and storage costs.
method GNet employs Gaussian processes with nonparametric activation functions and a fast algorithm for efficient training and predictions.
result GNet achieves competitive performance across various test problems, including nonlinear function prediction and real-world data regression.
A linear model approximates Gaussian processes for efficient control.
problem Efficiently modeling and controlling Gaussian processes with many parameters.
method Developed a linear model using basis functions to approximate Gaussian processes.
result The linear model improves computational efficiency and feasibility of control strategies.
DGPFM uses deep Gaussian processes to map functions accurately and quantify uncertainty.
problem Learning mappings between functional spaces, especially when data are noisy, sparse, or irregularly sampled.
method Constructs a sequence of GP-based linear and nonlinear transformations directly in function space, leveraging kernel integral transforms, GP conditional means, and nonlinear activations sampled from Gaussian processes.
result Empirical results show DGPFM outperforms existing methods in predictive accuracy and uncertainty calibration.
Large-scale Gaussian process inference has long faced practical challenges due to time and space complexity that is superlinear in dataset size. While sparse variational Gaussian process models are capable of learning from large-scale data, standard strategies for sparsifying the model can prevent the approximation of …
This paper optimizes Bayesian acquisition functions in Gaussian Processes for better optimization.
problem Improving the efficiency of Bayesian optimization methods.
method Analysis of different acquisition functions and optimizers for optimizing Bayesian acquisition functions.
result Optimization of acquisition functions leads to faster and more accurate sampling points.
New Gaussian processes for Riemannian manifolds enable uncertainty quantification.
problem Modeling functions on Riemannian manifolds with uncertainty.
method Generalized Matérn Gaussian processes on compact manifolds via spectral theory.
result Efficient training of Riemannian Matérn Gaussian processes using scalable techniques.
We propose a simple method that combines neural networks and Gaussian processes. The proposed method can estimate the uncertainty of outputs and flexibly adjust target functions where training data exist, which are advantages of Gaussian processes. The proposed method can also achieve high generalization performance fo…
We consider a Gaussian process formulation of the multiple kernel learning problem. The goal is to select the convex combination of kernel matrices that best explains the data and by doing so improve the generalisation on unseen data. Sparsity in the kernel weights is obtained by adopting a hierarchical Bayesian approa…
Gaussian processes adapted for non-Euclidean spaces enhance decision-making.
problem Applying Gaussian processes in non-Euclidean spaces.
method Developed pathwise conditioning and Gaussian process models over non-Euclidean spaces.
result Efficient Gaussian process models for non-Euclidean spaces.
e-GGPs learn graph vertex transitions over time.
problem Static graph Gaussian Processes cannot handle dynamic graph structures.
method Proposes e-GGPs with a transition function and neighbourhood kernel.
result e-GGPs outperform static GGPs on time-series regression.
Develops intrinsic Gaussian process regression for manifold-valued data.
problem Lack of intrinsic Gaussian process methods for manifold-valued response variables.
method Proposes an intrinsic covariance structure and a novel intrinsic Gaussian process regression model.
result Establishes asymptotic properties and shows posterior consistency.
This note explains when neural networks can be seen as Gaussian processes.
problem Understanding the relationship between neural networks and Gaussian processes.
method Formulating a Gaussian process regression based on neural network outputs and analyzing the resulting posterior mean functions.
result The posterior mean functions of neural networks follow a Gaussian process in certain cases, providing an interpretation of reproducing kernel Hilbert spaces.
Gaussian processes are powerful, yet analytically tractable models for supervised learning. A Gaussian process is characterized by a mean function and a covariance function (kernel), which are determined by a model selection criterion. The functions to be compared do not just differ in their parametrization but in thei…
Improved Gaussian process experts model for complex data.
problem Limitations of standard Gaussian processes: scalability and predictive performance.
method Proposes a new mixture model of Gaussian process experts based on kernel stick-breaking processes.
result Improved predictive performance compared to existing models.
Multi-output Gaussian processes have received increasing attention during the last few years as a natural mechanism to extend the powerful flexibility of Gaussian processes to the setup of multiple output variables. The key point here is the ability to design kernel functions that allow exploiting the correlations betw…
New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
Gaussian process models are flexible, Bayesian non-parametric approaches to regression. Properties of multivariate Gaussians mean that they can be combined linearly in the manner of additive models and via a link function (like in generalized linear models) to handle non-Gaussian data. However, the link function formal…
Gaussian processes are used in machine learning to learn input-output mappings from observed data. Gaussian process regression is based on imposing a Gaussian process prior on the unknown regressor function and statistically conditioning it on the observed data. In system identification, Gaussian processes are used to …
Efficiently trains deep Gaussian processes with sparse approximations.
problem High computational complexity in training and inference for DGP models.
method Tensor Markov Gaussian Processes (TMGP) and hierarchical expansion to create DTMGP model.
result DTMGP model achieves superior computational efficiency compared to existing DGP models.
Optimizes Gaussian process hyperparameters using Bayesian autoregression.
problem Optimizing hyperparameters for Matérn kernel temporal Gaussian processes.
method Recursive Bayesian estimation for autoregressive parameters.
result Outperforms traditional optimization methods in runtime and accuracy.