Gaussian processes struggle with compositional functions, but deep Gaussian processes can outperform.
problem Gaussian process regression struggles with compositional functions.
method We study information-theoretic lower bounds for posterior contraction rates in Gaussian process regression for a continuous regression model.
result Posterior based on any mean-zero Gaussian process can only recover the truth at a rate strictly slower than the minimax rate for generalized additive functions.
Develops a new method for functional regression that works with non-Gaussian data.
problem Limited models for regression in function spaces with Gaussian process priors.
method Introduces Neural Operator Flows (OpFlow) for non-Gaussian function spaces.
result OpFlow enables robust and accurate uncertainty quantification for functional regression.
Develops a Gaussian model to compute the Alexander polynomial of knots.
problem Computing the Alexander polynomial of knots.
method Uses perturbed Gaussian functions, Heisenberg algebra, and tensor-contraction formalism.
result Associates a Gaussian function to a knot whose partition function recovers the Alexander polynomial.
New sparse Gaussian process method tackles unconstrained regression problems.
problem Dealing with physical systems that satisfy inequality constraints.
method Extends constrained Gaussian process by redefining hat basis functions.
result Reduces computational complexity from O(n3) to O(nm2). This paper proposes a method to approximate non-Gaussian likelihoods in Gaussian Processes.
problem Approximating non-Gaussian likelihoods in Gaussian Processes.
method Proposes a piece-wise constant approximation for the inverse-link function.
result Yields a closed form solution for the SVGP lower bound.
Bayesian approach approximates probability functions of Gaussian mixtures.
problem Approximating probability functions of non-spherical Gaussian mixtures.
method Bayesian decomposition, spherical radial decomposition, random sampling.
result Established differentiability and integral representation of gradient for probability functions.
The paper introduces a non-linear version of the process convolution formalism for building covariance functions for multi-output Gaussian processes. The non-linearity is introduced via Volterra series, one series per each output. We provide closed-form expressions for the mean function and the covariance function of t…
Simplified proof of Gaussian concentration inequality using covariance.
problem Gaussian concentration inequality proof
method Covariance representation based on characteristic functions
result Elementary proof of Gaussian concentration inequality
Extends Gaussian process theory to Banach spaces.
problem Extending Gaussian process theory to Banach spaces.
method Investigates the connection between Gaussian processes and Gaussian random elements in reproducing kernel Banach spaces.
result Characterizes positive definite functions that arise from covariance operators in Banach space setting.
The paper studies deep neural networks with Gaussian weights and finds their asymptotic behavior.
problem Understanding the behavior of deep neural networks with large width.
method Function-space perspective, Gaussian process analysis, weak convergence in large-width limit.
result Deep neural networks with large width converge to a continuous Gaussian process.
Novel Hilbert space Gaussian process improves sequential design accuracy and efficiency.
problem Efficiently implementing Gaussian process acquisition functions for expensive simulations.
method Proposed a truncated eigenbasis representation for closed-form evaluation of IMSE acquisition function.
result Significantly lower prediction error and reduced computation time compared to benchmarks.
Study non-Gaussian measures' concentration properties in metric spaces.
problem Concentration properties for non-linear Gaussian functionals with non-Gaussian tails.
method Prove generalised Transportation-Cost Inequalities (TCIs) for specific functionals.
result Extended TCIs for rough volatility and Parabolic Anderson Model.
Improved Gaussian process models for interpretable predictions.
problem Complex responses require high-dimensional interaction terms in additive Gaussian processes.
method Orthogonal additive kernel (OAK) with orthogonality constraint on additive functions.
result OAK models achieve similar or better predictive performance with fewer terms, retaining interpretability.
Projection pursuit model improves Gaussian process regression for high-dimensional data.
problem Scalability issues with traditional Gaussian process models in high dimensions.
method Additive Gaussian process regression with dimension expansion and gradient descent.
result The proposed method approximates more complex functions and outperforms traditional models.
New method for spatiotemporal data regression using Gaussian processes.
problem Regression in spatiotemporal random fields.
method Empirical Bayes approach, tight Gaussian measures, truncation scheme.
result Effective dimension reduction through time-varying angular spectra.
Introduces tunable basis functions for Gaussian processes.
problem Reduces computational complexity in Gaussian process approximations.
method Introduces tunable, local, and bounded basis functions for kernel approximation.
result Demonstrates superior performance compared to state-of-the-art methods, especially with poorly chosen kernel functions.
Paper presents characteristic function of Tsallis q-Gaussian and its applications.
problem Modeling input quantities in measurement models using Tsallis q-Gaussians.
method Developed a characteristic function and proposed a numerical method for its inversion.
result Exact probability distribution of output quantities can be determined.
Improves graph-based active learning for non-Gaussian models.
problem Efficiently selecting data points for labeling in graph-based semi-supervised learning.
method Approximates non-Gaussian distributions, introduces rank-one update and model change acquisition function.
result Enhanced active learning for graph-based SSL under non-Gaussian models.
New method approximates Gaussian curvature on discrete surfaces.
problem Approximating solutions to the prescribed Gaussian curvature problem.
method Discrete conformality and convex functional minimization.
result Efficient numerical method to compute solutions.
Deep neural networks and Gaussian processes are shown to be equivalent through activation functions.
problem Understanding the relationship between neural networks and Gaussian processes.
method Developing an equivalence theory based on activation functions and kernels.
result Models can be seen as neural networks with improved uncertainty prediction or deep Gaussian processes with increased accuracy.
New method optimizes processes under constraints using bivariate Gaussian models.
problem Optimizing processes with constraints using traditional methods.
method Developed a constrained expected improvement acquisition function using bivariate Gaussian process models.
result Demonstrated improved performance in a manufacturing cure process optimization.
GWI combines deep neural networks with Gaussian processes for better predictive performance and uncertainty quantification.
problem Combining deep learning with Gaussian process uncertainty quantification.
method Gaussian Wasserstein inference (GWI) using Wasserstein distance between Gaussian measures.
result GWI achieves state-of-the-art performance on benchmark datasets.
Wide neural networks can be closely approximated by Gaussian processes, with rates depending on the activation function's properties.
problem Approximating the behavior of wide neural networks using Gaussian processes.
method Established convergence rates for the central limit theorem in an infinite-dimensional functional space, using a transportation distance metric.
result Explicit convergence rates for neural networks approximated by Gaussian processes, varying based on the activation function's properties.
Graph Gaussian processes use Matérn models for better function learning.
problem Lack of Gaussian process models for graph input spaces.
method Stochastic partial differential equation characterization of Matérn Gaussian processes.
result Graph Matérn Gaussian processes inherit properties of Euclidean and Riemannian models and can be trained efficiently.
We show that the visible sector probability density function of the Riemann-Theta Boltzmann machine corresponds to a gaussian mixture model consisting of an infinite number of component multi-variate gaussians. The weights of the mixture are given by a discrete multi-variate gaussian over the hidden state space. This a…
This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.
problem Understanding the relationship between Tensor Networks and Gaussian Processes.
method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.
result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.
This paper aims at refined error analysis for binary classification using support vector machine (SVM) with Gaussian kernel and convex loss. Our first result shows that for some loss functions such as the truncated quadratic loss and quadratic loss, SVM with Gaussian kernel can reach the almost optimal learning rate, p…
DGPFM uses deep Gaussian processes to map functions accurately and quantify uncertainty.
problem Learning mappings between functional spaces, especially when data are noisy, sparse, or irregularly sampled.
method Constructs a sequence of GP-based linear and nonlinear transformations directly in function space, leveraging kernel integral transforms, GP conditional means, and nonlinear activations sampled from Gaussian processes.
result Empirical results show DGPFM outperforms existing methods in predictive accuracy and uncertainty calibration.
Advanced kernels improve Gaussian process accuracy by incorporating domain knowledge.
problem Improving function approximation accuracy in Gaussian processes.
method Advanced kernel designs that enforce specific function properties (symmetry, periodicity) and non-stationarity.
result Advanced kernels significantly enhance function approximation accuracy and relevance.
This paper improves Gaussian process predictions by integrating prior knowledge.
problem Gaussian processes lack predictive power when prior information is ignored.
method Derive mean and covariance functions from previous data using weighted sums of basis functions.
result Integrating prior knowledge significantly increases look-ahead time and accuracy.
The mixture of Gaussian distributions, a soft version of k-means , is considered a state-of-the-art clustering algorithm. It is widely used in computer vision for selecting classes, e.g., color, texture, and shapes. In this algorithm, each class is described by a Gaussian distribution, defined by its mean and covarianc…
In this paper, we present a new statistical approach to the problem of incorporating experimental observations into a mathematical model described by linear partial differential equations (PDEs) to improve the prediction of the state of a physical system. We augment the linear PDE with a functional that accounts for th…
Paper introduces a nonparametric functional graphical model for random functions.
problem Estimating probabilistic conditional independence in functional graphical models.
method Functional sufficient dimension reduction to relax Gaussian or copula Gaussian assumptions.
result Enhances estimation accuracy and retains probabilistic conditional independence.
We present a novel extension of multi-output Gaussian processes for handling heterogeneous outputs. We assume that each output has its own likelihood function and use a vector-valued Gaussian process prior to jointly model the parameters in all likelihoods as latent functions. Our multi-output Gaussian process uses a c…
Proves new concentration inequalities for sub-gaussian and sub-exponential variables.
problem Understanding functions of independent random variables better.
method Sub-gaussian and sub-exponential conditions, Rademacher complexities, Lipschitz function classes.
result Extension of Rademacher complexities to unbounded sub-exponential distributions.
Large deviation principle for deep neural networks with ReLU activation.
problem Understanding the behavior of deep neural networks with ReLU activation.
method Proving a large deviation principle for networks with Gaussian weights and ReLU activation functions.
result Simplified expressions and power-series expansions for the ReLU case.
New analysis explains pathology of deep Gaussian processes.
problem Pathology of deep Gaussian processes reduces learning capacities with increased layers.
method Study nonlinear dynamic systems corresponding to DGPs, derive recurrence relations.
result Provide tighter bounds and rate of convergence for dynamic systems.
The paper improves Gaussian process models for efficient batch optimization.
problem Poor scaling and optimization loop issues in Gaussian process models.
method Dual GP parameterization for linear scaling and non-Gaussian likelihood updates.
result Extends sparse models to greedy batch fantasizing acquisition functions.
Researchers explore gauge freedom in entropies of q-Gaussian measures.
problem Exploring the gauge freedom of entropies in q-Gaussian measures. method Introducing a refined q-logarithmic function to demonstrate gauge freedom. result Different escort expectations can lead to the same entropy but different relative entropies.
Gradient descent memorizes many Gaussians efficiently.
problem Memorizing many Gaussians with minimal parameters.
method Gradient descent on a depth-two neural network.
result One step of gradient descent memorizes $Ω\left(\frac{dq}{\log^4(d)}
ight)$ Gaussians.
Solves challenges in estimating parameters of softmax gating Gaussian mixture models.
problem Identifiability issues and complex interactions in Gaussian mixture of experts.
method Proposes novel Voronoi loss functions and establishes convergence rates of MLE.
result Connects convergence rate of MLE to a solvability problem of polynomial equations.
GNet uses Gaussian processes for scalable, flexible neural networks.
problem Large-scale predictive modeling with high computational and storage costs.
method GNet employs Gaussian processes with nonparametric activation functions and a fast algorithm for training and predictions.
result GNet achieves competitive performance across various test problems, including nonlinear function prediction and real-world data regression.
GNet uses Gaussian processes for scalable, flexible neural networks.
problem Large-scale predictive modeling with high computational and storage costs.
method GNet employs Gaussian processes with nonparametric activation functions and a fast algorithm for efficient training and predictions.
result GNet achieves competitive performance across various test problems, including nonlinear function prediction and real-world data regression.
Bayesian approach uses Gaussian process for reinforcement learning.
problem Robotic locomotion environments
method Bayesian actor-critic, model-free reinforcement learning with Gaussian process for exploration and policy optimization.
result Gaussian process method outperforms current algorithms in robotic locomotion environments.
A scalable Gaussian process clustering method for large datasets.
problem Infeasibility of Gaussian process clustering on large grids.
method Embedding Vecchia approximation in EM algorithm for scalability.
result Efficient Gaussian process clustering for large environmental applications.
Bayesian approach for inhomogeneous Poisson process intensity estimation.
problem Intractable integral in likelihood of Gaussian Cox process.
method Joint modeling of intensity and cumulative intensity as transformed Gaussian process; exact MCMC sampler.
result Exact posterior inference without approximations.
Gaussian process models are flexible, Bayesian non-parametric approaches to regression. Properties of multivariate Gaussians mean that they can be combined linearly in the manner of additive models and via a link function (like in generalized linear models) to handle non-Gaussian data. However, the link function formal…
This paper optimizes Bayesian acquisition functions in Gaussian Processes for better optimization.
problem Improving the efficiency of Bayesian optimization methods.
method Analysis of different acquisition functions and optimizers for optimizing Bayesian acquisition functions.
result Optimization of acquisition functions leads to faster and more accurate sampling points.