Deep learning models converge to Gaussian dynamics with mixed structured inputs.
problem Understanding neural network dynamics with complex input distributions.
method Extended hidden manifold model to Gaussian mixtures, analyzed via SGD.
result Learning dynamics with mixed inputs converge to Gaussian behavior.
Simplified DGPs training by fixing inducing inputs to subset of data.
problem Challenging training of deep Gaussian processes.
method Fixed subset of data for inducing inputs, variational sampling.
result Significant reduction in trainable parameters and computation cost without performance degradation.
The ACCRU framework improves probabilistic forecasts by capturing input-dependent uncertainty.
problem Uncertainty in deterministic predictions, especially for skewed and non-Gaussian errors.
method Neural network trained with a loss function balancing accuracy and reliability to learn input-dependent, non-Gaussian uncertainty distributions.
result Improves probabilistic forecasts relative to existing methods, capturing skewed and non-Gaussian errors.
Safe active learning for time-series models with Gaussian processes.
problem Learning time-series models while respecting safety constraints.
method Employing Gaussian processes with a nonlinear exogenous input structure, the approach dynamically explores the input space to generate data for model learning.
result The approach effectively learns time-series models under safety constraints, as demonstrated in a technical application.
Bayesian Gaussian process models handle uncertain data locations in PDE approximations.
problem Handling uncertainties in data locations for PDE approximations.
method Bayesian inference of uncertain inputs integrated into Gaussian process predictions.
result Substantial reduction in predictive uncertainties achieved through Bayesian inference.
Kernel improves Gaussian process scalability for wide datasets.
problem Under-performance of Gaussian processes on wide data.
method Introduces Bézier Gaussian Process kernel with exponential summarising variables growth and linear cost.
result Empirically demonstrates scalability to both tall and wide datasets.
ICA reveals deep learning's feature learning mechanisms from non-Gaussian data.
problem Understanding feature learning from non-Gaussian inputs in deep neural networks.
method Investigates ICA and SGD on synthetic and real data.
result FastICA requires n≳d4 samples for single non-Gaussian direction recovery, while SGD outperforms and optimised SGD reaches n≳d2. WE constructs GP kernels for mixed inputs using weighted EDMs.
problem Limitation of standard GP models in handling categorical variables.
method WEGP constructs kernel function using weighted EDMs for categorical inputs.
result WEGP improves GP model accuracy in both synthetic and real-world optimization problems.
A new method for Gaussian Processes handles mixed continuous and categorical inputs.
problem Modeling cross-correlations between continuous and categorical data.
method Low-Rank Correlation (LRC) method for Gaussian Processes with flexible rank approximation.
result LRC outperforms existing methods in estimating cross-correlations and predicting response surfaces.
We study the Gaussian Process regression model in the context of training data with noise in both input and output. The presence of two sources of noise makes the task of learning accurate predictive models extremely challenging. However, in some instances additional constraints may be available that can reduce the unc…
A new method warps inputs to learn nonstationary kernels efficiently.
problem Learning nonstationary patterns in data with varying smoothness.
method Sparse spectrum Gaussian processes with input warping as conditional Gaussian measures.
result Efficient learning of nonstationary patterns with fewer parameters.
We propose a novel deep learning paradigm of differential flows that learn a stochastic differential equation transformations of inputs prior to a standard classification or regression function. The key property of differential Gaussian processes is the warping of inputs through infinitely deep, but infinitesimal, diff…
Study shows perceptrons with random labels perform similarly to Gaussian data.
problem The assumption of Gaussian input data is often seen as a limitation in machine learning.
method Analyzed generalized linear classification (perceptron model) with random labels.
result Minimum training loss is independent of data covariance for high-dimensional input data.
Gaussian processes are used in machine learning to learn input-output mappings from observed data. Gaussian process regression is based on imposing a Gaussian process prior on the unknown regressor function and statistically conditioning it on the observed data. In system identification, Gaussian processes are used to …
GGMPs improve non-Gaussian conditional density estimation.
problem Multimodality, heteroscedasticity, and strong non-Gaussianity in conditional density estimation.
method GGMP combines local Gaussian mixture fitting, cross-input component alignment, and per-component heteroscedastic GP training.
result GGMPs improve distributional approximation on synthetic and real-world datasets.
LMGPs extend GPs to handle mixed data, offering better accuracy and interpretability.
problem Handling mixed data types (quantitative and qualitative) in metamodeling.
method Introduce LMGPs that learn a latent manifold for qualitative inputs, using a low-rank linear map.
result LMGPs outperform existing methods in accuracy and versatility.
Derives moments of PL networks for robust DNNs.
problem Analyzing robustness of DNNs to adversarial attacks.
method Derives exact analytic expressions for moments of PL networks, generalizes variance expression, and constructs sparse and smooth adversarial attacks.
result New variance expressions are tighter and can be efficiently approximated.
The class of Gaussian Process (GP) methods for Temporal Difference learning has shown promise for data-efficient model-free Reinforcement Learning. In this paper, we consider a recent variant of the GP-SARSA algorithm, called Sparse Pseudo-input Gaussian Process SARSA (SPGP-SARSA), and derive recursive formulas for its…
Gaussian processes with differential privacy protect both inputs and outputs.
problem Previous DP methods only protected model outputs, not inputs.
method Sparse GP with private variational approximation, adjusting covariance for DP noise.
result Accurate models can be produced under strong privacy protection with sufficient data.
Proposes GPHMEs using Gaussian processes for hierarchical expert models.
problem Hierarchical mixtures of experts with complex gating functions.
method Gaussian process-gated hierarchical mixtures of experts (GPHMEs) with non-linear gating and expert functions.
result Outperforms tree-based HMEs and achieves good performance with reduced complexity.
Enhances multi-fidelity modeling with DGPs for different input domains.
problem Improving prediction accuracy with multi-fidelity models using different input domains.
method Extends Deep Gaussian Processes (DGPs) to handle different input domains for high and low-fidelity models.
result Demonstrates improved performance on real-world physical problems.
Paper introduces a new model to handle multi-task learning across different input domains.
problem Learning correlated tasks across varying input domains.
method Develops a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) for multi-task learning.
result The proposed model outperforms existing models in diverse multi-task scenarios.
Improves active learning efficiency by warping input space based on observed outputs.
problem Insensitivity of Gaussian process uncertainty to actual observations.
method Input warping with learned monotone reparameterization to adjust acquisition function behavior.
result Significantly improved sample efficiency across various benchmarks, especially in non-stationary conditions.
Gaussian processes (GPs) provide flexible distributions over functions, with inductive biases controlled by a kernel. However, in many applications Gaussian processes can struggle with even moderate input dimensionality. Learning a low dimensional projection can help alleviate this curse of dimensionality, but introduc…
The problem of combined state and input estimation of linear structural systems based on measured responses and a priori knowledge of structural model is considered. A novel methodology using Gaussian process latent force models is proposed to tackle the problem in a stochastic setting. Gaussian process latent force mo…
Gaussian Process (GP) regression models typically assume that residuals are Gaussian and have the same variance for all observations. However, applications with input-dependent noise (heteroscedastic residuals) frequently arise in practice, as do applications in which the residuals do not have a Gaussian distribution. …
Active learning selects inputs for GPSSM to learn latent states.
problem Optimally learn latent states of a GPSSM through active selection of inputs.
method Use mutual information to select informative inputs; approximate mutual information for GPSSM.
result Effective active learning of GPSSM dynamics in physical systems.
We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables,…
This paper addresses Gaussian Process regression over probability measures, revealing a non-stationarity issue between Euclidean and Wasserstein kernels.
problem Non-stationarity issue between Euclidean and Wasserstein kernels in Gaussian Process regression over probability measures.
method Assuming Euclidean input space, applying algebraic transformation based on uncovered non-stationarity relationship to create a non-stationary and Wasserstein-based Gaussian Process model.
result An algebraic transformation simplifies learning a non-stationary Gaussian Process model over probability measures.
We propose a novel Bayesian nonparametric method to learn translation-invariant relationships on non-Euclidean domains. The resulting graph convolutional Gaussian processes can be applied to problems in machine learning for which the input observations are functions with domains on general graphs. The structure of thes…
New findings show learning deeper neural networks is hard even with Gaussian inputs and non-degenerate weights.
problem The computational complexity of learning neural networks, especially deeper ones.
method Smoothed analysis framework and local pseudorandom generators.
result Learning depth-3 ReLU networks under Gaussian input distribution is hard even if weight matrices are non-degenerate.
The paper analyzes why Gaussianization slows down with higher dimensions and proposes a solution.
problem The convergence rate of Gaussianization slows down as the dimension increases.
method Analytical and empirical analysis of Gaussianization with random rotations.
result The number of required layers scales linearly with the dimension for Gaussian input.
In this paper, we propose an auto-encoder based generative neural network model whose encoder compresses the inputs into vectors in the tangent space of a special Lie group manifold: upper triangular positive definite affine transform matrices (UTDATs). UTDATs are representations of Gaussian distributions and can strai…
Graph Gaussian processes use Matérn models for better function learning.
problem Lack of Gaussian process models for graph input spaces.
method Stochastic partial differential equation characterization of Matérn Gaussian processes.
result Graph Matérn Gaussian processes inherit properties of Euclidean and Riemannian models and can be trained efficiently.
Model predicts composite structures assembly quality with input uncertainty.
problem Accurate prediction of dimensional deviations and residual stress in composite structures assembly.
method Neural Network Gaussian Process considering input uncertainty.
result NNGPIU model outperforms other methods for nonsmooth, nonlinear responses.
A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.
problem Optimizing robustness measures under input uncertainty.
method Randomized robustness measure GP-UCB (RRGP-UCB) that samples β from a chi-squared-based distribution.
result RRGP-UCB provides tight bounds on expected regret.
Hardness proven for learning neural networks with polynomial size and Gaussian inputs.
problem Learning one hidden layer ReLU neural networks with polynomial size and Gaussian inputs.
method Based on the hardness of the Continuous Learning with Errors (CLWE) problem.
result Hardness of learning neural networks is proven under standard cryptographic assumptions.
Optimal kernel learning improves GP regression for high-dimensional inputs.
problem High computational costs and low prediction accuracy in GP models with many inputs.
method Approximates GP covariance with a convex combination of kernel functions, identifying active variables.
result Improves prediction accuracy and correctly identifies active input variables.
We introduce fully scalable Gaussian processes, an implementation scheme that tackles the problem of treating a high number of training instances together with high dimensional input data. Our key idea is a representation trick over the inducing variables called subspace inducing inputs. This is combined with certain m…
Develops new oracle inequalities for Gaussian ranking estimators.
problem Lack of rigorous theoretical support for Gaussian ranking estimators.
method Novel oracle inequalities for regularized pairwise ranking.
result Derives fast learning rates under general dimension assumptions.
A new framework for recycling Gaussian process approximations.
problem Efficiently combining multiple Gaussian process approximations.
method Construct variational ensembles using a dictionary of fitted Gaussian processes.
result Framework allows for various tasks and scalability.
Paper presents characteristic function of Tsallis q-Gaussian and its applications.
problem Modeling input quantities in measurement models using Tsallis q-Gaussians.
method Developed a characteristic function and proposed a numerical method for its inversion.
result Exact probability distribution of output quantities can be determined.
Gaussian processes are improved to account for input noise in earth observation.
problem Accurate error assessment in earth observation models.
method Propose a GP model that propagates input noise through the pipeline.
result Improved error representation in temperature predictions from infrared data.
Study on neuron dynamics for XOR classification with zero-margin.
problem Understanding neural network training dynamics in zero-margin classification problems.
method Analysis of Gaussian XOR problem, focusing on neuron block dynamics and generalization without margin assumptions.
result Neurons cluster into four directions and block-level signals evolve coherently, essential for reliable prediction in the Gaussian setting.
We propose non-stationary spectral kernels for Gaussian process regression. We propose to model the spectral density of a non-stationary kernel function as a mixture of input-dependent Gaussian process frequency density surfaces. We solve the generalised Fourier transform with such a model, and present a family of non-…
Using integration by parts on Gaussian space we construct a Stein Unbiased Risk Estimator (SURE) for the drift of Gaussian processes using their local and occupation times. By almost-sure minimization of the SURE risk of shrinkage estimators we derive an estimation and de-noising procedure for an input signal perturbed…
Improves Bayesian optimisation for engineering design problems with many variables.
problem Efficiently searching for global minima in high-dimensional design spaces.
method Integrates input and output data to identify a reduced latent subspace using probabilistic partial least squares.
result Significant improvements in convergence to the global minimum compared to existing methods.
Framework optimizes multiple objectives considering input uncertainty.
problem Efficiently optimizing multiple objectives with input uncertainty.
method Robust Gaussian Process model and two-stage Bayesian optimization process.
result Found a robust Pareto frontier considering input uncertainty.