Random neural networks with ReLU activations are non-Gaussian processes.
problem Understanding the behavior of neural networks with random initialization and rectified linear units.
method Proving these networks are non-Gaussian processes and deriving their properties.
result These networks can converge to non-Gaussian processes under certain conditions.
We derive Gaussian approximations for random forest predictions using region-based stabilization.
problem Improving the accuracy of random forest predictions for Poisson process data.
method Region-based stabilization and Malliavin-Stein method for multivariate Gaussian approximation.
result Established Gaussian approximation bounds for random forest predictions under Poisson process.
Gaussian process quadrature improves moment transformation accuracy.
problem Computing moments of transformed Gaussian variables with error accounting.
method Bayesian quadrature (Gaussian process quadrature) for numerically estimating integrals.
result Proposed method outperforms classical quadrature methods in accuracy.
The paper analyzes and mitigates biases in scalable Gaussian Process methods.
problem Modeling biases in scalable Gaussian Process methods.
method Randomized truncation estimators to eliminate bias in exchange for increased variance.
result Randomized truncation estimators meaningfully outperform biased counterparts with minimal additional computation.
Paper introduces a new method for Gaussian Processes that improves prediction and hyper-parameter optimization.
problem Efficiently predicting unknown functions and optimizing hyper-parameters in Gaussian Processes.
method Sequential randomized low-rank matrix factorization for incremental predictions and hyper-parameter optimization.
result The proposed method outperforms existing approaches in terms of accuracy and computational efficiency.
Gaussian processes have been successful in both supervised and unsupervised machine learning tasks, but their computational complexity has constrained practical applications. We introduce a new approximation for large-scale Gaussian processes, the Gaussian Process Random Field (GPRF), in which local GPs are coupled via…
Generalizes randomized SVD for better matrix approximations using Gaussian vectors.
problem Computing accurate rank-k approximations of matrices with limited data.
method Extends randomized SVD to multivariate Gaussian vectors, incorporating prior knowledge and using Gaussian processes.
result Demonstrates improved accuracy in approximating matrices and Hilbert-Schmidt operators.
Quantum-assisted Gaussian process speeds up data regression.
problem High computational complexity of Gaussian process regression for large datasets.
method Quantum-assisted sparse Gaussian process regression using random Fourier features.
result Achieves polynomial-order computational speedup compared to classical methods.
New method trains DGP models with random feature expansions for scalable inference.
problem Scalability and inference complexity in Deep Gaussian Processes.
method Random feature expansions combined with stochastic variational inference.
result Significantly advanced inference for Deep Gaussian Processes, scalable to large datasets.
Ensembles dynamic models using random feature approximations.
problem Online scalable Bayesian learning with dynamic models and ensembling.
method Random feature approximations and dynamic models using random walks.
result Better performance with alternative basis expansions like Hilbert space Gaussian processes.
The paper analyzes how subsampling Gaussian process regression works.
problem The accuracy and generalization of Gaussian process regression with subsampling.
method The authors use graphon theory to embed kernel matrices and analyze the approximation and generalization errors.
result Subsampling achieves a better trade-off between accuracy and runtime compared to other methods.
Improved kernel ridge regression for large datasets using weighted random binning.
problem Efficiently approximating kernel matrices for large-scale datasets.
method Introduced weighted random binning features for locality sensitive hashing.
result Weighted random binning features generate Gaussian processes of any desired smoothness.
Enhanced Gaussian process models accelerate optimization and posterior approximation.
problem Improving the accuracy and speed of Gaussian process models for optimization and inference.
method Introduces a random exploration step to classical GP-UCB algorithms, facilitating faster convergence.
result New algorithms achieve nearly optimal convergence rates and provide bounds for Hellinger distance.
Deep Random Splines model neural activity data with better dimensionality.
problem Modeling neural population data with shape constraints.
method Deep neural network transforming Gaussian noise into spline parameters.
result Better dimensionality reduction of neural spiking activity.
The paper links deep neural networks to Gaussian processes, showing convergence under certain conditions.
problem Understanding theoretical properties of deep neural networks.
method Study of random, wide, fully connected feedforward networks and Gaussian processes with recursive kernels.
result As network width increases, random functions converge to Gaussian processes under broad conditions.
Scalable algorithm for sampling Gaussian processes using sparse grids and preconditioners.
problem Generating high-dimensional Gaussian random vectors for GP sampling is computationally challenging.
method Proposes a scalable algorithm using inducing points approximation with sparse grids and additive Schwarz preconditioners.
result Demonstrates the efficacy and accuracy of the proposed method through experiments and comparisons.
New methods reduce computational cost for Gaussian Markov Random Fields with sparse constraints.
problem Inference and simulation of GMRFs are computationally prohibitive with many constraints.
method Proposes a basis transformation into blocks of constrained and non-constrained subspaces.
result Significantly outperforms existing alternatives in computational cost.
New method improves Gaussian kernel approximations for high-frequency data.
problem Limited scalability of kernel-based models to large data sets.
method Local random feature approximations using Maclaurin expansions and polynomial sketches.
result Significant improvement in kernel approximations and downstream performance for high-frequency data.
Paper calculates KL divergence for isotropic Gaussian-Markov fields.
problem Measuring divergence between isotropic Gaussian-Markov fields.
method Derives closed-form KL divergence expressions.
result Develops new similarity measures in image processing.
This paper uses random Fourier features to simplify latent force models and convolved Gaussian processes.
problem Expensive covariance matrix calculation in latent force models due to double integrals.
method Approximates double integrals using random Fourier features to obtain simpler analytical expressions.
result Simplified analytical expressions for covariance functions, leading to faster computation.
Bounds neural network output distribution to Gaussian for random initialization.
problem Quantifying the distribution of randomly initialized deep neural networks.
method Quantitative Gaussian approximation using quadratic Wasserstein distance.
result Explicit inequalities show how network sizes affect Gaussian behavior.
Gaussian processes adapted for non-Euclidean spaces enhance decision-making.
problem Applying Gaussian processes in non-Euclidean spaces.
method Developed pathwise conditioning and Gaussian process models over non-Euclidean spaces.
result Efficient Gaussian process models for non-Euclidean spaces.
Quantum neural networks converge to Gaussian processes as they grow.
problem Understanding the convergence of quantum neural networks to Gaussian processes.
method Analyzing Haar random unitary and orthogonal deep QNNs, considering input states, measurement observables, and non-independence of unitary matrix entries.
result Quantum neural networks outputs converge to Gaussian processes in the limit of large Hilbert space dimension.
Finite-width neural networks use non-Gaussian priors, extending Gaussian process theory.
problem Understanding the behavior of neural networks with finite width.
method Perturbative extension of Gaussian process theory to finite-width neural networks, tracking preactivation distributions.
result Non-Gaussian processes as priors in finite-width neural networks.
Efficient inference for nonparametric Hawkes processes using Pólya-Gamma augmentation.
problem Efficient inference for nonparametric Hawkes processes.
method Pólya-Gamma augmentation, EM algorithm, mean-field variational inference.
result The proposed algorithms can recover well the underlying prompting characteristics efficiently.
Bayesian inference for wide neural networks using Edgeworth expansion.
problem Analyzing the non-Gaussian behavior of wide neural networks in Bayesian inference.
method Proposed a non-Gaussian distribution using multivariate Edgeworth expansion for finite-width neural networks.
result Derived non-Gaussian posterior distribution in Bayesian regression tasks.
Random projections improve GP regression performance, reducing high-dimensional inputs to 1D.
problem Gaussian processes struggle with high-dimensional inputs, leading to overfitting and high computational cost.
method Use additive sums of kernels operating on random projections of inputs.
result Predictive performance converges to full-dimensional kernel performance with increasing projections, even in 1D.
The article explores Gaussian processes and Bayesian optimization in financial applications.
problem Modeling financial markets and optimizing strategies.
method Gaussian processes and Bayesian optimization methods.
result Gaussian processes and Bayesian optimization are effective in financial modeling and strategy construction.
New random forest method provides optimal rates and confidence bands.
problem Improving random forest regression rates and constructing confidence bands.
method Proposed Ehrenfest centered purely random forests achieve optimal rates; used Gaussian approximation for supremum of empirical processes.
result Explicit asymptotic uniform confidence bands constructed for both random forest types.
Quantitative CLTs show neural network distributions converge to Gaussian as width increases.
problem Understanding the distribution of fully connected neural networks with random weights and biases.
method Analyzing the distribution of a fully connected neural network with random Gaussian weights and biases, proving quantitative bounds on normal approximations.
result The distance between a random fully connected network and the corresponding infinite width Gaussian process scales like n−γ for γ>0. New bounds on random quadratic forms hold under dependence, useful for adaptive modeling.
problem Need for independence in bounds on random quadratic forms.
method Uniform bounds on random quadratic forms of conditionally independent and sub-Gaussian stochastic processes.
result Bounds hold under general dependencies and sequential design.
Study Gaussian-process limits of neural networks using tensor programs.
problem Understanding the behavior of neural networks as they approach infinite width.
method Quantitative analysis through tensor programs and Wasserstein distance.
result Explicit finite-width error bounds, showing convergence to Gaussian-process limits.
This work introduces efficient sampling methods for Gaussian processes by focusing on pathwise conditioning.
problem Intractable mathematical expressions in Gaussian process posteriors limit practical applications.
method Investigates a pathwise interpretation of conditioning to derive efficient sampling methods.
result Derives a general family of approximations that allow for efficient sampling of Gaussian process posteriors.
Extends Gaussian process theory to Banach spaces.
problem Extending Gaussian process theory to Banach spaces.
method Investigates the connection between Gaussian processes and Gaussian random elements in reproducing kernel Banach spaces.
result Characterizes positive definite functions that arise from covariance operators in Banach space setting.
BART's performance improves with more trees, converging to a Gaussian process.
problem Understanding and explaining BART's superior performance in prediction and causal inference.
method Analyzing BART as the number of trees grows towards infinity, showing convergence to a Gaussian process.
result BART converges to a Gaussian process with favorable inferential properties, explaining its excellent performance.
Develops a new method for nonlinear dimension reduction using random features.
problem Statistical challenges in generalizing Gaussian process-based latent variable models to non-Gaussian data.
method Random feature latent variable models (RFLVMs) that approximate nonlinear relationships with linear functions of random features.
result RFLVMs produce comparable results to state-of-the-art methods on various data types.
Paper connects neural networks to Gaussian processes for understanding double-descent.
problem Understanding the double-descent phenomenon in neural networks.
method Uses techniques from random matrix theory and Gaussian processes.
result Establishes a connection between NNGP and random matrix theory for neural networks.
Bayesian model for survival analysis using Gaussian processes.
problem Handling various types of censoring and nonparametric modeling of survival data.
method Semi-parametric Bayesian model with a Gaussian process for nonparametric variation.
result Model performs better than competing methods on synthetic and real data.
New bounds for optimal transport using Gaussian processes and rate-distortion functions.
problem Finding bounds for entropic optimal transport with mutual information constraints.
method Lifting technique to construct a Gaussian process and applying the majorizing measure theorem.
result Maximum expected inner product is equivalent to a truncated integral involving the rate-distortion function.
Introduces numerical Gaussian process Kalman filtering for infinite-dimensional systems.
problem Kalman filtering on infinite-dimensional systems.
method Embedding numerical Gaussian processes into Kalman filter equations.
result Ability to perform Kalman filtering on infinite-dimensional systems using Gaussian processes.
VIBNN accelerates Bayesian Neural Networks on FPGAs for efficient inference.
problem Overfitting and small-data training issues in BNNs.
method Hardware accelerator design for variational inference on BNNs, using novel Gaussian random number generators.
result VIBNN achieves high throughput and energy efficiency on FPGA, matching software performance.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
We study pathwise invariances of centred random fields that can be controlled through the covariance. A result involving composition operators is obtained in second-order settings, and we show that various path properties including additivity boil down to invariances of the covariance kernel. These results are extended…
Tutorials on preference learning with Gaussian Processes.
problem Understanding individual preferences and choices for efficient and personalized applications.
method Presentation of a comprehensive framework for preference learning with Gaussian Processes, incorporating rationality principles.
result Construction of preference learning models that encompass various utility models and scenarios.
We introduce a new method to explain Gaussian processes using Shapley values.
problem Explaining the uncertainty in Gaussian process models.
method Extending Shapley values to stochastic cooperative games for Gaussian processes.
result Our method generates explanations that are random variables and satisfy favorable axioms.
Wide neural networks converge to Gaussian processes, with implications for kernel behavior and gradient dynamics.
problem Understanding the behavior of wide neural networks and their convergence to Gaussian processes.
method Introducing a tensor program framework to study scaling limits of neural networks, characterizing their behavior under large tensor sizes and randomization.
result Wide neural networks converge to Gaussian processes, with implications for kernel behavior and gradient dynamics.
New method for ancestral inference in branching processes with random environments.
problem Determining ancestor distribution parameters in branching processes with random environments.
method Generalized method of moments for ancestral inference.
result Limiting distribution of ancestor and offspring estimators decouple and converge to independent Gaussian variables under certain conditions.
A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.
problem Optimizing robustness measures under input uncertainty.
method Randomized robustness measure GP-UCB (RRGP-UCB) that samples β from a chi-squared-based distribution.
result RRGP-UCB provides tight bounds on expected regret.