Deep convolutional Gaussian processes boost image classification accuracy.
problem Image classification with hierarchical feature combinations.
method Deep Gaussian process architecture with convolutional structure.
result Significantly improved image classification performance on MNIST and CIFAR-10 datasets.
Bayesian deep convolutional GPs improve image classification accuracy.
problem Inaccurate uncertainty estimates in traditional GPs for image classification.
method Translation-insensitive convolutional kernel, multi-output GPs, Bayesian approach.
result Improved performance in single-layer and deep models.
Combining distributed Gaussian Processes with deep CNNs improves performance on action recognition.
problem Improving performance on action recognition datasets.
method Combining distributed Gaussian Processes with multi-stream deep CNNs, treating each CNN as an expert and combining predictions using a Product of Experts (PoE) framework.
result Improves performance on HMDB-51 dataset by 0.4\% compared to hand-crafted feature frameworks.
Convolutional DGP models improve image classification performance.
problem Applying deep Gaussian processes to computer vision tasks.
method Developed Convolutional DGP models using convolution kernels.
result Outperforms strong GP baselines on multi-class image classification.
CNNs become Gaussian processes with many filters, achieving state-of-the-art performance.
problem Training deep CNNs is computationally expensive.
method Showed that CNNs with many filters can be approximated by Gaussian processes, and computed the equivalent kernel efficiently.
result The kernel equivalent of a 32-layer ResNet achieves 0.84% classification error on MNIST.
Improves Gaussian process models for large datasets.
problem Selecting proper covariance functions in Gaussian processes.
method Nonparametric process convolutions and deep GP models.
result Improves performance on benchmarks for GPs, especially for larger datasets.
Deep Gaussian Processes with polynomial kernels can collapse rapidly without proper hyperparameter tuning.
problem The collapse of Deep Gaussian Processes with polynomial kernels without careful hyperparameter tuning.
method Analysis using the Berry-Esseen Theorem and observation of prior behavior.
result The prior of a Deep Gaussian Process collapses rapidly towards zero or places negligible mass on low norm functions without proper hyperparameter tuning.
This paper removes the finite variance assumption for deep convolutional neural networks.
problem Removing the finite variance assumption for deep convolutional neural networks.
method Assuming iid parameters distributed according to a stable distribution, the paper shows that the infinite-channel limit of a deep feed-forward convolutional neural network is a multivariate stable stochastic process.
result The infinite-channel limit of a deep feed-forward convolutional neural network, under suitable scaling, is a multivariate stable stochastic process.
New method calibrates CNN-GP models for better uncertainty quantification.
problem Current CNN-GP models are miscalibrated, leading to unreliable uncertainty estimates.
method Proposes a novel combination of CNNs and GPs to improve calibration.
result Significantly outperforms previous approaches on calibration while maintaining state-of-the-art performance.
Novel CSK kernel improves GP model generalization for non-stationary patterns.
problem Improving generalization of Gaussian process models for non-stationary data.
method Introduced convolutional spectral kernel (CSK) derived from convolution of imaginary radial basis functions, using Fourier transform for interpretation.
result CSK improves GP model generalization on spatiotemporal datasets.
DLFM models complex systems with uncertainty, outperforming traditional methods.
problem Modeling highly nonlinear dynamical systems with robust uncertainty quantification.
method Deep latent force model (DLFM) using physics-informed kernels derived from ODEs.
result DLFM achieves comparable performance to non-physics-informed models on univariate tasks and captures dynamics in real-world data.
Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.
problem Capturing complex non-stationary patterns in non-linear dynamical systems.
method Integrates ODE-based RKHS Fourier features into DGPs using convolution operations for adaptive amplitude and phase modulation. Uses a doubly stochastic variational inference framework.
result Improved predictive performance across various regression tasks.
Study deep maxout networks and their equivalence to Gaussian processes.
problem Understanding neural networks with infinite width.
method Derive equivalence between deep maxout networks and Gaussian processes, characterize maxout kernel, and provide efficient numerical implementation.
result Bayesian inference based on deep maxout network kernel leads to competitive results compared to finite-width counterparts and deep neural network kernels.
Paper introduces non-linear process convolutions for multi-output Gaussian processes.
problem Building accurate covariance functions for multi-output Gaussian processes.
method Volterra series for non-linearity, closed-form expressions for mean and covariance.
result Non-linear model outperforms classical process convolution in synthetic and real datasets.
Convolutional DKMs improve kernel methods on MNIST, CIFAR-10, and CIFAR-100.
problem Improving kernel methods for image classification.
method Developed a novel inter-domain inducing point approximation and introduced various techniques to extend DKMs to convolutional networks.
result Achieved state-of-the-art performance on image classification benchmarks.
Theory captures feature learning effects in finite CNNs.
problem Feature learning in finite deep neural networks.
method Derive a self-consistent Gaussian Process theory.
result Good agreement with experiments and sharp transition between regimes.
Paper examines Gaussian process perspective of CNNs.
problem Understanding when and why CNNs perform well.
method Casts CNNs in a Gaussian process framework.
result Gains insights into CNN performance and assumptions.
Convolutional Gaussian Processes improve image classification accuracy.
problem Improving Gaussian Processes for high-dimensional inputs like images.
method Introducing convolutional structure into Gaussian processes with an inter-domain inducing point approximation.
result Convolutional Gaussian Processes achieve better generalization and faster inference on image datasets.
Graph Convolutional Gaussian Processes predict missing links.
problem Link prediction in large graphs.
method Simplified graph convolutions and variational inducing point method.
result Consistent improvements over existing models and competitive performance.
Graph convolutional Gaussian processes learn functions on graphs.
problem Learning translation-invariant relationships on non-Euclidean domains.
method Bayesian nonparametric method using graph convolutional neural networks.
result Graph convolutional Gaussian processes outperform existing methods on images and triangular meshes.
Framework simplifies GPs for deep learning models.
problem Inadequate implementations for variations in Gaussian processes.
method Modular system for scalable approximate inference in Gaussian processes.
result Unified interface for multioutput models and convolutional structures.
Bayesian approach improves deep image prior for image reconstruction.
problem Improving performance of deep image prior for image reconstruction tasks.
method Derive Bayesian approach using stochastic gradient Langevin, showing asymptotic equivalence to Gaussian process prior.
result Improves denoising and impainting results for image reconstruction tasks.
Infinitely wide GCNs perform as GPs for graph semi-supervised learning.
problem Graph-based semi-supervised classification with limited labeled data.
method Proposes GPGC model combining GCNs and GPs for semi-supervised learning.
result GPGC outperforms state-of-the-art methods on various datasets.
New interpretation of sparse Gaussian process approximations for scalability.
problem Scalability issues in Gaussian process models.
method Decomposes Gaussian process into two independent components: inducing points and remaining variation, leading to tighter bounds and new algorithms.
result Demonstrates efficiency in various Gaussian process models, including deep convolutional ones, achieving state-of-the-art results.
Graph convolutional deep kernel machine learns representations for graph tasks.
problem Limited representation learning in infinite-width neural networks.
method Developed a graph convolutional deep kernel machine as an infinite-width limit.
result Representation learning improves performance for heterophilous node classification tasks.
Bayesian CNNs with many channels are equivalent to Gaussian processes.
problem Understanding the behavior of deep convolutional networks in the infinite channel limit.
method Deriving an equivalence between multi-layer convolutional neural networks and Gaussian processes, introducing a Monte Carlo method for estimation.
result The GPs corresponding to CNNs with and without weight sharing are identical in the infinite channel limit.
Deep neural networks learn by averaging fast variables, revealing a Gaussian process.
problem Analyzing the complex behavior of deep neural networks (DNNs) with billions of parameters.
method Identifying slow variables that average the erratic behavior of fast microscopic variables in fully trained DNNs.
result DNN layers couple only through the second moment (kernels) of their activations and pre-activations, which fluctuate in a nearly Gaussian manner.
IAE extracts innovations sequences for non-Gaussian processes.
problem Extracting innovations sequences for non-Gaussian processes.
method Causal convolutional neural network.
result IAE effectively detects anomalies in non-Gaussian data.
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
This paper extends the Gaussian process interpretation of deep networks to more varied weight distributions.
problem Understanding the impact of different weight initialization schemes on deep learning dynamics.
method Extending the Gaussian process interpretation to PSEUDO-IID weight distributions, including sparse and low-rank networks.
result PSEUDO-IID initialized networks are effectively equivalent up to variance, enabling tractable posterior distributions.
A new kernel for multi-output Gaussian processes reduces undesirable scale effects.
problem Predicting multiple output variables simultaneously with Gaussian processes.
method Design a new kernel (MOCSM) using convolution in the spectral domain to model cross channel dependencies.
result MOCSM kernel reduces undesirable scale effects compared to the Multi-Output Spectral Mixture kernel.
Graph transformers outperform graph convolutions by preserving community information.
problem Understanding why graph transformers perform well in node-level prediction tasks.
method Analyzing the Gaussian process limits of graph transformers with infinite width and infinite heads.
result Graph transformers maintain discriminative node representations even in deep layers, preventing oversmoothing.
Bayesian Optimization improves hyperparameter tuning for ConvNets.
problem Optimizing hyperparameters in ConvNets is crucial but computationally expensive.
method Sequential model-based Bayesian Optimization with Gaussian process prior.
result Bayesian Optimization achieves lower error rates in ConvNets.
Analyzes DNNs trained with noisy gradients, finding FWCs negligible for large n.
problem Analyzing DNNs trained with noisy gradients.
method Introduced analytical framework to analyze non-Gaussian stochastic process.
result FWCs negligible for large n, improving CNN performance.
This work extends Gaussian process priors to neural operators for function space mappings.
problem Improving uncertainty quantification in deep neural networks.
method Extending Gaussian process priors to neural operators with conditions for convergence and computation of covariance functions.
result Arbitrary-depth neural operators with Gaussian kernels converge to function-valued GPs, enabling posterior computation in regression scenarios.
Improved inference for heterogeneous multi-output Gaussian processes using natural gradient optimization.
problem Challenges in adaptive gradient optimization for multi-output Gaussian processes.
method Introducing a fully natural gradient scheme to overcome optimization issues.
result Better local optima solutions and higher test performance rates compared to adaptive gradient methods.
Deep learning and lazy learner improve early sepsis detection.
problem Detecting sepsis early in high-resolution ICU records.
method Deep learning model with temporal convolutional network and Gaussian Process Adapter; lazy learner with dynamic time warping.
result Improves sepsis detection from 0.25 to 0.40 AUPRC 7 hours before onset.
DeepCAM learns convolutional dictionaries for image processing.
problem Processing high-dimensional signals like images efficiently.
method Introduces a Deep Convolutional Analysis Dictionary Model (DeepCAM) using convolutional dictionaries.
result DeepCAM achieves performance comparable to other methods on single image super-resolution.
Improved model for non-smooth signals with complex spectra.
problem Current models struggle with non-smooth signals and complex spectral structures.
method CGPCM and RGPCM models with causality and Bayesian nonparametric interpretations, improved variational inference.
result Proposed models show better performance on synthetic and real-world data.
Bayesian Gaussian Processes layer detects out-of-distribution data in medical imaging.
problem Detecting out-of-distribution data in medical imaging tasks.
method Parameter-efficient hierarchical convolutional Gaussian Processes in Wasserstein-2 space.
result Uncertainty estimates enable superior out-of-distribution detection compared to previous methods.
Study on Bayesian deep linear networks with multiple outputs and convolutional layers.
problem Characterize feature learning in finite-width Bayesian deep linear networks.
method Exact and analytical formulas for joint and posterior distributions, using large deviation theory.
result Quantitative description of feature learning in infinite-width regime.
New model solves PDEs using probabilistic random grids.
problem Solving parametric PDEs with probabilistic collocation grids.
method Random Grid Neural Processes (RGNPs) with GICNets.
result Significant computational advantages and improved predictive capabilities.
Time series analysis is a key component of machine learning, with applications in various fields.
problem Time series analysis in machine learning
method Basic concepts, classical statistical models, modern machine learning approaches
result Machine learning techniques for time series analysis
SVGP KAN integrates uncertainty quantification into Kolmogorov-Arnold networks.
problem Uncertainty quantification in scientific machine learning models.
method Sparse variational Gaussian process inference with Kolmogorov-Arnold topology.
result Demonstrated ability to distinguish aleatoric and epistemic uncertainty in various scientific applications.
Characterization of lung nodules as benign or malignant is one of the most important tasks in lung cancer diagnosis, staging and treatment planning. While the variation in the appearance of the nodules remains large, there is a need for a fast and robust computer aided system. In this work, we propose an end-to-end tra…
Framework predicts events from longitudinal and time-to-event data using MGCP and Cox model.
problem Predicting events from mixed longitudinal and time-to-event data.
method Uses multivariate Gaussian convolution process (MGCP) and Cox model for joint modeling. Implements variational inference to estimate parameters.
result Framework outperforms state-of-the-art approaches in synthetic and real-world data.
Unified deep framework for personalized recommendations with uncertainty.
problem Uncertainty in user preferences in recommendation systems.
method Gaussian embeddings, Monte-Carlo sampling, convolutional neural networks.
result Superior performance in recommendation accuracy compared to state-of-the-art models.
The paper analyzes how gradient descent learns convolutional filters for non-Gaussian inputs.
problem Learning convolutional filters with ReLU for non-Gaussian input distributions.
method Analysis of gradient descent convergence for ReLU activation with polynomial time complexity.
result Gradient descent can learn convolutional filters in polynomial time, with convergence rate dependent on input distribution smoothness and patch similarity.