Neural non-stationary spectral kernels improve performance on benchmark datasets.
problem Learning and discovering complex patterns in data.
method Generalized spectral mixture kernels with input-dependent functions modeled as Gaussian processes and hyperparameter functions as neural networks.
result Neural non-stationary spectral kernels achieve the best performance on benchmark datasets.
Study spectral properties of radial kernels for high-dimensional mixtures.
problem Understanding spectral properties of radial kernels for high-dimensional mixtures.
method High-dimensional analysis focusing on concentration properties of components in mixtures.
result Kernel PCA can successfully cluster mixtures with common means but different covariances, even in high dimensions.
A new kernel for multi-output Gaussian processes reduces undesirable scale effects.
problem Predicting multiple output variables simultaneously with Gaussian processes.
method Design a new kernel (MOCSM) using convolution in the spectral domain to model cross channel dependencies.
result MOCSM kernel reduces undesirable scale effects compared to the Multi-Output Spectral Mixture kernel.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Develops nonstationary MOGP kernels for better performance.
problem Limited applicability of existing MOGP kernels for nonstationary data.
method Harmonizable spectral mixture kernels for nonstationary MOGP.
result Automatic identification of nonstationary behavior in data.
PCKID kernel improves spectral clustering on incomplete data.
problem Handling incomplete data in spectral clustering.
method Combining posterior distributions of Gaussian Mixture Models on different scales.
result PCKID kernel outperforms baseline methods for all fractions of missing values.
Paper compresses SM kernels with time-phase modulated dependency structures for better GP performance.
problem Improving the expressiveness and generalization of Gaussian processes with complex patterns.
method Introducing time-phase modulated dependency structures and a novel structure adaptation algorithm to compress SM kernels.
result The proposed SMD kernel shows improved performance on both synthetic and real-life applications.
Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…
New spectral mixture kernels improve MOGP cross-covariance interpretation.
problem Limited parametric interpretation of cross-covariances in MOGPs.
method Complex-valued cross-spectral densities, Cramér's Theorem, phase shifts, delays.
result Improved expressive and interpretable multivariate covariance functions.
Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.
problem Inefficient marginalization over Gaussian Process kernels for large datasets.
method Bayesian Quadrature scheme with maximum mean discrepancies and invariances between Spectral Mixture kernels.
result Achieves more accurate predictions and better calibrated uncertainty than state-of-the-art baselines.
New kernel HMK improves Gaussian process expressiveness and supports harmonizable covariances.
problem Improving the expressiveness of Gaussian processes with non-stationary kernels.
method Proposed harmonizable mixture kernel (HMK) and variational Fourier features.
result HMK interpolates between local patterns and offers robust kernel learning.
In this paper we propose a family of tractable kernels that is dense in the family of bounded positive semi-definite functions (i.e. can approximate any bounded kernel with arbitrary precision). We start by discussing the case of stationary kernels, and propose a family of spectral kernels that extends existing approac…
Enhances GPLVM for multi-view data with scalable latent representation learning.
problem Limited kernel expressiveness and computational inefficiency in multi-view GPLVM.
method Introduces a new duality between spectral density and kernel function, uses NG-SM kernel, and applies random Fourier feature approximation for scalability.
result Consistently outperforms state-of-the-art models in learning meaningful latent representations across diverse datasets.
A new Lévy process kernel model for robust function extrapolation.
problem Kernel uncertainty in Gaussian process predictions for long-range extrapolation.
method Modeling spectral mixture density with a Lévy process to form a distribution over kernels.
result Automatic and data-efficient learning, long-range extrapolation, and state-of-the-art predictive performance.
Scalable model detects multidimensional changes in data.
problem Detecting and characterizing smooth multidimensional changepoints.
method Random Kitchen Sink features and spectral mixture kernels for flexible and expressive modeling, with additive non-separable kernels for scalability.
result Model identifies previously unknown heterogeneous changes in space and time.
New kernels model non-stationary data efficiently.
problem Efficiently modeling non-stationary data with Gaussian processes.
method Model spectral density as a mixture of frequency surfaces, solve generalised Fourier transform.
result Derives efficient inference methods for non-stationary kernels.
A new method for nonstationary Gaussian processes using Fourier features.
problem Efficient simulation of nonstationary Gaussian processes with high-dimensional distributions.
method Discretizes the spectral representation of nonstationary processes, avoiding probability measure assumptions.
result An efficient low-rank approximation of nonstationary spectral densities, consistent and positive semi-definite.
The paper presents a method to reduce computational and storage costs in PCA and spectral clustering.
problem Efficiently reducing computational and storage costs in PCA and spectral clustering.
method Randomly 'puncturing' the data matrix and kernel matrix through Bernoulli masks.
result The spectral behavior of the resulting kernel is fully tractable and can be drastically punctured without significant loss in performance.
The paper tackles model collapse in GPLVMs by improving kernel flexibility and projection variance.
problem Model collapse in GPLVMs leading to vague latent representations.
method Theoretical analysis of projection variance, integration of SM and RFF kernels, and variational inference.
result The advisedRFLVM outperforms competing models in informative latent representations and missing data imputation.
Kernel matrix concentration leads to KSC consistency.
problem High-dimensional clustering with noisy data.
method Nonasymptotic concentration inequalities for Lipschitz kernels.
result KSC algorithm consistency for noisy nested manifolds.
Spectral methods have greatly advanced the estimation of latent variable models, generating a sequence of novel and efficient algorithms with strong theoretical guarantees. However, current spectral algorithms are largely restricted to mixtures of discrete or Gaussian distributions. In this paper, we propose a kernel m…
This paper uses Nested Sampling to improve Gaussian Process uncertainty quantification.
problem Underestimating predictive uncertainty and overfitting in Gaussian Process models.
method Marginalises hyperparameters using Nested Sampling for spectral mixture kernels.
result Improves predictive performance and uncertainty quantification across various data sets.
Two adaptive kernel selection methods improve the accuracy of Kernelized Diffusion Maps.
problem Selecting an appropriate kernel for Kernelized Diffusion Maps.
method Two complementary approaches: variational outer loop and unsupervised cross-validation.
result Both methods improve the quality and stability of the recovered eigenfunctions.
DEQs and explicit networks are nearly equivalent for Gaussian mixtures.
problem Understanding the equivalence between DEQs and explicit neural networks.
method Random matrix theory and analysis of kernel matrices.
result A shallow explicit network can mimic the kernel of a DEQ.
Efficient sparse GP model improves audio source separation.
problem Sparse Gaussian Process (GP) inference is computationally expensive for long audio frames.
method Used GP regression, spectral mixture kernels, and variational sparse GPs.
result Proposed method outperforms LD-PSDTF, KL-NMF, and IS-NMF.
Gaussian processes are rich distributions over functions, which provide a Bayesian nonparametric approach to smoothing and interpolation. We introduce simple closed form kernels that can be used with Gaussian processes to discover patterns and enable extrapolation. These kernels are derived by modelling a spectral dens…
Study the geometric structure of graph Laplacian embeddings for manifold data.
problem Identifying coarse structure in manifold data sampled from a mixture model.
method Analyze spectral clustering procedure for data sampled from a manifold, focusing on graph Laplacian embeddings.
result Embedded data concentrates on cones centered around orthogonal vectors when the mixture model is well-separated.
Robust clustering algorithm for datasets with outliers.
problem Clustering with arbitrary outliers.
method Spectral clustering with a rounding scheme on a Gaussian kernel matrix.
result Misclassification error decays exponentially with signal-to-noise ratio.
We prove a conjecture about approximating Gaussian Processes on one dimension.
problem Computational scaling issues with Gaussian Processes on one dimension.
method Developed a new family of state-space models (LEG) to approximate any stationary GP on one dimension.
result Proved that any stationary GP on one dimension can be approximated using the LEG family.
We present in this work a new family of kernels to compare positive measures on arbitrary spaces $\Xcal$ endowed with a positive kernel κ, which translates naturally into kernels between histograms or clouds of points. We first cover the case where $\Xcal$ is Euclidian, and focus on kernels which take into account th…
Deep kernels integrate deep learning and kernel methods for scalable, flexible predictions.
problem Scalable and flexible kernel methods for deep learning.
method Transform inputs with a deep architecture and use local kernel interpolation, inducing points, and algebra for scalable kernel representation. Jointly learn kernel properties through marginal likelihood of a Gaussian process.
result Improved performance over scalable Gaussian processes and standalone deep architectures on large datasets.
New algorithms improve spectral clustering for finite mixture models.
problem Issues with EM algorithm in spectral clustering.
method Spectral decomposition and non-parametric bootstrap sampling.
result Improved convergence and avoidance of poor solutions.
New theorem improves spectral gap for sampling from mixture distributions.
problem Sampling from multimodal distributions with simulated tempering.
method Introduced a decomposition theorem for the restricted spectral gap of simulated tempering.
result Lower bound on the restricted spectral gap for mixture distributions.
New Hida-Matérn kernels enable flexible process priors and efficient GP inference.
problem Flexible modeling of stationary processes with oscillatory components.
method Introducing a new class of covariance functions (Hida-Matérn kernels) and their state space representations.
result Efficient Gaussian Process inference and improved numerical stability.
Criterion extends identifiability for continuous mixtures of kernels.
problem Identify continuous mixtures of kernels.
method Generating-function accessibility criterion based on moment-generating functions or Laplace transforms.
result Criterion applies to mixtures of discrete and continuous variables.
EnEMF uses Epanechnikov kernel for high-dimensional filtering, improving accuracy and robustness.
problem Suboptimal Gaussian mixture kernel density estimates in high-dimensional settings.
method Ensemble Epanechnikov mixture filter (EnEMF) using optimal Epanechnikov kernel.
result EnEMF reduces error per particle on high-dimensional systems like Lorenz '96.
Study proposes a new metric for comparing Gaussian mixtures in RKHS.
problem Comparing complex multimodal densities in RKHS.
method Wasserstein-type metric for kernel Gaussian mixtures.
result Enhanced capability to model multimodal densities.
Essential principal components simplify spectral analysis with minimal training data.
problem Accurate spectral quantification from complex mixtures.
method Identifying essential principal components and using molar extinction coefficients.
result Near one-to-one projection from principal components to mixture constituents.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
Kernel mixture network estimates complex conditional densities.
problem Nonparametric estimation of complex conditional densities.
method Neural network with kernel mixture model.
result Kernel mixture network outperforms existing methods in filtering and generative modeling.
Study provides guarantees for kernel clustering under non-parametric mixtures.
problem Statistical guarantees for kernel-based clustering without strong assumptions.
method Non-parametric mixture models, kernel-based clustering, consistency guarantees.
result Necessary and sufficient separability conditions for consistent clustering recovery.
A new kernel function centers at different points improves robust learning.
problem Robust learning in the presence of outliers.
method Introduces multi-kernel correntropy (MKC) with kernels centered at different points.
result Learning algorithms using MMKCC outperform those using MCC and MMCC.
New method for spectral and Bergman kernels under local spectral gap condition.
problem Analyzing spectral and Bergman kernels for complex manifolds.
method Developed a new scaling method to study spectral and Bergman kernels.
result Established pointwise asymptotics of spectral and Bergman kernels.
New techniques model related samples using kernel mixtures, addressing shared and varying components with misalignments.
problem Modeling related samples with shared and varying components, accounting for misalignments.
method Introduces ψ-stick breaking for mixing weights and kernel perturbation for misalignment. result Efficient Bayesian inference for models incorporating these techniques.
Refined analysis of Mitra's algorithm for discrete mixtures.
problem Classifying general discrete mixture distribution models.
method Spectral clustering tailored to bipartite stochastic block models.
result Improved separation conditions for probability distributions.
The paper explores the identifiability and interpretability of Gaussian process models using different kernel structures.
problem Identifiability and interpretability issues in Gaussian process models.
method The paper examines both single-output and multi-output Gaussian process models using additive and multiplicative mixtures of Matérn kernels.
result The smoothness of a mixture of Matérn kernels is determined by the least smooth component, and none of the mixing weights or parameters are identifiable.