Study proposes a new metric for comparing Gaussian mixtures in RKHS.
problem Comparing complex multimodal densities in RKHS.
method Wasserstein-type metric for kernel Gaussian mixtures.
result Enhanced capability to model multimodal densities.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Kernel Gaussian Mixtures”
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
EnEMF uses Epanechnikov kernel for high-dimensional filtering, improving accuracy and robustness.
problem Suboptimal Gaussian mixture kernel density estimates in high-dimensional settings.
method Ensemble Epanechnikov mixture filter (EnEMF) using optimal Epanechnikov kernel.
result EnEMF reduces error per particle on high-dimensional systems like Lorenz '96.
As a novel similarity measure that is defined as the expectation of a kernel function between two random variables, correntropy has been successfully applied in robust machine learning and signal processing to combat large outliers. The kernel function in correntropy is usually a zero-mean Gaussian kernel. In a recent …
New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
Standard kernels such as Matérn or RBF kernels only encode simple monotonic dependencies within the input space. Spectral mixture kernels have been proposed as general-purpose, flexible kernels for learning and discovering more complicated patterns in the data. Spectral mixture kernels have recently been generalized in…
The paper explores the identifiability and interpretability of Gaussian process models using different kernel structures.
problem Identifiability and interpretability issues in Gaussian process models.
method The paper examines both single-output and multi-output Gaussian process models using additive and multiplicative mixtures of Matérn kernels.
result The smoothness of a mixture of Matérn kernels is determined by the least smooth component, and none of the mixing weights or parameters are identifiable.
Develops nonstationary MOGP kernels for better performance.
problem Limited applicability of existing MOGP kernels for nonstationary data.
method Harmonizable spectral mixture kernels for nonstationary MOGP.
result Automatic identification of nonstationary behavior in data.
Multi-output Gaussian processes (MOGPs) are an extension of Gaussian Processes (GPs) for predicting multiple output variables (also called channels, tasks) simultaneously. In this paper we use the convolution theorem to design a new kernel for MOGPs, by modeling cross channel dependencies through cross convolution of t…
A spectral mixture representation of isotropic kernels with application to random Fourier featurescs.LG
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatmen…
Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.
problem Inefficient marginalization over Gaussian Process kernels for large datasets.
method Bayesian Quadrature scheme with maximum mean discrepancies and invariances between Spectral Mixture kernels.
result Achieves more accurate predictions and better calibrated uncertainty than state-of-the-art baselines.
The paper proposes a Gaussian mixture model for Hilbert-space-valued data.
problem Challenges in characterizing probability measures for infinite-dimensional random objects.
method Gaussian mixture framework based on kernel mean embeddings.
result The proposed algorithm yields a dense class of approximations in infinite-dimensional spaces.
Neural networks outperform kernel methods in classifying high-dimensional Gaussian mixtures.
problem Classifying high-dimensional Gaussian mixtures using kernel methods and neural networks.
method Theoretical analysis and derivation of learning dynamics for 2LNN and comparison with kernel methods.
result 2LNN can achieve near-optimal performance on high-dimensional Gaussian mixture classification tasks, surpassing kernel methods.
Improved Gaussian process experts model for complex data.
problem Limitations of standard Gaussian processes: scalability and predictive performance.
method Proposes a new mixture model of Gaussian process experts based on kernel stick-breaking processes.
result Improved predictive performance compared to existing models.
DEQs and explicit networks are nearly equivalent for Gaussian mixtures.
problem Understanding the equivalence between DEQs and explicit neural networks.
method Random matrix theory and analysis of kernel matrices.
result A shallow explicit network can mimic the kernel of a DEQ.
Study provides guarantees for kernel clustering under non-parametric mixtures.
problem Statistical guarantees for kernel-based clustering without strong assumptions.
method Non-parametric mixture models, kernel-based clustering, consistency guarantees.
result Necessary and sufficient separability conditions for consistent clustering recovery.
Kernel Topic Modelscs.LG
Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with e…
New distances measure mixtures of Gaussians, useful in machine learning.
problem Comparing distributions with disjoint supports.
method Schoenberg-Rao distances based on concave Rao's entropy.
result Closed-form distances for mixtures of Gaussians.
Gaussian processes are rich distributions over functions, which provide a Bayesian nonparametric approach to smoothing and interpolation. We introduce simple closed form kernels that can be used with Gaussian processes to discover patterns and enable extrapolation. These kernels are derived by modelling a spectral dens…
Kernel functions in support vector machines (SVM) are needed to assess the similarity of input samples in order to classify these samples, for instance. Besides standard kernels such as Gaussian (i.e., radial basis function, RBF) or polynomial kernels, there are also specific kernels tailored to consider structure in t…
We introduce a balloon estimator in a generalized expectation-maximization method for estimating all parameters of a Gaussian mixture model given one data sample per mixture component. Instead of limiting explicitly the model size, this regularization strategy yields low-complexity sparse models where the number of eff…
Paper proposes new costs for learning multiple centers in MDNs.
problem Learning multiple centers for density approximation in MDNs.
method Combines MDNs with contrastive costs using four types of kernelized matrix costs.
result New costs improve data density approximation in MDNs.
This paper introduces the kernel mixture network, a new method for nonparametric estimation of conditional probability densities using neural networks. We model arbitrarily complex conditional densities as linear combinations of a family of kernel functions centered at a subset of training points. The weights are deter…
Posterior regularization enhances Bayesian hierarchical mixture clustering by improving node separation.
problem High nodal variance in BHMC trees, leading to weak separation between nodes at higher levels.
method Employing Posterior Regularization to impose max-margin constraints on nodes at every level.
result Improves cluster separation in BHMC models, enhancing overall model performance.
Spectral mixture (SM) kernels comprise a powerful class of generalized kernels for Gaussian processes (GPs) to describe complex patterns. This paper introduces model compression and time- and phase (TP) modulated dependency structures to the original (SM) kernel for improved generalization of GPs. Specifically, by adop…
The paper tackles model collapse in GPLVMs by improving kernel flexibility and projection variance.
problem Model collapse in GPLVMs leading to vague latent representations.
method Theoretical analysis of projection variance, integration of SM and RFF kernels, and variational inference.
result The advisedRFLVM outperforms competing models in informative latent representations and missing data imputation.
FastKCI speeds up KCI tests for causal inference on large datasets.
problem Cubic computational complexity of kernel-based conditional independence tests.
method Mixture-of-experts approach with parallel Gaussian process inference.
result Substantial computational speedups with maintained statistical power.
We propose non-stationary spectral kernels for Gaussian process regression. We propose to model the spectral density of a non-stationary kernel function as a mixture of input-dependent Gaussian process frequency density surfaces. We solve the generalised Fourier transform with such a model, and present a family of non-…
This paper uses Nested Sampling to improve Gaussian Process uncertainty quantification.
problem Underestimating predictive uncertainty and overfitting in Gaussian Process models.
method Marginalises hyperparameters using Nested Sampling for spectral mixture kernels.
result Improves predictive performance and uncertainty quantification across various data sets.
Continuous-time interpolation of volatility surfaces preserving mixtures and arbitrage-free.
problem Interpolation of volatility surfaces
method Constructing a mixture-preserving, arbitrage-free interpolation
result Lifts Brigo-Mercurio to time-varying weights with additive cost
Enhances GPLVM for multi-view data with scalable latent representation learning.
problem Limited kernel expressiveness and computational inefficiency in multi-view GPLVM.
method Introduces a new duality between spectral density and kernel function, uses NG-SM kernel, and applies random Fourier feature approximation for scalability.
result Consistently outperforms state-of-the-art models in learning meaningful latent representations across diverse datasets.
A new method for density estimation using nearest neighbor Dirichlet mixtures.
problem Slow and unstable Bayesian density estimation methods.
method Nearest neighbor grouping, local Bayesian parametric models, Dirichlet prior, Monte Carlo sampling.
result Effective density estimation with improved computational efficiency.
Early approaches to multiple-output Gaussian processes (MOGPs) relied on linear combinations of independent, latent, single-output Gaussian processes (GPs). This resulted in cross-covariance functions with limited parametric interpretation, thus conflicting with the ability of single-output GPs to understand lengthscal…
Deep Kernel Learningcs.LG
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods. Specifically, we transform the inputs of a spectral mixture base kernel with a deep architecture, using local kernel interpolation, inducing points, and struc…
Generative models use kernel smoothing for conditioning on small example sets.
problem Improving generative models' performance with limited conditioning examples.
method Showed that cross-attention conditioning is equivalent to kernel smoothing, specifically a Nadaraya--Watson kernel smoother.
result The approach predicts and confirms three failure regimes for kernel-based conditioning.
We analyze training dynamics in Gaussian mixture models using a comparison theorem.
problem Analyzing training algorithms with Gaussian mixture data.
method Applying a Gaussian comparison theorem to a specific family of training algorithms.
result Validated dynamic mean-field expressions and provided iterative refinement schemes.
We investigate a Gaussian mixture model (GMM) with component means constrained in a pre-selected subspace. Applications to classification and clustering are explored. An EM-type estimation algorithm is derived. We prove that the subspace containing the component means of a GMM with a common covariance matrix also conta…
We prove a conjecture about approximating Gaussian Processes on one dimension.
problem Computational scaling issues with Gaussian Processes on one dimension.
method Developed a new family of state-space models (LEG) to approximate any stationary GP on one dimension.
result Proved that any stationary GP on one dimension can be approximated using the LEG family.
In this paper, we propose an outlier-robust regularized kernel-based method for linear system identification. The unknown impulse response is modeled as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential …
Batch Normalization (BN) is essential to effectively train state-of-the-art deep Convolutional Neural Networks (CNN). It normalizes inputs to the layers during training using the statistics of each mini-batch. In this work, we study BN from the viewpoint of Fisher kernels. We show that assuming samples within a mini-ba…
This study analyzes theoretical guarantees for VI with fixed-variance Gaussian mixtures.
problem Theoretical analysis of variational inference with non-Gaussian distributions.
method Formulates variational inference as minimizing a mollified relative entropy, solving it through gradient descent on particle positions.
result Establishes descent lemma and approximation error bounds for optimization of variational inference.
We present a scalable Gaussian process model for identifying and characterizing smooth multidimensional changepoints, and automatically learning changes in expressive covariance structure. We use Random Kitchen Sink features to flexibly define a change surface in combination with expressive spectral mixture kernels to …
Many machine learning problems can be framed in the context of estimating functions, and often these are time-dependent functions that are estimated in real-time as observations arrive. Gaussian processes (GPs) are an attractive choice for modeling real-valued nonlinear functions due to their flexibility and uncertaint…
A new HMM model captures kernel dependencies using context-specific Bayesian networks.
problem Traditional HMMs struggle with non-Gaussian data and independence assumptions.
method Kernel density estimation with context-specific Bayesian networks.
result The proposed model outperforms related HMMs in likelihood and classification accuracy.
Efficient GP framework for scalable non-stationary processes.
problem Heavy memory and computational requirements in Gaussian process regression for large data sets.
method Exploits structure in the kernel matrix, uses multiple sets of non-equidistant inducing points, and employs Toeplitz and Kronecker structure for efficient inference.
result Demonstrated scalability on numerical examples and large biomedical datasets.
Algorithm finds best Dirac mass approximation of target measure.
problem Finding optimal Dirac mass approximation of target measure.
method Minimizes statistical distance between original measure and quantized version using Huber-energy kernel.
result HEMQ algorithm robust and versatile, matches intuitive behavior.
Hida-Matérn Kernelstat.ML
New Hida-Matérn kernels enable flexible process priors and efficient GP inference.
problem Flexible modeling of stationary processes with oscillatory components.
method Introducing a new class of covariance functions (Hida-Matérn kernels) and their state space representations.
result Efficient Gaussian Process inference and improved numerical stability.