New KQEs improve probability metrics without mean function constraints.
problem Improving probability metrics without relying on mean function representations.
method Kernel quantile embeddings (KQEs) to construct new distances.
result KQEs offer a competitive alternative to MMD with near-linear cost.
A new method estimates multi-dimensional value distributions using Hilbert space embeddings.
problem Estimating value distributions in complex, multi-dimensional reinforcement learning settings.
method Hilbert space mappings and kernel mean embeddings to estimate the kernel mean embedding of multi-dimensional value distributions.
result Uniform convergence guarantees and robust off-policy evaluation demonstrated in simulations.
A faster graph kernel using optical random features.
problem High computation cost of graphlet kernel due to isomorphism test.
method Kernel random features, optical random features, mean kernel metric.
result The proposed method is orders of magnitude faster with similar or better accuracy.
New algorithm detects changes in Markov kernels with unknown post-change kernel.
problem Detecting changes in Markov kernels with unknown post-change kernel.
method Developed a new change detection algorithm assuming uniform ergodicity.
result Derived upper and lower bounds on mean delay and time between false alarms.
New metrics improve probabilistic forecasting, especially for rare events.
problem Current evaluation frameworks for probabilistic forecasting assume independence and lack sensitivity to tail events.
method Proposed signature kernel-based metrics: Sig-MMD and CSig-MMD.
result These metrics capture complex dependencies and prioritize tail event prediction.
A new method for distribution regression using sliced Wasserstein distance.
problem Learning functions over spaces of probabilities.
method Proposes an OT-based estimator using the Sliced Wasserstein distance.
result Proves universal consistency and excess risk bounds for the proposed estimator.
Development of metrics for structural data-generating mechanisms is fundamental in machine learning and the related fields. In this paper, we give a general framework to construct metrics on random nonlinear dynamical systems, defined with the Perron-Frobenius operators in vector-valued reproducing kernel Hilbert space…
Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.
problem Understanding long-term behavior of finite-particle systems in relation to their mean-field limits.
method Developed uniform-in-time propagation-of-chaos results for continuous-time SVGD using cutoff strategies and finite-dimensional theories.
result Uniform-in-time propagation-of-chaos bounds in various metrics, including Langevin kernel Stein discrepancy, Wasserstein-1, and Wasserstein-2 distances.
New conditions ensure MMDs separate and converge to target distributions.
problem Ensuring MMDs separate and converge to target distributions.
method Deriving new sufficient and necessary conditions for MMDs on separable metric spaces.
result First KSDs that exactly metrize weak convergence to P.
Survey of kernels, RKHS, and their applications in machine learning.
problem Understanding kernels and their applications in machine learning.
method Review of historical context, mathematical definitions, and practical applications of kernels.
result Comprehensive overview of kernels, RKHS, and their applications.
A new metric CKCE improves model calibration comparison.
problem Comparing the calibration of probabilistic models is challenging.
method CKCE based on Hilbert-Schmidt norm of conditional mean operators.
result CKCE provides more consistent and robust model calibration comparisons.
A new distance metric compares probability distributions using kernel covariance operators.
problem Comparing probability distributions in machine learning tasks.
method Introduces a novel distance metric based on Schatten norm of kernel covariance operators.
result The new distance metric is more discriminative and robust to hyperparameters.
We lay theoretical foundations for new database release mechanisms that allow third-parties to construct consistent estimators of population statistics, while ensuring that the privacy of each individual contributing to the database is protected. The proposed framework rests on two main ideas. First, releasing (an esti…
Study on kernel tests for high-dimensional data, focusing on MMD and CLT.
problem Asymptotic behavior of kernel two-sample tests in high dimensions and large samples.
method Maximum mean discrepancy (MMD) with isotropic kernels, deriving asymptotic expansions and CLT.
result Interplay between moment discrepancy and dimension-and-sample orders in kernel tests.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
A new metric compares true and learned causal graphs considering data and graph structure.
problem Comparing true and learned causal graphs accurately.
method Continuous Structural Intervention Distance (CSID) using conditional mean embeddings and maximum mean discrepancy.
result Validated the CSID with synthetic data, showing its effectiveness in comparing causal graphs.
Kernel mean embeddings have recently attracted the attention of the machine learning community. They map measures μ from some set M to functions in a reproducing kernel Hilbert space (RKHS) with kernel k. The RKHS distance of two mapped measures is a semi-metric dk over M. We study three questions. (I) For a…
Paper develops a unified framework for measuring differences between conditional distributions.
problem Comparing conditional distributions in a unified and theoretically sound manner.
method Kernel embeddings and conditional maximum mean discrepancy (CMMD) framework.
result Established a coherent framework for measuring divergence between conditional distributions.
Mean embeddings provide an extremely flexible and powerful tool in machine learning and statistics to represent probability distributions and define a semi-metric (MMD, maximum mean discrepancy; also called N-distance or energy distance), with numerous successful applications. The representation is constructed as the e…
Proposes DP-MERF for privacy-preserving synthetic data generation.
problem Privacy-preserving data generation for synthetic datasets.
method Differentially private mean embeddings with random features.
result Achieves better privacy-utility trade-offs than existing methods.
Adapts manifold structure for better clustering performance.
problem Lack of consideration for local manifold structure in existing multiple kernel k-means methods.
method Adopts manifold adaptive kernel to integrate local manifold structure of kernels.
result Proposed method outperforms state-of-the-art methods.
Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the …
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
Paper improves clustering risk bounds for kernel k-means.
problem Improving clustering risk bounds for kernel k-means.
method Analyzes kernel k-means and Nyström approximation.
result Achieves nearly optimal excess clustering risk bound.
Clustering samples according to an effective metric and/or vector space representation is a challenging unsupervised learning task with a wide spectrum of applications. Among several clustering algorithms, k-means and its kernelized version have still a wide audience because of their conceptual simplicity and efficacy.…
The sequence of moments of a vector-valued random variable can characterize its law. We study the analogous problem for path-valued random variables, that is stochastic processes, by using so-called robust signature moments. This allows us to derive a metric of maximum mean discrepancy type for laws of stochastic proce…
Kernel methods are studied in a mean field limit for high-dimensional data.
problem Analyzing kernel methods in high-dimensional data with many variables.
method Investigation of kernel methods in the mean field limit of interacting particle systems.
result Rigorous mean field limit of kernels and detailed analysis of the limiting reproducing kernel Hilbert space.
This paper provides a dictionary of closed-form kernel mean embeddings.
problem Challenges in deriving closed-form kernel mean embeddings.
method Comprehensive dictionary and practical tools for deriving new embeddings.
result Provides a Python library with minimal implementations of embeddings.
A new method compresses conditional distributions of labelled data.
problem No existing method directly compresses the conditional distribution of labelled data.
method Introduce Average Maximum Conditional Mean Discrepancy (AMCMD), derive a closed form estimator, and extend Kernel Herding (KH) to Average Conditional Kernel Herding (ACKH).
result Directly compressing conditional distributions outperforms joint distribution compression and greedy selection.
A new graph kernel uses LCS and Wasserstein distance for better graph comparisons.
problem Graph learning methods can be limited by information from distant vertices and path length constraints.
method Proposes a Graph Kernel based on LCS similarity and Wasserstein distance in a novel metric space.
result The new kernel emphasizes comparisons between similar paths and reduces information loss.
A Hilbert space embedding for probability measures has recently been proposed, with applications including dimensionality reduction, homogeneity testing, and independence testing. This embedding represents any probability measure as a mean element in a reproducing kernel Hilbert space (RKHS). A pseudometric on the spac…
We apply a fast kernel method for mask-based single-channel speech enhancement. Specifically, our method solves a kernel regression problem associated to a non-smooth kernel function (exponential power kernel) with a highly efficient iterative method (EigenPro). Due to the simplicity of this method, its hyper-parameter…
Large scale agglomerative clustering is hindered by computational burdens. We propose a novel scheme where exact inter-instance distance calculation is replaced by the Hamming distance between Kernelized Locality-Sensitive Hashing (KLSH) hashed values. This results in a method that drastically decreases computation tim…
A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…
LGKDE learns graph density using neural networks and perturbations.
problem Graph density estimation challenges in capturing structural patterns and semantic variations.
method LGKDE uses graph neural networks to represent graphs as discrete distributions and learns graph metrics via maximum mean discrepancy.
result LGKDE outperforms state-of-the-art baselines in graph anomaly detection.
Distance-based tests, also called "energy statistics", are leading methods for two-sample and independence tests from the statistics community. Kernel-based tests, developed from "kernel mean embeddings", are leading methods for two-sample and independence tests from the machine learning community. A fixed-point transf…
Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inferenc…
Kernel methods are one of the mainstays of machine learning, but the problem of kernel learning remains challenging, with only a few heuristics and very little theory. This is of particular importance in methods based on estimation of kernel mean embeddings of probability measures. For characteristic kernels, which inc…
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in…
To cluster data that are not linearly separable in the original feature space, k-means clustering was extended to the kernel version. However, the performance of kernel k-means clustering largely depends on the choice of kernel function. To mitigate this problem, multiple kernel learning has been introduced into th…
Kernel K-means clusters probability distributions.
problem Clustering a sample of probability distributions.
method Mapping distributions to kernel mean embeddings in RKHS, then applying K-means.
result Effective unsupervised classification of probability distributions.
Kernel k-Means algorithm improves clustering of non-linear data.
problem Non-convexity of kernel k-Means objective function leads to local minima.
method Generalizes MM approach to solve non-convex problem in kernel and multi-kernel settings.
result Establishes strong consistency guarantees for Kernel Power k-Means.
Study on kernel methods in large-scale machine learning problems.
problem Large-scale machine learning with many interacting variables.
method Mean field limit analysis of kernels and their Hilbert spaces.
result Mean field convergence of empirical and infinite-sample solutions.
Proposes CCE to assess point-wise reliability of neural network predictions.
problem Overconfidence and misaligned predictive distributions in neural networks.
method Introduces Conditional Congruence (CCE) metric using conditional kernel mean embeddings.
result CCE exhibits correctness, monotonicity, reliability, and robustness in high-dimensional regression tasks.
Kernelized Taylor diagram visualizes data populations with fewer assumptions.
problem Limitations of Taylor diagram in capturing non-linear relationships and sensitivity to outliers.
method Proposes a kernelized version of the Taylor diagram that uses maximum mean discrepancy and kernel mean embedding.
result Kernelized Taylor diagram visualizes data populations with minimal assumptions of data distributions.
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We …
New recursive algorithm estimates conditional kernel mean embeddings in Hilbert space.
problem Estimating conditional distributions in RKHS for supervised learning.
method Recursive algorithm in L2 space for conditional kernel mean map. result Strong L2 consistency of recursive estimator proved. Study guarantees convergence of mean shift mode estimation.
problem Ensuring reliable mode estimation in KDE using mean shift.
method Utilizes Łojasiewicz inequality to prove convergence rate.
result Extends convergence guarantees to biweight kernel.