This paper provides a dictionary of closed-form kernel mean embeddings.
problem Challenges in deriving closed-form kernel mean embeddings.
method Comprehensive dictionary and practical tools for deriving new embeddings.
result Provides a Python library with minimal implementations of embeddings.
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
Bayesian deconditional embeddings solve complex function recovery.
problem Recovering original functions from conditional mean observations.
method Formalizes deconditional kernel mean embeddings as Bayesian inference, connects to task-transformed Gaussian processes.
result Establishes deconditional kernel means as posterior predictive mean, providing Bayesian interpretations and uncertainty.
Kernel methods are one of the mainstays of machine learning, but the problem of kernel learning remains challenging, with only a few heuristics and very little theory. This is of particular importance in methods based on estimation of kernel mean embeddings of probability measures. For characteristic kernels, which inc…
We offer a new, rigorous approach to conditional mean embeddings without operator constraints.
problem Lack of rigorous, operator-free approach to conditional mean embeddings.
method Measure-theoretic approach to conditional mean embeddings.
result Natural regression interpretation and universal consistency of empirical estimates.
Kernel mean estimation for functions of random variables provides consistent estimators.
problem Estimating functions of random variables using kernel mean embeddings.
method Kernel mean embeddings for continuous functions of random variables.
result Consistent estimators of mean embeddings of functions of random variables.
New KQEs improve probability metrics without mean function constraints.
problem Improving probability metrics without relying on mean function representations.
method Kernel quantile embeddings (KQEs) to construct new distances.
result KQEs offer a competitive alternative to MMD with near-linear cost.
A new method learns hyperparameters for conditional kernel mean embeddings using Rademacher complexity bounds.
problem Hyperparameter tuning for conditional kernel mean embeddings is challenging and computationally expensive.
method Proposes a hyperparameter learning framework based on Rademacher complexity bounds for scalable kernel hyperparameter tuning.
result Demonstrates improved performance over competing methods and can incorporate deep neural network weights.
Efficiently approximates kernel mean embeddings using Nyström method.
problem Computational cost of kernel mean embeddings in large-scale settings.
method Nyström method for approximating a small random subset of the dataset.
result Upper bound on approximation error with sufficient subsample size conditions.
New recursive algorithm estimates conditional kernel mean embeddings in Hilbert space.
problem Estimating conditional distributions in RKHS for supervised learning.
method Recursive algorithm in L2 space for conditional kernel mean map. result Strong L2 consistency of recursive estimator proved. Novel approach to OT using kernel mean embeddings controls overfitting and achieves dimension-free sample complexity.
problem Consistently estimate optimal transport plan from samples.
method Pose OT as learning kernel mean embedding, employ MMD regularization.
result ε-optimal recovery of transport plan and map with dimension-free sample complexity.
A new method estimates multi-dimensional value distributions using Hilbert space embeddings.
problem Estimating value distributions in complex, multi-dimensional reinforcement learning settings.
method Hilbert space mappings and kernel mean embeddings to estimate the kernel mean embedding of multi-dimensional value distributions.
result Uniform convergence guarantees and robust off-policy evaluation demonstrated in simulations.
Faster convergence of kernel mean embeddings using variance information.
problem Speeding up the convergence rate of kernel mean embeddings.
method Leveraging variance information in reproducing kernel Hilbert space and estimating variance from data.
result Efficiently estimate variance information from data to achieve distribution-agnostic convergence bounds.
Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the …
Develops a rigorous theory for conditional mean embeddings.
problem Efficient conditioning of probability distributions in RKHSs.
method Mathematical theory for both centred and uncentred covariance operators.
result Significantly weakens conditions for applicability of CMEs.
MONK improves mean embedding estimation by reducing outlier sensitivity.
problem Outliers severely affect classical mean embedding estimators.
method Uses median-of-means principle to design robust estimators.
result Optimal sub-Gaussian deviation bounds with resistance to outliers.
A Hilbert space embedding for probability measures has recently been proposed, wherein any probability measure is represented as a mean element in a reproducing kernel Hilbert space (RKHS). Such an embedding has found applications in homogeneity testing, independence testing, dimensionality reduction, etc., with the re…
IDK improves anomaly detection for points and groups without explicit learning.
problem Anomaly detection for points and groups using kernel methods.
method Isolation Distributional Kernel (IDK) addresses data independence and intractable dimensionality issues.
result IDK outperforms existing methods for both point and group anomaly detection.
Paper generalizes kernel mean embedding to von Neumann-algebra-valued measures.
problem Analyzing complex multivariate distributions and quantum mechanics.
method Generalizes kernel mean embedding to von Neumann-algebra-valued measures in reproducing kernel Hilbert modules.
result Injectivity and universality of the generalized KME are confirmed.
Paper introduces RKHM and KME for richer data analysis.
problem Lack of rich data structures in kernel methods.
method Proposes RKHM and KME for functional data analysis.
result RKHM captures structural properties in functional data.
Paper develops a unified framework for measuring differences between conditional distributions.
problem Comparing conditional distributions in a unified and theoretically sound manner.
method Kernel embeddings and conditional maximum mean discrepancy (CMMD) framework.
result Established a coherent framework for measuring divergence between conditional distributions.
Kernelized Taylor diagram visualizes data populations with fewer assumptions.
problem Limitations of Taylor diagram in capturing non-linear relationships and sensitivity to outliers.
method Proposes a kernelized version of the Taylor diagram that uses maximum mean discrepancy and kernel mean embedding.
result Kernelized Taylor diagram visualizes data populations with minimal assumptions of data distributions.
New method protects privacy while allowing accurate statistical inference from synthetic data.
problem Ensuring privacy in database release while maintaining statistical utility.
method Kernel mean embedding with differential privacy constraints.
result Consistent estimators of population statistics can be constructed while protecting individual privacy.
Hermite polynomials improve private data generation by reducing feature count.
problem Infinite-dimensional features in kernel mean embedding are impractical for private data generation.
method Replace random features with Hermite polynomial features, leveraging their ordered nature.
result Hermite polynomial features yield a more accurate approximation of kernel mean embedding with fewer features.
Paper improves MMD estimation for analytical mean embeddings.
problem Improving MMD estimation for distributions with analytical mean embeddings.
method Proposes a tighter concentration result for MMD estimation under semi-explicit settings and extends to unbounded kernels.
result Demonstrates efficiency in real-world applications like index replication and calibration.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
Proposes DP-MERF for privacy-preserving synthetic data generation.
problem Privacy-preserving data generation for synthetic datasets.
method Differentially private mean embeddings with random features.
result Achieves better privacy-utility trade-offs than existing methods.
KELFI improves inference accuracy in likelihood-free settings with limited simulations.
problem Intractable likelihood evaluations in likelihood-free inference.
method Kernel embedding likelihood-free inference (KELFI) learns model hyperparameters to balance accuracy and efficiency.
result Improved accuracy and efficiency on challenging inference problems in ecology.
New method uses Fokker-Planck equation for sampling and inference.
problem Intractability of evaluating probability density in practical applications.
method Reformulates Fokker-Planck equation as a particle flow method, using velocity field.
result Turns intractable density evaluation into an advantage for variational inference, kernel mean embeddings, and sequential Monte Carlo.
Meta-learning for estimating complex conditional distributions.
problem Estimating conditional densities in multimodal distributions.
method Noise contrastive estimation with kernel mean embeddings.
result Meta-learning can share representations across tasks for conditional density estimation.
The paper introduces new KMEs to capture stochastic process filtrations.
problem Missing filtration information in stochastic processes.
method Higher order kernel mean embeddings (KMEs) conditioned on filtrations.
result Consistent estimators and tests for filtration-sensitive information.
CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.
problem Estimating counterfactual policy outcomes for decision-making.
method Counterfactual Policy Mean Embedding (CPME) framework in RKHS, plug-in and doubly robust estimators, kernel test statistic.
result Doubly robust estimator improves convergence rates and asymptotic normality.
Neural-Kernel CME tackles scalability and expressiveness challenges in conditional distribution representation.
problem Scalability and expressiveness challenges in kernel conditional mean embeddings.
method Combines deep learning with CMEs using a neural network optimization framework.
result Achieves competitive and often superior performance in conditional density estimation and RL.
A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be ex…
A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…
Improved multi-task averaging reduces mean squared error in high-dimensional data.
problem Joint estimation of multiple distributions using independent data sets.
method Exploits similarities between tasks by shrinking naive estimators towards local averages.
result The method provides a significant reduction in mean squared error, especially in high-dimensional spaces.
New tools for uncertainty in dynamical systems without distribution assumptions.
problem Uncertainty representation in dynamical systems without distributional assumptions.
method Kernel mean embedding and kernel probabilistic programming.
result Distribution-free representation, comparison, and propagation of uncertainties.
A new method for distribution regression using sliced Wasserstein distance.
problem Learning functions over spaces of probabilities.
method Proposes an OT-based estimator using the Sliced Wasserstein distance.
result Proves universal consistency and excess risk bounds for the proposed estimator.
This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in…
Study optimizes learning rates for conditional mean embedding estimates.
problem Consistency of kernel ridge regression for conditional mean embedding.
method Adaptive statistical learning rate derived for misspecified setting.
result Upper bound matches optimal O(logn/n) rates without assuming finite dimensionality. Proposes a new method for MIR using kernel mean embeddings.
problem Multiple instance regression (MIR) where bags contain multiple instances with a single label.
method Computes kernel mean embeddings of predicted label distributions and learns a regressor from these embeddings.
result Better results than baseline instance-MIR across all datasets, state-of-the-art on two.
Paper proposes a novel method to test differences in spatial point patterns.
problem Detecting differences in the first-order structures of spatial point patterns.
method Kernel mean embedding with approximate version tailored for spatial point processes, reducing comparison to Euclidean space t-tests.
result The proposed method is powerful and well-calibrated, demonstrated on real-world data.
DKMD is a fast signed statistic for comparing univariate distributions.
problem Comparing univariate distributions, especially preserving directionality.
method DKMD integrates kernel mean embeddings against an odd weighting function.
result DKMD preserves directionality and is robust to outliers.
Kernel K-means clusters probability distributions.
problem Clustering a sample of probability distributions.
method Mapping distributions to kernel mean embeddings in RKHS, then applying K-means.
result Effective unsupervised classification of probability distributions.
Method calibrates simulators under covariate shift using kernel techniques.
problem Dealing with covariate shift in simulator inputs.
method Bayesian inference with kernel mean embedding and importance-weighted reproducing kernel.
result The method effectively calibrates simulators and demonstrates sensitivity analysis.
Kernel embeddings separate distinct probability distributions, simplifying testing.
problem Testing equality of non-atomic probability distributions.
method Kernel covariance embeddings and Gaussian measures in reproducing kernel Hilbert spaces.
result Testing for singularity between Gaussian measures is equivalent to testing for equality of non-atomic probability distributions.
We propose a new method to model multi-way similarities into hypergraphs for clustering.
problem Clustering real-valued data using hypergraphs with multi-way similarities.
method Formulate multi-way similarities using kernel functions, establish connections to hypergraph cut, and develop a fast spectral clustering algorithm.
result Our method outperforms existing graph and heuristic modeling methods in clustering performance.