arXiv research

New estimator reduces kernel mean estimation error.

A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…

A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in…

Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the …

Study guarantees convergence of mean shift mode estimation.

New recursive algorithm estimates conditional kernel mean embeddings in Hilbert space.

Faster convergence of kernel mean embeddings using variance information.

The problem of estimating the kernel mean in a reproducing kernel Hilbert space (RKHS) is central to kernel methods in that it is used by classical approaches (e.g., when centering a kernel PCA matrix), and it also forms the core inference step of modern kernel methods (e.g., kernel-based non-parametric tests) that rel…

Paper introduces MoM-KDE for robust density estimation robust to anomalous data.

We propose a method for nonparametric density estimation that exhibits robustness to contamination of the training sample. This method achieves robustness by combining a traditional kernel density estimator (KDE) with ideas from classical $M$-estimation. We interpret the KDE based on a radial, positive semi-definite ke…

The paper proves existence and growth estimates for inverse mean curvature flow and related $p$-Laplacian Green kernel decay.

Improved estimation of higher order integrals using shrinkage techniques.

The paper studies kernel smoothing and mean shift for directional data, deriving convergence rates and mode estimation.

TAKDE optimizes kernel density estimation for real-time dynamic processes.

Kernel methods are one of the mainstays of machine learning, but the problem of kernel learning remains challenging, with only a few heuristics and very little theory. This is of particular importance in methods based on estimation of kernel mean embeddings of probability measures. For characteristic kernels, which inc…

We provide a theoretical foundation for non-parametric estimation of functions of random variables using kernel mean embeddings. We show that for any continuous function $f$, consistent estimators of the mean embedding of a random variable $X$ lead to consistent estimators of the mean embedding of $f(X)$. For Matérn ke…

Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inferenc…

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample s…

New tests for distributional causal effects using improved kernel estimators.

Improved multi-task averaging reduces mean squared error in high-dimensional data.

The article introduces practical estimators for kernel discrepancies.

A new method estimates multi-dimensional value distributions using Hilbert space embeddings.

Paper shows DMS as an EM algorithm with improved convergence.

A new method for distribution regression using sliced Wasserstein distance.

We lay theoretical foundations for new database release mechanisms that allow third-parties to construct consistent estimators of population statistics, while ensuring that the privacy of each individual contributing to the database is protected. The proposed framework rests on two main ideas. First, releasing (an esti…

In recent years, kernel density estimation has been exploited by computer scientists to model machine learning problems. The kernel density estimation based approaches are of interest due to the low time complexity of either O(n) or O(n*log(n)) for constructing a classifier, where n is the number of sampling instances.…

The objective in statistical Optimal Transport (OT) is to consistently estimate the optimal transport plan/map solely using samples from the given source and target marginal distributions. This work takes the novel approach of posing statistical OT as that of learning the transport plan's kernel mean embedding from sam…

The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been deve…

Proves existence and uniqueness of mean curvature flow.

Kernel $ε$-Greedy optimizes multi-armed bandits with covariates for sub-linear regret.

LGKDE learns graph density using neural networks and perturbations.

New KQEs improve probability metrics without mean function constraints.

Paper improves MMD estimation for analytical mean embeddings.

CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.

DKMD is a fast signed statistic for comparing univariate distributions.

Proposes CCME framework for estimating heterogeneous treatment effects.

Estimates multiple means in high dimensions using convex combinations.

Algorithm learns interaction kernels for particle systems from data.

Estimates mean and covariance for large, unbalanced stock returns panels.

This thesis improves kernel-based distances for statistical inference and integration.

Mean embeddings provide an extremely flexible and powerful tool in machine learning and statistics to represent probability distributions and define a semi-metric (MMD, maximum mean discrepancy; also called N-distance or energy distance), with numerous successful applications. The representation is constructed as the e…

Kernelized cumulants improve statistical analysis in high-dimensional spaces.

New method combines strengths of two PCL approaches without density ratio estimation.

In real supervised learning scenarios, it is not uncommon that the training and test sample follow different probability distributions, thus rendering the necessity to correct the sampling bias. Focusing on a particular covariate shift problem, we derive high probability confidence bounds for the kernel mean matching (…

Develops an MS-inspired algorithm for regression mode finding and space partitioning.

Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.

Paper develops a unified framework for measuring differences between conditional distributions.

We present an operator-free, measure-theoretic approach to the conditional mean embedding (CME) as a random variable taking values in a reproducing kernel Hilbert space. While the kernel mean embedding of unconditional distributions has been defined rigorously, the existing operator-based approach of the conditional ve…