Paper proposes a new method to learn distribution kernels via entropy maximization.
problem Challenges in applying kernel methods to distribution regression tasks.
method Proposes a novel objective for unsupervised learning of data-dependent distribution kernels based on entropy maximization.
result Demonstrates the effectiveness of the learned kernel across different modalities.
One-shot algorithm for feature-distributed kernel PCA reduces communication costs.
problem Efficiently perform kernel PCA in distributed computing environments.
method Inspired by dual relationship between sample-distributed and feature-distributed scenarios, proposes a one-shot algorithm for feature-distributed kernel PCA.
result The algorithm provides high-quality results with low communication costs, especially when eigenvalues decay fast.
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
Kernel K-means clusters probability distributions.
problem Clustering a sample of probability distributions.
method Mapping distributions to kernel mean embeddings in RKHS, then applying K-means.
result Effective unsupervised classification of probability distributions.
Paper studies distributed kernel regression with imperfect kernels, achieving optimal rates.
problem Optimal rates of distributed regression with imperfect kernels.
method Divide and conquer approach, response weighted base algorithms, leave one out analysis, bias correction.
result Achieves capacity independent optimal rates for distributed kernel regression with imperfect kernels.
IDK improves anomaly detection for points and groups without explicit learning.
problem Anomaly detection for points and groups using kernel methods.
method Isolation Distributional Kernel (IDK) addresses data independence and intractable dimensionality issues.
result IDK outperforms existing methods for both point and group anomaly detection.
Paper studies t-SNE convergence with generalized kernels.
problem Understanding convergence of t-SNE with generalized kernels.
method Concrete formulation of generalized kernels, proving convergence to an equilibrium distribution.
result t-SNE converges to an equilibrium distribution under certain conditions for generalized kernels.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
Paper develops a new method for distribution regression with indefinite kernels.
problem Distribution regression with indefinite kernels.
method Coefficient-based regularized distribution regression with two-stage sampling.
result Optimal learning rates derived for the algorithm under mild conditions.
We connect shift-invariant characteristic kernels to infinitely divisible distributions on Rd. Characteristic kernels play an important role in machine learning applications with their kernel means to distinguish any two probability measures. The contribution of this paper is two-fold. First, we show, usi…
Improved learning theory for kernel distribution regression with two-stage sampling.
problem Distribution regression problem and two-stage sampling setting.
method Kernel methods, near-unbiased condition, new error bounds, convergence rates.
result Strictly improved convergence rates for three important classes of kernels.
Study on how kernel regression models generalize to out-of-distribution data.
problem Understanding generalization in machine learning models under distributional shifts.
method Replica method from statistical physics to derive analytical formula for generalization error.
result Identified overlap matrix as key determinant of generalization performance under distribution shift.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
A new method compresses conditional distributions of labelled data.
problem No existing method directly compresses the conditional distribution of labelled data.
method Introduce Average Maximum Conditional Mean Discrepancy (AMCMD), derive a closed form estimator, and extend Kernel Herding (KH) to Average Conditional Kernel Herding (ACKH).
result Directly compressing conditional distributions outperforms joint distribution compression and greedy selection.
Improves probability distribution compression with KT algorithm.
problem Efficiently compressing probability distributions.
method Kernel thinning (KT) algorithm with four improvements.
result KT yields tighter, dimension-free guarantees for any kernel.
A new distributed clustering framework using distributional kernel.
problem Clustering in distributed networks with arbitrary shapes, sizes, and densities.
method Distributed Clustering based on Distributional Kernel (KDC) using similarity of distributions.
result KDC guarantees equivalent clustering outcomes to centralized methods, reduces runtime, and discovers arbitrary clusters.
Paper analyzes distributed learning with non-i.i.d. samples.
problem Learning rate analysis for distributed kernel ridge regression with dependent samples.
method Integral operator approach and covariance inequality for strong mixing sequences.
result Derives optimal learning rates for distributed kernel ridge regression.
A new method for deep Wishart processes improves kernel-based models.
problem Inference in deep Wishart processes is challenging due to the need for flexible distributions over positive semi-definite matrices.
method Developed a novel approach to flexible distributions over positive semi-definite matrices using the Bartlett decomposition of the Wishart probability density. Used this to create an approximate posterior for the DWP.
result Improved performance of inference in the DWP compared to DGP with equivalent prior.
New tests for distributional causal effects using improved kernel estimators.
problem Testing for higher-order moments and multidimensional outcomes affected by treatment.
method Improved kernel estimators based on doubly robust mean embeddings.
result New permutation-based tests for distributional causal effects with improved convergence rates.
Kernel tests assess equivalence between distributions without assuming specific moments.
problem Traditional goodness-of-fit tests fail to detect meaningful distributional differences.
method Proposes kernel-based tests using kernel Stein discrepancy and Maximum Mean Discrepancy.
result Tests assess the absence of meaningful distributional differences under controlled error rates.
A new method estimates multi-dimensional value distributions using Hilbert space embeddings.
problem Estimating value distributions in complex, multi-dimensional reinforcement learning settings.
method Hilbert space mappings and kernel mean embeddings to estimate the kernel mean embedding of multi-dimensional value distributions.
result Uniform convergence guarantees and robust off-policy evaluation demonstrated in simulations.
Adaptive kernel density estimation improves accuracy in high dimensions.
problem Challenges in high-dimensional density estimation with traditional methods.
method Pre-training a neural network to recommend location-adaptive kernels.
result Effective density estimation in high dimensions with improved accuracy.
A new method for analyzing adaptive experiments using kernel treatment effects.
problem Efficiently analyzing adaptive experiments that adjust treatment assignments based on outcomes.
method Kernel Treatment Effects (KTE) framework combining RKHS scores and witness functions.
result Effective for both mean shifts and higher-moment differences, outperforming adaptive baselines.
In this paper, we propose the distributed tree kernels (DTK) as a novel method to reduce time and space complexity of tree kernels. Using a linear complexity algorithm to compute vectors for trees, we embed feature spaces of tree fragments in low-dimensional spaces where the kernel computation is directly done with dot…
Kernel thinning compresses distributions more effectively than i.i.d. sampling or standard thinning.
problem Efficiently compressing distributions for better sampling and integration accuracy.
method Introduces kernel thinning, a procedure that compresses an n-point approximation of a distribution into a sqrt(n)-point approximation with comparable integration error.
result Kernel thinning achieves a maximum discrepancy in integration error of O_d(n^(-1/2) sqrt(log n)) in probability for compactly supported distributions and O_d(n^(-1/2) (log n)^(d+1/2) sqrt(log log n)) for sub-exponential distributions.
The role of kernels is central to machine learning. Motivated by the importance of power-law distributions in statistical modeling, in this paper, we propose the notion of power-law kernels to investigate power-laws in learning problem. We propose two power-law kernels by generalizing Gaussian and Laplacian kernels. Th…
We study the mixtures of factorizing probability distributions represented as visible marginal distributions in stochastic layered networks. We take the perspective of kernel transitions of distributions, which gives a unified picture of distributed representations arising from Deep Belief Networks (DBN) and other netw…
Deep neural nets optimize kernel parameters for non-parametric two-sample tests.
problem Determining if two samples come from the same distribution.
method Deep kernels trained to maximize test power, adapting to distribution smoothness and shape.
result Deep kernels outperform simpler kernels in high dimensions and complex data.
A new kernel for probability measures based on optimal transport.
problem Efficiently comparing and modeling distributions.
method Kernel over probability measures using regularized optimal transport and Hilbertian embedding.
result The proposed kernel enables Gaussian process modeling on distributions with theoretical and computational advantages.
We study the concentration of random kernel matrices around their mean. We derive nonasymptotic exponential concentration inequalities for Lipschitz kernels assuming that the data points are independent draws from a class of multivariate distributions on Rd, including the strongly log-concave distributions u…
Improved KSD test for better detection of differences in distributions.
problem Low power of KSD test when distributions have same modes but different mixing proportions.
method Perturb the observed sample using Markov transition kernels to improve KSD test power.
result Perturbed KSD test can lead to substantially higher power than the original KSD test.
Kernel SIVI improves variational inference by avoiding lower-level optimization.
problem Intractable densities in semi-implicit variational distributions.
method Kernel SIVI-SM uses a minimax formulation and kernel tricks to avoid lower-level optimization.
result Kernel Stein discrepancy (KSD) objective is computable and leads to convergence guarantees.
Kernel embeddings help estimate causal effects from observational data.
problem Estimating causal effects from observational data with confounding variables.
method Kernel embeddings in reproducing kernel Hilbert spaces (RKHS).
result Robust nonparametric framework for causal inference.
Faster convergence of kernel mean embeddings using variance information.
problem Speeding up the convergence rate of kernel mean embeddings.
method Leveraging variance information in reproducing kernel Hilbert space and estimating variance from data.
result Efficiently estimate variance information from data to achieve distribution-agnostic convergence bounds.
A new framework improves kernel Stein discrepancy tests for validating distributions.
problem Improving goodness-of-fit testing for non-normal distributions.
method Introducing Sf-KSD, a unifying framework for studying Stein operators in KSD-based tests.
result Sf-KSD guides the development of new tests and outperforms existing methods.
New method interpolates training data and is consistent for various data distributions.
problem Establishing generalization guarantees for ensemble methods in the interpolating regime.
method Developed manifold-Hilbert kernel for Riemannian manifolds and used it in ensemble classification.
result Consistent ensemble classification method for broad data distributions.
New recursive algorithm estimates conditional kernel mean embeddings in Hilbert space.
problem Estimating conditional distributions in RKHS for supervised learning.
method Recursive algorithm in L2 space for conditional kernel mean map. result Strong L2 consistency of recursive estimator proved. The paper analyzes and improves the learning rates of distributed kernel ridge regression.
problem Generalization performance and learning rates of distributed kernel ridge regression.
method The paper derives optimal learning rates for DKRR in expectation and probability, proposes a communication strategy to improve learning performance, and evaluates these through theory and experiments.
result The communication strategy significantly improves the learning performance of DKRR, as demonstrated by both theoretical assessments and numerical experiments.
Robust hypothesis testing designs a test for worst-case distributions using kernel methods.
problem Design a robust test for hypothesis testing under uncertainty sets.
method Data-driven uncertainty sets constructed using kernel mean embeddings and maximum mean discrepancy (MMD). Bayesian and Neyman-Pearson settings investigated.
result Proposed robust kernel tests are exponentially consistent and asymptotically optimal.
We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, distances between embeddings of distributions to reproducing kernel Hilbert spaces (RKHS), as establ…
A new method for distribution regression using sliced Wasserstein distance.
problem Learning functions over spaces of probabilities.
method Proposes an OT-based estimator using the Sliced Wasserstein distance.
result Proves universal consistency and excess risk bounds for the proposed estimator.
Formula for Toeplitz operator kernel on CR manifolds.
problem Analyzing Toeplitz operators on CR manifolds.
method Formula for the symbol of the kernel, asymptotic expansions.
result Formula for the values at the diagonal of the second coefficient in the expansion of the symbol of the kernel.
This work closes the theory-practice gap for distributed optimization methods by introducing a new regularity condition.
problem Existing convergence conditions for distributed optimization methods are violated by nearly all kernels used in practice.
method Introduces Hessian relative uniform continuity (HRUC) to guarantee convergence under mild conditions.
result Derives convergence guarantees for mirror descent-based gradient tracking without restrictive assumptions.
CTT compresses samples to test distributions near-linearly, outperforming existing methods.
problem Efficiently testing distributions with high power and near-linear runtime.
method Sample compression followed by permutation testing.
result CTT achieves near-linear runtime while maintaining high statistical power.
Algorithm reduces regret in distributed kernel bandits with shared randomness.
problem Minimizing regret in collaborative function maximization.
method Uniform exploration at local agents and shared randomness with central server.
result Achieves optimal regret order with sublinear communication cost.
A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be ex…
Kernel embeddings separate distinct probability distributions, simplifying testing.
problem Testing equality of non-atomic probability distributions.
method Kernel covariance embeddings and Gaussian measures in reproducing kernel Hilbert spaces.
result Testing for singularity between Gaussian measures is equivalent to testing for equality of non-atomic probability distributions.