Proposes a new method for posterior sampling using MMD with negative distance kernel.
problem Posterior sampling and conditional generative modeling.
method Approximates joint distribution using discrete Wasserstein gradient flows of MMD with negative distance kernel.
result Establishes an error bound for posterior distributions and proves the method is a Wasserstein gradient flow.
This paper connects Wasserstein distances to MMD norms for compressive statistical learning.
problem Comparing and controlling Wasserstein distances between probability distributions.
method Establishing conditions under which Wasserstein distances can be controlled by MMD norms.
result Introducing Wasserstein regularity for compressive statistical learning.
This thesis improves kernel-based distances for statistical inference and integration.
problem Efficiently measuring distances between probability distributions for robust and smooth modeling.
method Kernel-based distances, focusing on maximum mean discrepancy (MMD) and novel kernel quantile discrepancies.
result Improved MMD estimators for simulation-based inference and conditional expectations.
Paper improves MMD flow efficiency with Riesz kernels for image generation.
problem High computational costs in MMD flows for large scale computations.
method Introduces Riesz kernels and sliced MMD for efficient computation.
result Efficient computation of MMD gradients in one-dimensional setting.
This paper refines MMD for domain adaptation by balancing intra-class and inter-class distances.
problem Balancing intra-class and inter-class distances for better feature discriminability in domain adaptation.
method The paper theoretically proves two facts about MMD and proposes a novel discriminative MMD method to balance intra-class and inter-class distances.
result The proposed method improves feature discriminability and outperforms state-of-the-art methods.
New KQEs improve probability metrics without mean function constraints.
problem Improving probability metrics without relying on mean function representations.
method Kernel quantile embeddings (KQEs) to construct new distances.
result KQEs offer a competitive alternative to MMD with near-linear cost.
Maximum mean discrepancy (MMD), also called energy distance or N-distance in statistics and Hilbert-Schmidt independence criterion (HSIC), specifically distance covariance in statistics, are among the most popular and successful approaches to quantify the difference and independence of random variables, respectively. T…
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs. As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimizat…
New kernel improves MMDs with theoretical guarantees for gradient flows.
problem Non-smoothness of negative distance kernel in MMDs.
method Smoothed 1D absolute value function followed by fractional integral transform.
result Improved theoretical guarantees for Wasserstein gradient flows.
We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, maximum mean discrepancies (MMD), that is, distances between embeddings of distributions to reproduc…
A new distance metric compares probability distributions using kernel covariance operators.
problem Comparing probability distributions in machine learning tasks.
method Introduces a novel distance metric based on Schatten norm of kernel covariance operators.
result The new distance metric is more discriminative and robust to hyperparameters.
Improved MMD estimator for likelihood-free inference.
problem Computational challenges in estimating MMD for likelihood-free inference.
method Optimally-weighted MMD estimator with improved sample complexity.
result Significantly improved sample complexity for accurate MMD estimation.
New method approximates MMD using pseudo-differential operators and singular values.
problem Approximating MMD with pseudo-differential operators and singular values.
method Corresponding pseudo-differential operators to Mercer kernels, approximating p(x,y) with its first r singular values. result The new MMD distance measures the difference of two distributions with respect to r∗ local moments, where r∗ depends on singular values decay rate. MMD-Flagger detects hallucinations in LLMs by tracking MMD between outputs and temperature-generated counterparts.
problem Detecting hallucinations in large language models.
method Maximum Mean Discrepancy (MMD) to track the difference between model outputs and temperature-generated counterparts.
result MMD-Flagger detects most hallucinations by analyzing the shape of the MMD trajectory.
A new test statistic speeds up MMD while maintaining power.
problem Efficiently testing two distributions without permutations.
method Cross-MMD statistic based on sample-splitting and studentization.
result Cross-MMD has a limiting standard Gaussian distribution under the null.
Paper provides GOT convergence guarantees for sub-gamma distributions and dependent samples.
problem Estimating GOT distance under general settings.
method Gaussian-smoothed optimal transport (GOT) framework, sub-gamma distributions, dependent samples, kernel MMD distances.
result Convergence guarantees for GOT distance under more general settings.
The maximum mean discrepancy (MMD) is a kernel-based distance between probability distributions useful in many applications (Gretton et al. 2012), bearing a simple estimator with pleasing computational and statistical properties. Being able to efficiently estimate the variance of this estimator is very helpful to vario…
The paper describes flows of MMD functionals with distance kernel and quantile functions.
problem Wasserstein gradient flows of MMD functionals with negative distance kernel.
method Characterization via Cauchy problem on L2(0,1), solution via subdifferential construction. result Flow invariance and smoothing properties on subsets of C(0,1), absolute continuity of initial measures. Optimizes two-sample tests for non-Euclidean domains using spectral regularization.
problem Optimizing two-sample tests for non-Euclidean domains.
method Spectral regularization of MMD test to achieve minimax optimality.
result Proposes a spectral regularization method that improves test optimality.
A new variable importance measure for DRFs detects broader impacts on output distributions.
problem Estimating full conditional distributions of multivariate outputs given inputs.
method Based on the drop and relearn principle and MMD distance.
result Consistent and high-performing variable importance measure for DRFs.
New distances measure mixtures of Gaussians, useful in machine learning.
problem Comparing distributions with disjoint supports.
method Schoenberg-Rao distances based on concave Rao's entropy.
result Closed-form distances for mixtures of Gaussians.
New metrics improve quantum ensemble learning efficiency and power.
problem Quantum ensembles' distances poorly understood due to measurement constraints.
method Introduce MMD-k hierarchy of integral probability metrics for quantum ensembles. result MMD-k requires fewer samples for full discriminative power at higher k. Generative moment matching network (GMMN) is a deep generative model that differs from Generative Adversarial Network (GAN) by replacing the discriminator in GAN with a two-sample test based on kernel maximum mean discrepancy (MMD). Although some theoretical guarantees of MMD have been studied, the empirical performanc…
New algorithm improves clustering and quantization using MMD.
problem Approximating probability distributions with weighted mixtures of Dirac measures.
method Gradient flow, mean shift, and MMD-optimal quantization.
result MSIP algorithm is more robust than state-of-the-art methods.
Distributionally robust optimization (DRO) has attracted attention in machine learning due to its connections to regularization, generalization, and robustness. Existing work has considered uncertainty sets based on phi-divergences and Wasserstein distances, each of which have drawbacks. In this paper, we study DRO wit…
A new measure scales MMD to assess distribution closeness.
problem Testing statistical significance of distribution closeness.
method Norm-adaptive MMD (NAMMD) for distributional discrepancy.
result NAMMD-based DCT has higher test power than MMD-based DCT.
Study extends DRO with IPMs, linking robustness to regularization and GANs.
problem Addressing robustness of deep neural networks to adversarial attacks.
method Distributionally Robust Optimization (DRO) with Integral Probability Metrics (IPMs).
result DRO under any IPM corresponds to a family of regularization penalties.
Proposes GFMMD for comparing signals on graphs.
problem Computing distances between distributions on graphs.
method Graph Fourier MMD (GFMMD) using optimal witness functions.
result Analytical solution and embedding of distributions.
Mean embeddings provide an extremely flexible and powerful tool in machine learning and statistics to represent probability distributions and define a semi-metric (MMD, maximum mean discrepancy; also called N-distance or energy distance), with numerous successful applications. The representation is constructed as the e…
Boosts kernel two-sample test power with multiple kernels.
problem Detecting differences between two distributions over metric spaces.
method Combining MMD estimates over multiple kernels using Mahalanobis distance.
result More powerful in detecting a wide range of alternatives in finite samples.
Implicit Generative Models (IGMs) such as GANs have emerged as effective data-driven models for generating samples, particularly images. In this paper, we formulate the problem of learning an IGM as minimizing the expected distance between characteristic functions. Specifically, we minimize the distance between charact…
This work aims to improve semi-supervised learning in a neural network architecture by introducing a hybrid supervised and unsupervised cost function. The unsupervised component is trained using a differentiable estimator of the Maximum Mean Discrepancy (MMD) distance between the network output and the target dataset. …
Deep neural networks can approximate any target probability distribution given certain conditions.
problem Approximating complex probability distributions with deep neural networks.
method Proving the existence of a deep neural network mapping that approximates a target distribution under various integral probability metrics.
result Upper bounds on the size of the neural network in terms of dimension and approximation error for different metrics.
PolyGraph Discrepancy improves graph generative model evaluation.
problem Inability of existing metrics to provide an absolute performance measure and comparability across different graph descriptors.
method Approximates Jensen-Shannon distance using binary classifiers trained to distinguish between real and generated graphs.
result PGD provides a more robust and insightful evaluation compared to MMD metrics.
Stochastic-sampling-based Generative Neural Networks, such as Restricted Boltzmann Machines and Generative Adversarial Networks, are now used for applications such as denoising, image occlusion removal, pattern completion, and motion synthesis. In scenarios which involve performing such inference tasks with these model…
Study on kernel tests for high-dimensional data, focusing on MMD and CLT.
problem Asymptotic behavior of kernel two-sample tests in high dimensions and large samples.
method Maximum mean discrepancy (MMD) with isotropic kernels, deriving asymptotic expansions and CLT.
result Interplay between moment discrepancy and dimension-and-sample orders in kernel tests.
New method finds points for approximating distributions faster.
problem Approximating target probability distributions using finite points.
method Stationary MMD points computed via MMD gradient flows.
result Stationary MMD points converge faster than global minimizers.
While likelihood-based inference and its variants provide a statistically efficient and widely applicable approach to parametric inference, their application to models involving intractable likelihoods poses challenges. In this work, we study a class of minimum distance estimators for intractable generative models, tha…
Plug-in robust NPE method adapts summaries independently of pretrained NPE.
problem Misspecification of neural posterior estimators under test data distribution.
method Minimum-distance summaries using maximum mean discrepancy (MMD).
result Substantial robustness gains with minimal additional overhead.
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
Study on Wasserstein gradient flow for MMD between Coulomb measures.
problem Analyzing the long-time behavior of MMD between probability and target measures using Coulomb kernels.
method Existence of global weak solutions, ultracontractive estimate, regularity analysis, exponential decay proof, defective Polyak-Lojasiewicz inequality.
result Exponential decay of squared MMD toward a uniformly positive target measure on flat torus.
A new method uses neural tangent kernel to efficiently compute MMD statistic.
problem Efficiently computing Maximum Mean Discrepancy (MMD) statistic with low memory and computational complexity.
method Identifies a connection between neural tangent kernel (NTK) and MMD to develop a computationally and memory-efficient approach.
result The proposed NTK-MMD statistic is validated through numerical experiments on synthetic and real-world datasets.
Gradient descent on MMD GAN parameter space converges globally to target distribution.
problem Convergence of gradient descent in Maximum Mean Discrepancy (MMD) GANs.
method Proposes a parametric kernelized gradient flow that mimics the min-max game in gradient regularized MMD GAN.
result Gradient descent on the generator's parameter space in gradient regularized MMD GAN is globally convergent to the target distribution under certain conditions.
We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical Lp losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the …
Dependent MMD coresets help compare multiple related datasets.
problem Comparing multiple related datasets for insights into model generalization.
method Dependent MMD coresets for collections of datasets.
result Dependent MMD coresets facilitate comparison and understanding of multiple related datasets.
A new method optimizes MMD test power by dynamically selecting kernels, overcoming traditional trade-offs.
problem Fixed kernels fail to distinguish certain distributions, leading to overfitting and variance collapse.
method Complexity-Penalized MMD (CP-MMD) criterion, derived from concentration inequality, optimizes kernel selection.
result CP-MMD maximizes true test power while ensuring unconditional Type-I validity, matching or exceeding state-of-the-art performance.
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
EVI-MMD approximates target distributions via MMD minimization with adaptive kernel.
problem Approximating target distributions using kernel discrepancy methods.
method EVI-MMD uses Maximum Mean Discrepancy (MMD) to minimize kernel discrepancy, solving ODEs with implicit Euler scheme and L-BFGS optimization.
result EVI-MMD with adaptive bandwidth selection significantly improves performance in sampling problems.