Gradient descent on MMD GAN parameter space converges globally to target distribution.
problem Convergence of gradient descent in Maximum Mean Discrepancy (MMD) GANs.
method Proposes a parametric kernelized gradient flow that mimics the min-max game in gradient regularized MMD GAN.
result Gradient descent on the generator's parameter space in gradient regularized MMD GAN is globally convergent to the target distribution under certain conditions.
Regularizes f-divergences with MMD to analyze Wasserstein flows.
problem Limitations of f-divergences in measures' support. method Rewriting MMD regularization as Moreau envelope in RKHS, analyzing gradients.
result Analysis of Wasserstein flows of MMD-regularized f-divergences. Improved MMD test for non-Euclidean data with spectral regularization.
problem Inefficient and impractical MMD goodness-of-fit tests for non-Euclidean data.
method Spectral regularization of MMD test, extending results to general cases.
result Minimax optimal test for non-Euclidean data with appropriate regularization.
Optimizes two-sample tests for non-Euclidean domains using spectral regularization.
problem Optimizing two-sample tests for non-Euclidean domains.
method Spectral regularization of MMD test to achieve minimax optimality.
result Proposes a spectral regularization method that improves test optimality.
Distributionally robust optimization (DRO) has attracted attention in machine learning due to its connections to regularization, generalization, and robustness. Existing work has considered uncertainty sets based on phi-divergences and Wasserstein distances, each of which have drawbacks. In this paper, we study DRO wit…
We propose a simple neural network model to deal with the domain adaptation problem in object recognition. Our model incorporates the Maximum Mean Discrepancy (MMD) measure as a regularization in the supervised learning to reduce the distribution mismatch between the source and target domains in the latent space. From …
We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties. The MMD is an integral probability metric defined for a reproducing kernel Hilbert space (RKHS), and serves as a metric on probability measures for a sufficiently rich RKHS. We obtain conditions for conv…
We propose a principled method for gradient-based regularization of the critic of GAN-like models trained by adversarially optimizing the kernel of a Maximum Mean Discrepancy (MMD). We show that controlling the gradient of the critic is vital to having a sensible loss function, and devise a method to enforce exact, ana…
Study extends DRO with IPMs, linking robustness to regularization and GANs.
problem Addressing robustness of deep neural networks to adversarial attacks.
method Distributionally Robust Optimization (DRO) with Integral Probability Metrics (IPMs).
result DRO under any IPM corresponds to a family of regularization penalties.
A new method for representation and metric learning on manifolds boosts performance.
problem Improving representation and metric learning on complex, non-Euclidean data.
method Atlas-based manifold representation with a modified encoder and MMD loss.
result Substantial performance boost over baseline for low-dimensional encodings.
This paper connects Wasserstein distances to MMD norms for compressive statistical learning.
problem Comparing and controlling Wasserstein distances between probability distributions.
method Establishing conditions under which Wasserstein distances can be controlled by MMD norms.
result Introducing Wasserstein regularity for compressive statistical learning.
A new gradient flow for MMD with closed-form implementation.
problem Existing gradient flows either lack tractable numerical implementation or require strong assumptions.
method Introduces a (de)-regularized Maximum Mean Discrepancy (DrMMD) and its gradient flow.
result Guarantees near-global convergence for a broad class of targets in both continuous and discrete time.
The paper shows how MMD metrizes weak convergence for certain kernels.
problem Characterizing MMD metrizing weak convergence for a wide class of kernels.
method Proving MMD metrizes weak convergence for specific kernels on a locally compact space.
result Corrected prior results and identified new kernels metrizing weak convergence.
New method finds points for approximating distributions faster.
problem Approximating target probability distributions using finite points.
method Stationary MMD points computed via MMD gradient flows.
result Stationary MMD points converge faster than global minimizers.
Generative adversarial network for probabilistic forecasting of random systems.
problem Forecasting random dynamical systems without distributional assumptions.
method Recurrent neural network and generative adversarial network (GAN) with regularization based on maximum mean discrepancy (MMD).
result The proposed model successfully forecasts complex stochastic processes with multiple-step predictions.
KALE flow approximates KL divergence for distributions with disjoint support.
problem Approximating KL divergence for distributions with disjoint support.
method Relaxed KL gradient flow using RKHS, continuously interpolating between KL and MMD.
result Global convergence of KALE flow under sufficient smoothness assumptions.
A new method uses neural tangent kernel to efficiently compute MMD statistic.
problem Efficiently computing Maximum Mean Discrepancy (MMD) statistic with low memory and computational complexity.
method Identifies a connection between neural tangent kernel (NTK) and MMD to develop a computationally and memory-efficient approach.
result The proposed NTK-MMD statistic is validated through numerical experiments on synthetic and real-world datasets.
Paper improves MMD flow efficiency with Riesz kernels for image generation.
problem High computational costs in MMD flows for large scale computations.
method Introduces Riesz kernels and sliced MMD for efficient computation.
result Efficient computation of MMD gradients in one-dimensional setting.
A new test statistic speeds up MMD while maintaining power.
problem Efficiently testing two distributions without permutations.
method Cross-MMD statistic based on sample-splitting and studentization.
result Cross-MMD has a limiting standard Gaussian distribution under the null.
Dependent MMD coresets help compare multiple related datasets.
problem Comparing multiple related datasets for insights into model generalization.
method Dependent MMD coresets for collections of datasets.
result Dependent MMD coresets facilitate comparison and understanding of multiple related datasets.
A new method optimizes MMD test power by dynamically selecting kernels, overcoming traditional trade-offs.
problem Fixed kernels fail to distinguish certain distributions, leading to overfitting and variance collapse.
method Complexity-Penalized MMD (CP-MMD) criterion, derived from concentration inequality, optimizes kernel selection.
result CP-MMD maximizes true test power while ensuring unconditional Type-I validity, matching or exceeding state-of-the-art performance.
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
EVI-MMD approximates target distributions via MMD minimization with adaptive kernel.
problem Approximating target distributions using kernel discrepancy methods.
method EVI-MMD uses Maximum Mean Discrepancy (MMD) to minimize kernel discrepancy, solving ODEs with implicit Euler scheme and L-BFGS optimization.
result EVI-MMD with adaptive bandwidth selection significantly improves performance in sampling problems.
sig-MMD tests compare path distributions using kernel methods.
problem Comparing path distributions in stochastic processes.
method Signature kernel for path space valued distributions.
result sig-MMD can lead to Type 2 errors in limited data settings.
Study improves MMD estimation for two distributions with mismeasured data.
problem Estimating MMD under contaminated data.
method Partial identification of MMD bounds with a fast convergence rate.
result Proposes a method to estimate MMD bounds that converge faster than alternatives.
MMD-Flagger detects hallucinations in LLMs by tracking MMD between outputs and temperature-generated counterparts.
problem Detecting hallucinations in large language models.
method Maximum Mean Discrepancy (MMD) to track the difference between model outputs and temperature-generated counterparts.
result MMD-Flagger detects most hallucinations by analyzing the shape of the MMD trajectory.
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs. As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimizat…
Study on Wasserstein gradient flow for MMD between Coulomb measures.
problem Analyzing the long-time behavior of MMD between probability and target measures using Coulomb kernels.
method Existence of global weak solutions, ultracontractive estimate, regularity analysis, exponential decay proof, defective Polyak-Lojasiewicz inequality.
result Exponential decay of squared MMD toward a uniformly positive target measure on flat torus.
Paper optimizes PCA for fairness using MMD and Stiefel manifold optimization.
problem Fair principal component analysis (PCA) to minimize MMD between protected classes.
method Formulates fair PCA as non-convex optimization over Stiefel manifold, solves using REPMS with theoretical guarantees.
result Our approach outperforms prior work in fairness, explained variance, and runtime.
New method for MMD with unequal sample sizes improves test power.
problem Existing MMD methods assume equal sample sizes, discarding valuable data.
method Extended generalized U-statistics to handle unequal sample sizes.
result New asymptotic distributions and power optimization for MMD with unequal sample sizes.
A new MMD-based test combines kernels for two-sample testing without splitting data.
problem Efficiently testing if two datasets come from the same distribution without splitting data.
method Proposes a novel statistic based on Maximum Mean Discrepancy (MMD) that combines kernels, proving concentration bounds and showing data-dependent kernel selection.
result Exponential concentration bounds and improved test power compared to existing methods.
MMD test detects adversarial attacks by addressing kernel limitations and non-independence issues.
problem MMD test's failure to detect adversarial attacks.
method Replaced Gaussian kernel with deep kernel, maximized test power, and used wild bootstrap for non-independence.
result MMD test is aware of adversarial attacks.
MMD-B-Fair learns fair representations by minimizing MMD test power.
problem Learning fair representations of data while preserving target attributes.
method Kernel two-sample testing and block testing schemes.
result Minimizing MMD test power allows hiding sensitive attribute information.
Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. F…
Improved MMD test for two-sample testing with random Fourier features.
problem Quadratic-time complexity of MMD test for large-scale analysis.
method Approximated MMD test using random Fourier features, investigating time-power trade-off.
result Sub-quadratic time complexity with same minimax separation rates as MMD test.
A test for comparing function samples using MMD.
problem Testing if two functional data samples come from the same distribution.
method Maximum Mean Discrepancy (MMD) for functional data, with theoretical scaling analysis.
result The proposed test is effective and robust to functional reconstructions.
We study the maximum mean discrepancy (MMD) in the context of critical transitions modelled by fast-slow stochastic dynamical systems. We establish a new link between the dynamical theory of critical transitions with the statistical aspects of the MMD. In particular, we show that a formal approximation of the MMD near …
Learning with kernels is an important concept in machine learning. Standard approaches for kernel methods often use predefined kernels that require careful selection of hyperparameters. To mitigate this burden, we propose in this paper a framework to construct and learn a data-dependent kernel based on random features …
New method improves MMD estimation without convexity assumptions.
problem Lack of theoretical guarantees for MMD estimation algorithms.
method Preconditioned gradient descent (PGD) scheme for MMD optimization.
result PGD scheme converges globally under specific conditions.
The maximum mean discrepancy (MMD) is a kernel-based distance between probability distributions useful in many applications (Gretton et al. 2012), bearing a simple estimator with pleasing computational and statistical properties. Being able to efficiently estimate the variance of this estimator is very helpful to vario…
The Maximum Mean Discrepancy (MMD) has found numerous applications in statistics and machine learning, most recently as a penalty in the Wasserstein Auto-Encoder (WAE). In this paper we compute closed-form expressions for estimating the Gaussian kernel based MMD between a given distribution and the standard multivariat…
New framework improves model reliability under distribution shifts.
problem Lack of formal guarantees connecting shift magnitude to prediction reliability in TTA methods.
method Develops a PAC-Bayesian framework interpreting MMD-balls as credal sets.
result Establishes generalization bounds and provides epistemic uncertainty quantification.
Proposes a new method for posterior sampling using MMD with negative distance kernel.
problem Posterior sampling and conditional generative modeling.
method Approximates joint distribution using discrete Wasserstein gradient flows of MMD with negative distance kernel.
result Establishes an error bound for posterior distributions and proves the method is a Wasserstein gradient flow.
Two-sample tests using MMD control type I error and achieve optimal power.
problem Developing reliable nonparametric two-sample tests for small sample sizes.
method Maximum Mean Discrepancy (MMD) for constructing novel nonparametric tests, proving non-asymptotic error control and optimality.
result MMDAgg test controls type I error and achieves minimax rate over Sobolev balls, outperforming other tests.
Unified method for MMD variance estimation improves accuracy and computational efficiency.
problem Variance estimation for MMD in nonparametric testing.
method Unified finite-sample characterization of MMD variance through U-statistic and Hoeffding decomposition; exact acceleration method for univariate case.
result Unified estimators improve accuracy and computational efficiency for MMD variance.
"Which Generative Adversarial Networks (GANs) generates the most plausible images?" has been a frequently asked question among researchers. To address this problem, we first propose an \emph{incomplete} U-statistics estimate of maximum mean discrepancy MMDinc to measure the distribution discrepancy betwee…
Paper provides GOT convergence guarantees for sub-gamma distributions and dependent samples.
problem Estimating GOT distance under general settings.
method Gaussian-smoothed optimal transport (GOT) framework, sub-gamma distributions, dependent samples, kernel MMD distances.
result Convergence guarantees for GOT distance under more general settings.
The maximum mean discrepancy (MMD) is a recently proposed test statistic for two-sample test. Its quadratic time complexity, however, greatly hampers its availability to large-scale applications. To accelerate the MMD calculation, in this study we propose an efficient method called FastMMD. The core idea of FastMMD is …