arXiv research

Novel algorithm speeds up computation of Sobolev IPM for graph-based probability measures.

We study the minimax optimal rates for estimating a range of Integral Probability Metrics (IPMs) between two unknown probability measures, based on $n$ independent samples from them. Curiously, we show that estimating the IPM itself between probability measures, is not significantly easier than estimating the probabili…

We propose a new Integral Probability Metric (IPM) between distributions: the Sobolev IPM. The Sobolev IPM compares the mean discrepancy of two distributions for functions (critic) restricted to a Sobolev ball defined with respect to a dominant measure $μ$. We show that the Sobolev IPM compares two distributions in hig…

Unified view of KL-divergence and IPMs via DRE, with new DRM metrics.

We introduce new families of Integral Probability Metrics (IPM) for training Generative Adversarial Networks (GAN). Our IPMs are based on matching statistics of distributions embedded in a finite dimensional feature space. Mean and covariance feature matching IPMs allow for stable training of GANs, which we will call M…

A new IPM uses ReLU networks to measure probability discrepancies.

Generalizes Sobolev IPM for graph-based measures using Orlicz geometric structure.

Study minimax rates for density estimation under Huber contamination and Besov IPM losses.

Toda flow explained as a porous medium equation.

Proposes tests for comparing high-dimensional manifold samples.

New framework improves experimental design using integral probability metrics.

The paper explores fair predictors in supervised learning using IPMs and Kolmogorov distance.

New bounds use IPMs to improve generalization in machine learning.

IPMs struggle with hyperbolic spaces due to polynomially growing barrier parameters.

We study the problem of estimating a nonparametric probability density under a large family of losses called Besov IPMs, which include, for example, $\mathcal{L}^p$ distances, total variation distance, and generalizations of both Wasserstein and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide b…

Study extends DRO with IPMs, linking robustness to regularization and GANs.

Improved image generation quality using closed-form discriminator guidance in diffusion models.

New error bounds for GANs with nonlinear objective functions derived.

A nonparametric two-sample test using a parametric integral probability metric

The paper proposes a new method for covariate balancing using IPM to improve causal inference.

Develops a new divergence framework that combines $f$-divergences and IPMs.

This paper is concerned with sequential filtering based stochastic optimization (FSO) approaches that leverage a probabilistic perspective to implement the incremental proximity method (IPM). The present FSO methods are derived based on the Kalman filter (KF) and the extended KF (EKF). In contrast with typical methods …

Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…

New divergences improve estimation and GAN training performance.

TMLE improves IPM estimation for ecological population dynamics.

We present an extension of the Kolmogorov-Smirnov (KS) two-sample test, which can be more sensitive to differences in the tails. Our test statistic is an integral probability metric (IPM) defined over a higher-order total variation ball, recovering the original KS test as its simplest case. We give an exact representer…

Paper proposes a new method to quantify uncertainty in machine learning models.

The paper bounds the expectation of empirical processes indexed by Hölder classes.

New findings show fixed-kernel discriminators are weaker than feature-learning ones.

We propose the Sobolev Independence Criterion (SIC), an interpretable dependency measure between a high dimensional random variable X and a response variable Y . SIC decomposes to the sum of feature importance scores and hence can be used for nonlinear feature selection. SIC can be seen as a gradient regularized Integr…

Expands learning paradigm to stochastic orders using Choquet-Toland distance and Variational Dominance Criterion.

A new meta-learning framework that assigns weights to source tasks based on target samples.

Paper proposes a new framework to improve policy optimization by aligning real and simulated data distributions.

The recent empirical success of unsupervised cross-domain mapping algorithms, between two domains that share common characteristics, is not well-supported by theoretical justifications. This lacuna is especially troubling, given the clear ambiguity in such mappings. We work with adversarial training methods based on IP…

While Generative Adversarial Networks (GANs) have empirically produced impressive results on learning complex real-world distributions, recent works have shown that they suffer from lack of diversity or mode collapse. The theoretical work of Arora et al. suggests a dilemma about GANs' statistical properties: powerful d…

A new GAN model uses characteristic functions to improve image generation.

Unified framework for convergence of discrete diffusion models without state space size dependence.

We present a quantum interior-point method (IPM) for second-order cone programming (SOCP) that runs in time $\widetilde{O} \left( n\sqrt{r} \frac{ζκ}{δ^2} \log \left(1/ε\right) \right)$ where $r$ is the rank and $n$ the dimension of the SOCP, $δ$ bounds the distance of intermediate solutions from the cone boundary, $ζ$…

New test uses neural networks to compare distributions, outperforming traditional methods.

Paper explores robust estimators for kernel exponential families using smoothed total variation distances.

Unified score and distance-based GoF tests for model adequacy.

In standard generative adversarial network (SGAN), the discriminator estimates the probability that the input data is real. The generator is trained to increase the probability that fake data is real. We argue that it should also simultaneously decrease the probability that real data is real because 1) this would accou…

As an effective way of metric learning, triplet loss has been widely used in many deep learning tasks, including face recognition and person-ReID, leading to many states of the arts. The main innovation of triplet loss is using feature map to replace softmax in the classification task. Inspired by this concept, we prop…

An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…

The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.

Develops a two-sample test using projected Wasserstein distance to handle high-dimensional data.

Measuring conditional dependencies among the variables of a network is of great interest to many disciplines. This paper studies some shortcomings of the existing dependency measures in detecting direct causal influences or their lack of ability for group selection to capture strong dependencies and accordingly introdu…

New method solves optimization problems with stochastic objectives and constraints.