arXiv research

We know SGAN may have a risk of gradient vanishing. A significant improvement is WGAN, with the help of 1-Lipschitz constraint on discriminator to prevent from gradient vanishing. Is there any GAN having no gradient vanishing and no 1-Lipschitz constraint on discriminator? We do find one, called GAN-QP. To construct a …

1-Lipschitz networks are as accurate as classical networks and offer robustness.

The paper explores theoretical insights into WGANs for better understanding and stability.

In this paper, we study the convergence of generative adversarial networks (GANs) from the perspective of the informativeness of the gradient of the optimal discriminative function. We show that GANs without restriction on the discriminative function space commonly suffer from the problem that the gradient produced by …

This paper examines weight initialization for 1-Lipschitz networks to improve robustness against adversarial attacks.

Maps preserving mass and injective on boundary are isometries.

Orthogonium offers unified, efficient layers for robust deep learning.

In this paper, we study the Lévy-Milman concentration phenomenon of 1-Lipschitz maps into infinite dimensional metric spaces. Our main theorem asserts that the concentration to an infinite dimensional $\ell^p$-ball with the $\ell^q$-distance function for $1\leq p<q\leq +\infty$ is equivalent to the concentration to the…

Existing depth separation results for constant-depth networks essentially show that certain radial functions in $\mathbb{R}^d$, which can be easily approximated with depth $3$ networks, cannot be approximated by depth $2$ networks, even up to constant accuracy, unless their size is exponential in $d$. However, the func…

We prove that if a geodesic metric measure space satisfies a comparison condition for isoperimetric profile and if the observable variance is maximal, then the space is foliated by minimal geodesics, where the observable variance is defined to be the supremum of the variance of 1-Lipschitz functions on the space. Our r…

1-Lipschitz neural networks produce clearer, more focused Saliency Maps for explainable AI.

Despite being impactful on a variety of problems and applications, the generative adversarial nets (GANs) are remarkably difficult to train. This issue is formally analyzed by \cite{arjovsky2017towards}, who also propose an alternative direction to avoid the caveats in the minmax two-player training of GANs. The corres…

In this paper, we consider a concentration of measure problem on Riemannian manifolds with boundary. We study concentration phenomena of non-negative $1$-Lipschitz functions with Dirichlet boundary condition around zero, which is called boundary concentration phenomena. We first examine relation between boundary concen…

New error bounds for GANs with nonlinear objective functions derived.

Optimizes optimal transport distances using low-dimensional embeddings.

The Nash-Kuiper Theorem states that the collection of $C^1$-isometric embeddings from a Riemannian manifold $M^n$ into $\mathbb{E}^N$ is $C^0$-dense within the collection of all smooth 1-Lipschitz embeddings provided that $n < N$. This result is now known to be a consequence of Gromov's more general $h$-principle. Ther…

Maps between certain Lipschitz manifolds are isometries if they preserve volume.

New neural network design resists small $\ell_\infty$-norm adversarial perturbations.

For a given $1$-Lipschitz map $u\colon\mathbb{R}^n\to\mathbb{R}^m$ we define a partition, up to a set of Lebesgue measure zero, of $\mathbb{R}^n$ into maximal closed convex sets such that restriction of $u$ is an isometry on these sets. We consider a disintegration, with respect to this partition, of a log-concave meas…

Proves rigidity for maps between manifolds using degree theory and current developments.

Maps persistence diagrams into Hilbert and Euclidean spaces with explicit distortions.

We show that for a metric space with an even number of points there is a 1-Lipschitz map to a tree-like space with the same matching number. This result gives the first basic version of an unoriented Kantorovich duality. The study of the duality gives a version of global calibrations for 1-chains with coefficients in $…

We study the homeomorphic extension of biholomorphisms between convex domains in $\mathbb C^d$ without boundary regularity and boundedness assumptions. Our approach relies on methods from coarse geometry, namely the correspondence between the Gromov boundary and the topological boundaries of the domains and the dynamic…

Let $X$ and $Y$ be length metric spaces. Let $\mathcal H^n$ denote the $n$-dimensional Hausdorff measure. The Lipschitz-Volume Rigidity is a property that if there exists a 1-Lipschitz map $f\colon X\to Y$ and $0<\mathcal H^n(X)=\mathcal H^n(f(X))<\infty$, then $f$ preserves the length of path. This property holds for …

We prove a Lipschitz-Volume rigidity theorem in Alexandrov geometry, that is, if a 1-Lipschitz map $f\colon X=\amalg X_\ell\to Y$ between Alexandrov spaces preserves volume, then it is a path isometry and an isometry when restricted to the interior of $X$. We furthermore characterize the metric structure on $Y$ with re…

LOT improves adversarial robustness by training 1-Lipschitz convolution layers.

Researchers found counterexamples to conjectures about optimal transport maps on curved spaces.

Lipschitz maps on metric surfaces are rigid if they preserve area.

We study time-like hypersurfaces with vanishing mean curvature in the (3+1) dimensional Minkowski space, which are the hyperbolic counterparts to minimal embeddings of Riemannian manifolds. The catenoid is a stationary solution of the associated Cauchy problem. This solution is linearly unstable, and we show that this …

New memory-query tradeoffs for convex optimization algorithms.

The degree condition affects the rigidity of maps between manifolds.

Memory-constrained algorithms need superlinear memory for efficient convex optimization.

The study quantifies and compares aleatoric and epistemic discrimination in ML models.

The measure concentration property of an mm-space $X$ is roughly described as that any 1-Lipschitz map on $X$ to a metric space $Y$ is almost close to a constant map. The target space $Y$ is called the screen. The case of $Y=\mathbb{R}$ is widely studied in many literature (see \cite{gromov}, \cite{ledoux}, \cite{mil2}…

Training neural networks under a strict Lipschitz constraint is useful for provable adversarial robustness, generalization bounds, interpretable gradients, and Wasserstein distance estimation. By the composition property of Lipschitz functions, it suffices to ensure that each individual affine transformation or nonline…

Discrimination-aware classification is receiving an increasing attention in data science fields. The pre-process methods for constructing a discrimination-free classifier first remove discrimination from the training data, and then learn the classifier from the cleaned data. However, they lack a theoretical guarantee f…

Discriminator guidance improves autoregressive diffusion models for generating molecular graphs.

Kernel discriminant analysis uses nonlinear embeddings to improve classification.

The present paper is composed of two parts. In the first one we define two pseudo-metrics $L_F$ and $K_F$ on the Teichmuüller space of semi-translation surfaces $\mathcal{TQ}_g(\underline k,ε)$, which are the symmetric counterparts to the metrics defined by William Thurston on $\mathcal{T}_g^n$. We prove some nice prop…

Discriminative clustering uses mutual information to cluster data.

Unified plug-in approach for estimating symmetric properties of distributions efficiently.

Generative adversarial networks (GANs) are one of the most popular approaches when it comes to training generative models, among which variants of Wasserstein GANs are considered superior to the standard GAN formulation in terms of learning stability and sample quality. However, Wasserstein GANs require the critic to b…

PBN combines generative and discriminative capabilities in a neural network.

JacNet learns Jacobians to enforce structure on derivatives for invertibility and Lipschitz functions.

We propose to tackle the mode collapse problem in generative adversarial networks (GANs) by using multiple discriminators and assigning a different portion of each minibatch, called microbatch, to each discriminator. We gradually change each discriminator's task from distinguishing between real and fake samples to disc…

Study infinite Euclidean distance discriminants of algebraic varieties.

Optimized GAN discriminator using polyharmonic interpolation.

Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.