arXiv research

This paper examines weight initialization for 1-Lipschitz networks to improve robustness against adversarial attacks.

1-Lipschitz networks are as accurate as classical networks and offer robustness.

Orthogonium offers unified, efficient layers for robust deep learning.

Existing depth separation results for constant-depth networks essentially show that certain radial functions in $\mathbb{R}^d$, which can be easily approximated with depth $3$ networks, cannot be approximated by depth $2$ networks, even up to constant accuracy, unless their size is exponential in $d$. However, the func…

1-Lipschitz neural networks produce clearer, more focused Saliency Maps for explainable AI.

New neural network design resists small $\ell_\infty$-norm adversarial perturbations.

Maps preserving mass and injective on boundary are isometries.

In this paper, we study the Lévy-Milman concentration phenomenon of 1-Lipschitz maps into infinite dimensional metric spaces. Our main theorem asserts that the concentration to an infinite dimensional $\ell^p$-ball with the $\ell^q$-distance function for $1\leq p<q\leq +\infty$ is equivalent to the concentration to the…

We know SGAN may have a risk of gradient vanishing. A significant improvement is WGAN, with the help of 1-Lipschitz constraint on discriminator to prevent from gradient vanishing. Is there any GAN having no gradient vanishing and no 1-Lipschitz constraint on discriminator? We do find one, called GAN-QP. To construct a …

Optimizes optimal transport distances using low-dimensional embeddings.

The paper explores theoretical insights into WGANs for better understanding and stability.

We prove that if a geodesic metric measure space satisfies a comparison condition for isoperimetric profile and if the observable variance is maximal, then the space is foliated by minimal geodesics, where the observable variance is defined to be the supremum of the variance of 1-Lipschitz functions on the space. Our r…

LOT improves adversarial robustness by training 1-Lipschitz convolution layers.

In this paper, we consider a concentration of measure problem on Riemannian manifolds with boundary. We study concentration phenomena of non-negative $1$-Lipschitz functions with Dirichlet boundary condition around zero, which is called boundary concentration phenomena. We first examine relation between boundary concen…

The Nash-Kuiper Theorem states that the collection of $C^1$-isometric embeddings from a Riemannian manifold $M^n$ into $\mathbb{E}^N$ is $C^0$-dense within the collection of all smooth 1-Lipschitz embeddings provided that $n < N$. This result is now known to be a consequence of Gromov's more general $h$-principle. Ther…

Maps between certain Lipschitz manifolds are isometries if they preserve volume.

JacNet learns Jacobians to enforce structure on derivatives for invertibility and Lipschitz functions.

Training neural networks under a strict Lipschitz constraint is useful for provable adversarial robustness, generalization bounds, interpretable gradients, and Wasserstein distance estimation. By the composition property of Lipschitz functions, it suffices to ensure that each individual affine transformation or nonline…

For a given $1$-Lipschitz map $u\colon\mathbb{R}^n\to\mathbb{R}^m$ we define a partition, up to a set of Lebesgue measure zero, of $\mathbb{R}^n$ into maximal closed convex sets such that restriction of $u$ is an isometry on these sets. We consider a disintegration, with respect to this partition, of a log-concave meas…

Proves rigidity for maps between manifolds using degree theory and current developments.

Deep neural networks can approximate complex functions through repeated compositions of a fixed-size ReLU network.

Framework for designing nonlinearities in neural networks with slope constraints.

Generative adversarial networks (GANs) are one of the most popular approaches when it comes to training generative models, among which variants of Wasserstein GANs are considered superior to the standard GAN formulation in terms of learning stability and sample quality. However, Wasserstein GANs require the critic to b…

Maps persistence diagrams into Hilbert and Euclidean spaces with explicit distortions.

We show that for a metric space with an even number of points there is a 1-Lipschitz map to a tree-like space with the same matching number. This result gives the first basic version of an unoriented Kantorovich duality. The study of the duality gives a version of global calibrations for 1-chains with coefficients in $…

Proves depth 2 neural networks can't approximate certain functions as well as depth 3 networks.

New neural network architecture with height adds expressive power.

We study the homeomorphic extension of biholomorphisms between convex domains in $\mathbb C^d$ without boundary regularity and boundedness assumptions. Our approach relies on methods from coarse geometry, namely the correspondence between the Gromov boundary and the topological boundaries of the domains and the dynamic…

Let $X$ and $Y$ be length metric spaces. Let $\mathcal H^n$ denote the $n$-dimensional Hausdorff measure. The Lipschitz-Volume Rigidity is a property that if there exists a 1-Lipschitz map $f\colon X\to Y$ and $0<\mathcal H^n(X)=\mathcal H^n(f(X))<\infty$, then $f$ preserves the length of path. This property holds for …

Two-layer neural networks need more neurons to be robust.

New framework enhances neural network robustness against adversarial attacks.

Deep networks improve by progressively refining approximations at each layer.

We prove a Lipschitz-Volume rigidity theorem in Alexandrov geometry, that is, if a 1-Lipschitz map $f\colon X=\amalg X_\ell\to Y$ between Alexandrov spaces preserves volume, then it is a path isometry and an isometry when restricted to the interior of $X$. We furthermore characterize the metric structure on $Y$ with re…

Proves a new law of robustness for interpolating arbitrary data distributions.

Researchers found counterexamples to conjectures about optimal transport maps on curved spaces.

Lipschitz maps on metric surfaces are rigid if they preserve area.

We study time-like hypersurfaces with vanishing mean curvature in the (3+1) dimensional Minkowski space, which are the hyperbolic counterparts to minimal embeddings of Riemannian manifolds. The catenoid is a stationary solution of the associated Cauchy problem. This solution is linearly unstable, and we show that this …

In this paper, we study the convergence of generative adversarial networks (GANs) from the perspective of the informativeness of the gradient of the optimal discriminative function. We show that GANs without restriction on the discriminative function space commonly suffer from the problem that the gradient produced by …

New memory-query tradeoffs for convex optimization algorithms.

The degree condition affects the rigidity of maps between manifolds.

Memory-constrained algorithms need superlinear memory for efficient convex optimization.

This work analyzes how deep neural networks' expressiveness increases with depth and width.

The measure concentration property of an mm-space $X$ is roughly described as that any 1-Lipschitz map on $X$ to a metric space $Y$ is almost close to a constant map. The target space $Y$ is called the screen. The case of $Y=\mathbb{R}$ is widely studied in many literature (see \cite{gromov}, \cite{ledoux}, \cite{mil2}…

The present paper is composed of two parts. In the first one we define two pseudo-metrics $L_F$ and $K_F$ on the Teichmuüller space of semi-translation surfaces $\mathcal{TQ}_g(\underline k,ε)$, which are the symmetric counterparts to the metrics defined by William Thurston on $\mathcal{T}_g^n$. We prove some nice prop…

Unified plug-in approach for estimating symmetric properties of distributions efficiently.

The paper extends localisation technique to multiple constraints in Euclidean spaces.

Two-layer neural networks must be robust, even with arbitrary weights.

A deep learning approach solves probabilistic inverse problems with physical constraints.