Orthogonium offers unified, efficient layers for robust deep learning.
problem Fragmented and computationally demanding implementations of orthogonal and 1-Lipschitz layers.
method Unified, efficient PyTorch library providing orthogonal and 1-Lipschitz layers.
result Reduced overhead and standardized tools for robust experimentation.
This paper examines weight initialization for 1-Lipschitz networks to improve robustness against adversarial attacks.
problem Improving the robustness of deep neural networks against adversarial attacks.
method Examined weight parametrization of AOL and SLL networks, calculated weight variance bounds, and demonstrated weight decay.
result Weight initialization causes deep 1-Lipschitz networks to decay to zero, and weight variance does not affect output variance distribution.
LOT improves adversarial robustness by training 1-Lipschitz convolution layers.
problem Improving adversarial robustness of deep neural networks.
method LOT: Layer-wise Orthogonal Training for 1-Lipschitz convolution layers.
result LOT significantly enhances certified robustness of Lipschitz-bounded models.
Maps preserving mass and injective on boundary are isometries.
problem Stability of mass-preserving maps in integral current spaces.
method Proving rigidity of mass-preserving 1-Lipschitz maps.
result Maps preserving mass and injective on boundary are isometries.
1-Lipschitz networks are as accurate as classical networks and offer robustness.
problem Misconceptions about 1-Lipschitz neural networks and their properties.
method Analysis of 1-Lipschitz neural networks' accuracy, robustness, and generalization.
result 1-Lipschitz neural networks are as accurate as classical networks and can fit arbitrarily difficult boundaries.
In this paper, we study the Lévy-Milman concentration phenomenon of 1-Lipschitz maps into infinite dimensional metric spaces. Our main theorem asserts that the concentration to an infinite dimensional ℓp-ball with the ℓq-distance function for 1≤p<q≤+∞ is equivalent to the concentration to the…
Existing depth separation results for constant-depth networks essentially show that certain radial functions in Rd, which can be easily approximated with depth 3 networks, cannot be approximated by depth 2 networks, even up to constant accuracy, unless their size is exponential in d. However, the func…
We prove that if a geodesic metric measure space satisfies a comparison condition for isoperimetric profile and if the observable variance is maximal, then the space is foliated by minimal geodesics, where the observable variance is defined to be the supremum of the variance of 1-Lipschitz functions on the space. Our r…
1-Lipschitz neural networks produce clearer, more focused Saliency Maps for explainable AI.
problem Noisy and limited Saliency Maps from traditional neural networks.
method Dual loss of optimal transport problem for 1-Lipschitz neural networks.
result Saliency Maps from 1-Lipschitz networks are highly concentrated and less noisy, aligning with human explanations.
Two-layer neural networks need more neurons to be robust.
problem Understanding the robustness of two-layer neural networks and the role of overparametrization.
method Investigation of the tradeoffs between network size and robustness, using Lipschitz constant as a measure.
result A conjecture that robustness requires overparametrization, with precise bounds for different cases.
Training neural networks under a strict Lipschitz constraint is useful for provable adversarial robustness, generalization bounds, interpretable gradients, and Wasserstein distance estimation. By the composition property of Lipschitz functions, it suffices to ensure that each individual affine transformation or nonline…
Deep networks improve by progressively refining approximations at each layer.
problem Standard approximation theory doesn't explain the role of intermediate layers in deep neural networks.
method Developed a mixed-activation architecture with a geometric scale interpretation of depth.
result Each intermediate layer approximates the target function with a geometric rate.
In this paper, we consider a concentration of measure problem on Riemannian manifolds with boundary. We study concentration phenomena of non-negative 1-Lipschitz functions with Dirichlet boundary condition around zero, which is called boundary concentration phenomena. We first examine relation between boundary concen…
We know SGAN may have a risk of gradient vanishing. A significant improvement is WGAN, with the help of 1-Lipschitz constraint on discriminator to prevent from gradient vanishing. Is there any GAN having no gradient vanishing and no 1-Lipschitz constraint on discriminator? We do find one, called GAN-QP. To construct a …
Optimizes optimal transport distances using low-dimensional embeddings.
problem High computational cost of optimal transport distances in high dimensions.
method Approximate OT distances using 1-Lipschitz maps in a lower-dimensional space.
result Efficiently approximates optimal transport distances with lower computational cost.
The Nash-Kuiper Theorem states that the collection of C1-isometric embeddings from a Riemannian manifold Mn into EN is C0-dense within the collection of all smooth 1-Lipschitz embeddings provided that n<N. This result is now known to be a consequence of Gromov's more general h-principle. Ther…
Two-layer neural networks must be robust, even with arbitrary weights.
problem Proving the robustness of two-layer neural networks with arbitrary weights.
method Developed a new function-space covering method to prove the robustness law, replacing parameter-space covering.
result Proved the conjectured law for two-layer networks with arbitrary real weights, biases, and affine skip connections.
Maps between certain Lipschitz manifolds are isometries if they preserve volume.
problem Volume preservation and isometry conditions for Lipschitz manifolds.
method Volume-preserving 1-Lipschitz maps from integral currents onto infinitesimally Euclidean Lipschitz manifolds.
result Volume-preserving maps are isometries under given conditions.
New neural network design resists small ℓ∞-norm adversarial perturbations.
problem Vulnerability of neural networks to small ℓ∞-norm adversarial perturbations. method Designing ℓ∞-dist neurons and constructing ℓ∞-dist nets, proving their 1-Lipschitz property and expressive power. result Certified robustness of ℓ∞-dist nets with state-of-the-art performance on various datasets. Framework for designing nonlinearities in neural networks with slope constraints.
problem Designing nonlinearities with specific properties for signal processing.
method Variational framework with regularization for slope constraints and optimization of adaptive splines.
result Adaptive nonuniform linear splines achieve global optimum in constrained optimization.
For a given 1-Lipschitz map u:Rn→Rm we define a partition, up to a set of Lebesgue measure zero, of Rn into maximal closed convex sets such that restriction of u is an isometry on these sets. We consider a disintegration, with respect to this partition, of a log-concave meas…
The paper explores theoretical insights into WGANs for better understanding and stability.
problem Stabilizing the training process of GANs.
method Theoretical analysis and statistical convergence study of WGANs.
result Theoretical properties and convergence of WGANs are clarified.
Proves rigidity for maps between manifolds using degree theory and current developments.
problem Lipschitz-volume rigidity for maps between metric manifolds and Riemannian manifolds.
method Degree theory and recent developments of Lipschitz-volume rigidity for integral currents.
result Proves a Lipschitz-volume rigidity result for 1-Lipschitz maps.
Proves a new law of robustness for interpolating arbitrary data distributions.
problem Understanding robust interpolation for arbitrary data distributions.
method Proves a Lipschitzness lower bound for robust interpolation.
result Demonstrates a two-fold law of robustness for interpolating functions.
Maps persistence diagrams into Hilbert and Euclidean spaces with explicit distortions.
problem Embedding persistence diagrams into Euclidean spaces for statistical analysis.
method Explicit geometric maps with distortion functions.
result Controlled geometric information loss through explicit distortion functions.
We show that for a metric space with an even number of points there is a 1-Lipschitz map to a tree-like space with the same matching number. This result gives the first basic version of an unoriented Kantorovich duality. The study of the duality gives a version of global calibrations for 1-chains with coefficients in $…
We study the homeomorphic extension of biholomorphisms between convex domains in Cd without boundary regularity and boundedness assumptions. Our approach relies on methods from coarse geometry, namely the correspondence between the Gromov boundary and the topological boundaries of the domains and the dynamic…
Let X and Y be length metric spaces. Let Hn denote the n-dimensional Hausdorff measure. The Lipschitz-Volume Rigidity is a property that if there exists a 1-Lipschitz map f:X→Y and 0<Hn(X)=Hn(f(X))<∞, then f preserves the length of path. This property holds for …
We prove a Lipschitz-Volume rigidity theorem in Alexandrov geometry, that is, if a 1-Lipschitz map f:X=⨿Xℓ→Y between Alexandrov spaces preserves volume, then it is a path isometry and an isometry when restricted to the interior of X. We furthermore characterize the metric structure on Y with re…
Researchers found counterexamples to conjectures about optimal transport maps on curved spaces.
problem Extending Caffarelli's contraction theorem to curved spaces.
method Constructing counterexamples to precise conjectures.
result Found counterexamples to Milman's conjectures about optimal transport maps on curved spaces.
Lipschitz maps on metric surfaces are rigid if they preserve area.
problem Understanding the rigidity of Lipschitz maps on metric surfaces.
method Established a coarea inequality for continuous Sobolev functions on metric surfaces.
result Proved that 1-Lipschitz maps from a closed metric surface to a closed Riemannian surface preserving area are isometries.
We study time-like hypersurfaces with vanishing mean curvature in the (3+1) dimensional Minkowski space, which are the hyperbolic counterparts to minimal embeddings of Riemannian manifolds. The catenoid is a stationary solution of the associated Cauchy problem. This solution is linearly unstable, and we show that this …
New memory-query tradeoffs for convex optimization algorithms.
problem Optimizing memory usage for convex optimization algorithms.
method Analyzing randomized first-order algorithms for minimizing convex functions.
result Cutting plane methods are optimal in terms of memory and query complexity.
The degree condition affects the rigidity of maps between manifolds.
problem Investigating the degree condition for scalar curvature rigidity.
method Analyzing maps between Riemannian manifolds with scalar curvature constraints.
result The degree condition is necessary for scalar curvature rigidity but not for Ricci curvature rigidity.
Memory-constrained algorithms need superlinear memory for efficient convex optimization.
problem Efficiently minimizing convex functions with limited memory.
method Analyzing first-order algorithms with superlinear memory constraints.
result Superlinear memory is necessary for optimal performance in convex optimization.
The measure concentration property of an mm-space X is roughly described as that any 1-Lipschitz map on X to a metric space Y is almost close to a constant map. The target space Y is called the screen. The case of Y=R is widely studied in many literature (see \cite{gromov}, \cite{ledoux}, \cite{mil2}…
The present paper is composed of two parts. In the first one we define two pseudo-metrics LF and KF on the Teichmuüller space of semi-translation surfaces TQg(k,ε), which are the symmetric counterparts to the metrics defined by William Thurston on Tgn. We prove some nice prop…
Unified plug-in approach for estimating symmetric properties of distributions efficiently.
problem Estimating symmetric properties of distributions with high accuracy and efficiency.
method Profile-maximum-likelihood (PML) based estimator.
result Achieves theoretical limit for universal symmetric property estimation.
Generative adversarial networks (GANs) are one of the most popular approaches when it comes to training generative models, among which variants of Wasserstein GANs are considered superior to the standard GAN formulation in terms of learning stability and sample quality. However, Wasserstein GANs require the critic to b…
JacNet learns Jacobians to enforce structure on derivatives for invertibility and Lipschitz functions.
problem Enforcing structure on derivatives of neural network mappings.
method Proposes using a neural network to directly learn the Jacobian of the input-output function, allowing control over derivative structure.
result Demonstrates learning invertible approximations to simple and 1-Lipschitz functions.
The paper extends localisation technique to multiple constraints in Euclidean spaces.
problem Proving log-concavity of conditional measures in decomposed convex sets.
method Defining partitions of maximal closed convex sets and proving log-concavity of conditional measures.
result Existence of a partition and log-concavity of conditional measures for almost every set of the partition.
This thesis uses Kantorovich-Rubinstein distance for classifying points based on their measures.
problem Classifying points based on their measures in a metric space.
method Using Kantorovich-Rubinstein distance as a metric in the space of measures to capture geometry and topology.
result A large Kantorovich-Rubinstein distance indicates the existence of a 1-Lipschitz classifier that well classifies the points.
Hyperbolic space outperforms Euclidean in learning hierarchical data.
problem Learning hierarchical data in Euclidean space requires exponentially many samples.
method Established geometric obstruction in Euclidean space and showed hyperbolic space's advantage.
result Hyperbolic space enables learning with O(mRlogm) samples, matching information-theoretic optimum. Despite being impactful on a variety of problems and applications, the generative adversarial nets (GANs) are remarkably difficult to train. This issue is formally analyzed by \cite{arjovsky2017towards}, who also propose an alternative direction to avoid the caveats in the minmax two-player training of GANs. The corres…
Reduced sample complexity for group-invariant distributions.
problem Improving sample complexity for estimating divergences of group-invariant distributions.
method Quantified reduction in sample complexity for Wasserstein-1 metric and Lipschitz-regularized α-divergences under finite and infinite groups.
result Sample complexity reduction proportional to group size for finite groups, and convergence rate depends on intrinsic dimension for infinite groups.
A map between manifolds is an isometry if it's Lipschitz and scalar curvature bounded.
problem Characterizing maps between manifolds based on their scalar curvature and Lipschitz continuity.
method Spectral properties of Dirac operators and index theory for low regularity metrics and bundles.
result A 1-Lipschitz map between manifolds is an isometry if it has bounded scalar curvature.
In this paper, we study the convergence of generative adversarial networks (GANs) from the perspective of the informativeness of the gradient of the optimal discriminative function. We show that GANs without restriction on the discriminative function space commonly suffer from the problem that the gradient produced by …
The paper proves manifold rigidity for specific scalar curvature conditions.
problem Proving rigidity of manifolds with positive scalar curvature.
method Using maps to model spaces and degree theory.
result Compact manifolds with specific scalar curvature conditions are locally isometric.