Constructs a map with prescribed local Lipschitz constants on a subset of a manifold.
problem Creating a Lipschitz map with specific local Lipschitz constants on a subset of a manifold.
method Constructs a Lipschitz map that matches a given map on a subset and has a local Lipschitz constant defined by a continuous function.
result A Lipschitz map can be constructed with a local Lipschitz constant prescribed by a continuous function.
We compute the local Lipschitz constant of ReLU networks precisely.
problem Estimating the local Lipschitz constant of ReLU networks is hard.
method We use a novel approach involving the generalized Jacobian and backpropagation.
result We provide an algorithm to compute the exact Lipschitz constant of ReLU networks.
Extends Lipschitz functions while preserving local constants.
problem Extending Lipschitz functions on metric spaces while maintaining local constants.
method Extends Lipschitz functions on metric spaces while locally preserving the asymptotic Lipschitz constant.
result Sobolev spaces on metric measure spaces are invariant under isomorphism of mm-structures.
The paper calculates upper bounds on ReLU network Lipschitz constants.
problem Determining the maximum perturbation size for robustness of neural networks.
method Analyzing ReLU, affine-ReLU, and max pooling functions; combining results; tracking zero elements; using a computational approach.
result The method produces the largest known bounds on minimum adversarial perturbations for large networks.
This paper bounds the Lipschitz constants of neural networks and their gradients.
problem Estimating the Lipschitz constant of complex models like neural networks.
method Local upper and lower bounds on Lipschitz constants computed with respect to network parameters.
result It is impossible to derive global upper bounds for the Lipschitz constants of neural networks.
The paper calculates bounds on the local Lipschitz constants of neural network layers.
problem Understanding the Lipschitz constants of neural network layers for robustness analysis.
method Analytical approach to determine upper bounds on local Lipschitz constants of affine-ReLU functions.
result The method produces tighter bounds than the standard conservative bound, especially for small perturbations.
HALO uses local Lipschitz constants to optimize functions efficiently.
problem Efficiently solving global optimization problems with complex objective functions.
method Hybrid Adaptive Lipschizian Optimization (HALO) algorithm that estimates local Lipschitz constants and balances global and local information.
result HALO outperforms other global optimization algorithms on numerous test functions.
Efficient local Lipschitz bounds improve neural network robustness.
problem Certifying robustness of neural networks is challenging and often leads to over-regularization.
method Proposes an efficient trainable local Lipschitz upper bound by considering activation functions and weight matrices.
result Consistently outperforms state-of-the-art methods in clean and certified accuracy on various datasets.
Characterizes hypergenerated stratified groups with flat boundaries.
problem Characterizing stratified groups with flat boundaries.
method Algebraic characterization and embedding analysis.
result Hypergenerated groups have locally bi-Lipschitz embeddings of non-characteristic hypersurfaces.
The paper extends a measure preserving property to bi-Lipschitz maps between Moran sets.
problem Extending the measure preserving property to Moran sets.
method Analyzing bi-Lipschitz maps between Moran sets.
result Bi-Lipschitz maps between Moran sets preserve measure locally.
Solutions to a specific problem are shown to be locally Lipschitz but not differentiable.
problem Locally Lipschitz viscosity solutions to the σk-Loewner-Nirenberg problem on annuli. method Analytical proof of regularity and non-differentiability.
result Solutions are $C^{1, rac{1}{k}}_{
m loc}$ in each of the annulus regions and have a jump in radial derivative.
The Jacobian matrix (or the gradient for single-output networks) is directly related to many important properties of neural networks, such as the function landscape, stationary points, (local) Lipschitz constants and robustness to adversarial attacks. In this paper, we propose a recursive algorithm, RecurJac, to comput…
New Dynkin condition for manifolds with boundary yields bi-Lipschitz equivalence and spectral properties.
problem Characterizing smooth Riemannian manifolds with boundary.
method Introducing a Dynkin-type condition and proving its equivalence to a weighted manifold.
result Bi-Lipschitz equivalence and various spectral properties of manifolds with boundary.
The paper proves Lipschitz continuity of cut times in spacetimes.
problem Lipschitz continuity of cut times in globally hyperbolic spacetimes.
method Adapted Itoh-Tanaka method to Lorentzian setting.
result Lipschitz continuity of cut times with quantitative estimates.
Adaptive sampling improves convergence in heterogeneous distributed optimization.
problem Poor performance of classical SGD and SVRG in heterogeneous distributed settings.
method Adaptive sampling of machines with an adaptive estimate of local Lipschitz constants.
result Significantly accelerates convergence rate from maximum to average Lipschitz constant.
Techniques known as Nonlinear Set Membership prediction, Kinky Inference or Lipschitz Interpolation are fast and numerically robust approaches to nonparametric machine learning that have been proposed to be utilised in the context of system identification and learning-based control. They utilise presupposed Lipschitz p…
New MIP formulations for neural network Lipschitz constant estimation.
problem Ensuring robustness of neural networks by calculating their Lipschitz constant.
method Reformulating the neural network Lipschitz estimation problem as a Quadratically Constrained MIP (MIQCQP) problem.
result Solutions of the MIQCQP formulations provide bounds on the Lipschitz constant, with conditions for exactness.
Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.
problem Estimating unknown Lipschitz functions from noisy observations.
method Extends max-affine methods to Lipschitz setting using nonlinear feature expansion and adaptive partitioning.
result Achieves minimax convergence rate with respect to intrinsic dimension, up to logarithmic factors.
Paper investigates Lipschitz constants of self-attention modules in neural networks.
problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.
New metric measure space theory for Lipschitz constants.
problem Defining and characterizing Cheeger energy in metric measure spaces.
method Adapting Cheeger theory to intrinsically Lipschitz sections.
result Characterization of intrinsic Cheeger energy in terms of relaxed slope.
Neural networks are dense among Lipschitz functions with fixed Lipschitz constant.
problem Characterizing neural network approximations to Lipschitz functions.
method Analyzing L-Lipschitz neural networks and their density in L-Lipschitz functions. result One layer neural networks are dense in the set of all L-Lipschitz functions. The paper studies properties of intrinsically Lipschitz constants in metric spaces.
problem Investigating properties of intrinsically Lipschitz constants.
method Introduced Leibniz and product formulas for intrinsic slope.
result Formulated Leibniz and product formulas for intrinsic slope.
The paper sharpens a theorem about surfaces with zero Gaussian curvature.
problem Quantifying the isometric property of surfaces with zero Gaussian curvature.
method Asymptotically sharp quantitative version of a classical theorem using isothermal coordinates.
result An isothermal coordinate map from a Riemannian disc to an Euclidean disc is bi-Lipschitz with a constant of exp(4ε).
Several recent papers have discussed utilizing Lipschitz constants to limit the susceptibility of neural networks to adversarial examples. We analyze recently proposed methods for computing the Lipschitz constant. We show that the Lipschitz constant may indeed enable adversarially robust neural networks. However, the m…
This paper analyzes the Lipschitz constants of deep neural networks with random weights.
problem Estimating the Lipschitz constants of deep neural networks with random parameters.
method High probability upper and lower bounds derived for ReLU neural networks with He initialization.
result The behavior of the Lipschitz constant varies significantly between p∈[1,2) and p∈[2,∞]. This paper studies bounds for the Lipschitz constant of random neural networks.
problem Quantifying the worst-case robustness of neural networks against adversarial perturbations.
method Analyzes upper and lower bounds for the Lipschitz constant of random ReLU neural networks under specific initialization conditions.
result For deep networks, the upper bound is larger than the lower bound by a logarithmic factor in width.
This work focuses on important step in quantitative topology: given homotopic mappings from Sm to Sn of Lipschitz constant L, build the (asymptotically) simplest homotopy between them (meaning having the least Lipschitz constant). The present paper resolves this problem for the first case where Hopf invariant p…
The paper proposes a method to train NNs with a small Lipschitz constant to improve robustness.
problem Neural networks' susceptibility to adversarial perturbations in safety-critical applications.
method The paper introduces a framework to train multi-layer NNs by minimizing their Lipschitz constant, using an optimization scheme based on the Alternating Direction Method of Multipliers.
result The proposed training procedure successfully increases the robustness of neural networks.
LiPopt uses polynomial optimization to estimate neural network Lipschitz constants efficiently.
problem Estimating the Lipschitz constant of neural networks efficiently.
method Sparse polynomial optimization, leveraging network connectivity to reduce complexity.
result Superior estimates of the ℓ∞-Lipschitz constant compared to existing methods. New method for differentially private optimization with general Lipschitz conditions.
problem Differentially private optimization under general Lipschitz conditions.
method Generalized Lipschitz condition for per-sample gradients, tuning clip norm based on minimum per-sample Lipschitz constant.
result Efficacy of the recommended clip norm tuning method verified on 8 datasets.
CLIP controls neural network stability by bounding Lipschitz constants.
problem Neural networks lack mathematical guarantees of stability, especially to adversarial examples.
method Develops a variational regularization method (CLIP) to control the Lipschitz constant of neural networks.
result CLIP provides a tighter bound on the actual Lipschitz constant compared to layer-wise methods.
The paper studies Lipschitz bounds for integral kernels under differentiability assumptions.
problem Understanding the Lipschitz continuity of feature maps associated with integral kernels.
method Analyzes differentiability assumptions to derive explicit formulas for Lipschitz constants and conditions for non-Lipschitz continuity.
result Explicit formulas and conditions for Lipschitz continuity of feature maps associated with various kernels.
We determine the asymptotic behavior of the optimal Lipschitz constant for the systole map from Teichmuller space to the curve complex.
In this note we prove that reconstruction from magnitudes of frame coefficients (the so called "phase retrieval problem") can be performed using Lipschitz continuous maps. Specifically we show that when the nonlinear analysis map α:H→Rm is injective, with (α(x))k=∣<x,fk>∣2, where $…
Let ρ:G→O(V) be a real finite dimensional orthogonal representation of a compact Lie group, let σ=(σ1,…,σn):V→Rn, where σ1,…,σn form a minimal system of homogeneous generators of the G-invariant polynomials on V, and set $d = \max_i \operatorname{deg} …
Scalable method bounds Lipschitz constant of generative models.
problem Bounding the Lipschitz constant of generative models.
method Layerwise convex approximations using zonotopes.
result Efficient and tight bounds on generative models.
Efficiently learns Single-Index Models with constant factor approximation.
problem Learning Single-Index Models under L22 loss with unknown link functions. method An efficient algorithm using alignment sharpness for optimization.
result Achieves constant factor approximation to optimal loss for various distributions and link functions.
Paper proves convergence for private FL on non-Lipschitz convex objectives using normalization instead of clipping.
problem Lack of convergence results for differentially private federated learning with non-Lipschitz objectives.
method Developed a convergence result for private FL on smooth convex objectives without assuming Lipschitzness, using normalization instead of clipping.
result Normalization-based private FL algorithm converges better than clipping-based counterpart on smooth convex functions.
Improved robustness of machine learning models with controlled Lipschitz constants.
problem Vulnerability of state-of-the-art models to adversarial attacks.
method Proposes a CLL loss that calibrates the margin and Lipschitz constant penalties, improving robustness certificates.
result Consistently outperforms other losses on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets.
Optimistic method adapted for faster convex-concave min-max problems.
problem Solving convex-concave min-max optimization problems efficiently.
method Adaptive, line search-free second-order methods combining optimistic updates and second-order information.
result Achieves optimal convergence rate without line search or backtracking.
Neural operators learn to solve LQ MFGs efficiently in infinite dimensions.
problem Solving many related LQ MFG problems in infinite-dimensional settings.
method Training neural operators to map problem data to equilibrium strategies.
result NOs reliably solve unseen LQ MFG variants with controlled parameters.
Square inscribed in a curve made of two graph functions.
problem Finding inscribed squares in curves formed by graph functions.
method Analysis of spectral invariants of Jordan Floer homology under curve perturbations.
result Existence of inscribed squares in curves with specific Lipschitz constants.
ECP optimizes expensive functions without knowing Lipschitz constant.
problem Optimizing expensive, non-convex functions with unknown Lipschitz constants.
method ECP minimizes evaluations by focusing on potentially optimal regions, eliminating Lipschitz constant estimation.
result Guaranteed no-regret performance and minimax-optimal regret bounds.
The asymptotic Plateau problem asks for the existence of smooth complete hypersurfaces of constant mean curvature with prescribed asymptotic boundary at infinity in the hyperbolic space Hn+1. The modified mean curvature flow (MMCF) was firstly introduced by Xiao and the second author a few years back, and…
Guarantees uniform convergence for square-root Lipschitz losses.
problem Uniform convergence guarantees for square-root Lipschitz losses.
method Using Rademacher complexity and square root of scalar loss function Lipschitz constant.
result Generalizes previous results and handles non-smooth loss functions.
Tight estimation of the Lipschitz constant for deep neural networks (DNNs) is useful in many applications ranging from robustness certification of classifiers to stability analysis of closed-loop systems with reinforcement learning controllers. Existing methods in the literature for estimating the Lipschitz constant su…
We develop a method to learn neural network activations with controlled Lipschitz constant.
problem Increase neural network capacity while controlling Lipschitz constant.
method Variational framework to learn activation functions with piecewise-linear constraints.
result Proves existence of solutions with continuous and piecewise-linear activations.
In this paper we consider Llarull's theorem in the foliation case and get a lower bound of the Lipschitz constant of the map M→Sn in the foliation case under the spin condition.