New MIP formulations for neural network Lipschitz constant estimation.
problem Ensuring robustness of neural networks by calculating their Lipschitz constant.
method Reformulating the neural network Lipschitz estimation problem as a Quadratically Constrained MIP (MIQCQP) problem.
result Solutions of the MIQCQP formulations provide bounds on the Lipschitz constant, with conditions for exactness.
We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipschitz models where we quantify the error using the Wasserstein metric. We go on to prove an error bound for the value-function estimate arisi…
Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.
problem Estimating unknown Lipschitz functions from noisy observations.
method Extends max-affine methods to Lipschitz setting using nonlinear feature expansion and adaptive partitioning.
result Achieves minimax convergence rate with respect to intrinsic dimension, up to logarithmic factors.
Method approximates Lipschitz domains with smoother shapes.
problem Approximating bounded Lipschitz domains.
method Sequence of smooth, bounded domains with weak curvatures.
result Uniform isocapacitary estimates for approximating sets.
We compute the local Lipschitz constant of ReLU networks precisely.
problem Estimating the local Lipschitz constant of ReLU networks is hard.
method We use a novel approach involving the generalized Jacobian and backpropagation.
result We provide an algorithm to compute the exact Lipschitz constant of ReLU networks.
New scalable Lipschitz bounds improve neural network robustness analysis.
problem Computing tight Lipschitz bounds for deep neural networks is challenging and computationally expensive.
method Derived new closed-form Lipschitz bounds using more general feasible points of LipSDP, avoiding SDP solvers.
result Improved scalability and precision of Lipschitz estimation for large neural networks.
LiPopt uses polynomial optimization to estimate neural network Lipschitz constants efficiently.
problem Estimating the Lipschitz constant of neural networks efficiently.
method Sparse polynomial optimization, leveraging network connectivity to reduce complexity.
result Superior estimates of the ℓ∞-Lipschitz constant compared to existing methods. Tight estimation of the Lipschitz constant for deep neural networks (DNNs) is useful in many applications ranging from robustness certification of classifiers to stability analysis of closed-loop systems with reinforcement learning controllers. Existing methods in the literature for estimating the Lipschitz constant su…
Sharp Lipschitz bounds and gradient estimates for fully nonlinear parabolic equations.
problem Understanding moduli of continuity for fully nonlinear parabolic equations.
method Proving moduli of continuity of viscosity solutions are subsolutions of one-dimensional parabolic equations.
result Sharp Lipschitz bounds and gradient estimates for fully nonlinear parabolic equations with bounded initial data.
The paper examines partial regularity of Lipschitz solutions to minimal surface system.
problem Understanding the regularity of solutions to the minimal surface system.
method Investigation of stationary, integral weak, and viscosity solutions; interior gradient estimate using maximum principle.
result Partial regularity results for Lipschitz solutions, including interior gradient estimate.
The study provides optimal estimates for surfaces close to constant mean curvature.
problem Optimizing estimates for surfaces near constant mean curvature.
method Bi-Lipschitz and W2,2 parametrization for surfaces with density close to one and small Willmore energy. result Quantitative rigidity for L2-almost CMC surfaces. Paper introduces a neural network for consistent estimation of optimal transport maps.
problem Statistically consistent estimation of optimal transport maps between probability distributions.
method Lipschitz-constrained GAN penalized by quadratic transportation cost.
result The generator converges uniformly to the optimal transport map as sample size increases.
New estimates for nodal and singular sets of parabolic inequalities.
problem Understanding the structure of nodal and singular sets in parabolic inequalities.
method Establishing new estimates for the size and structure of nodal and singular sets using parabolic Lipschitz coefficients.
result Almost all nodal and singular sets are covered by regular parabolic Lipschitz graphs with estimates.
Training neural networks under a strict Lipschitz constraint is useful for provable adversarial robustness, generalization bounds, interpretable gradients, and Wasserstein distance estimation. By the composition property of Lipschitz functions, it suffices to ensure that each individual affine transformation or nonline…
A Lipschitz hypersurface is a hypersurface which locally is the graph of a Lipschitz function. A Lipschitz (or C^1) hypersurface is said to be Levi-flat if it is locally foliated by complex manifolds of complex dimension (n-1). We shall prove that there exist no Lipschitz Levi-flat hypersurfaces in CP^n with n >= 3. Ou…
Bayesian optimization and Lipschitz optimization have developed alternative techniques for optimizing black-box functions. They each exploit a different form of prior about the function. In this work, we explore strategies to combine these techniques for better global optimization. In particular, we propose ways to use…
This is the first paper of two ones. Here we prove that two compact Alexandrov surfaces of bounded integral curvature having no peak points are bi-Lipschitz equivalent if they are homeomorphic one to the other. Also conditions under that two ends having finite integral negative curvature are bi-Lipschitz equivalent are…
Optimizes Lipschitz estimates for partitions of unity and characterizes spaces with Assouad-Nagata dimension.
problem Understanding the properties of partitions of unity and their Lipschitz bounds.
method Analyzes the standard partition of unity and its ℓp-generalizations, using the approximate midpoint property and Lebesgue number. result Optimal Lipschitz bounds for partitions of unity and characterizes metric spaces with Assouad-Nagata dimension.
Sparse Polyak improves high-dimensional statistical estimation.
problem High-dimensional statistical estimation problems with growing problem dimension.
method Sparse Polyak modifies Polyak's adaptive step size to estimate restricted Lipschitz smoothness.
result Sparse Polyak achieves optimal statistical precision with fewer iterations.
Enhances ENet's prediction accuracy while maintaining uncertainty estimation.
problem Gradient shrinkage problem in ENet's loss function.
method Proposes a multi-task learning framework with a modified MSE loss function.
result Improves ENet's prediction accuracy without losing uncertainty estimation.
Paper introduces risk assessment for contextual bandits without experiments.
problem Evaluate policies using logged data in context bandits.
method Lipschitz risk functionals and Off-Policy Risk Assessment (OPRA) framework.
result OPRA provides finite sample guarantees for various risk estimates.
Lipschitz constraints under L2 norm on deep neural networks are useful for provable adversarial robustness bounds, stable training, and Wasserstein distance estimation. While heuristic approaches such as the gradient penalty have seen much practical success, it is challenging to achieve similar practical performance wh…
Extends involutivity to non-Lipschitz subbundles and proves the Frobenius Theorem.
problem Defining involutivity for non-Lipschitz subbundles and proving the Frobenius Theorem.
method Using generalized functions, the Frobenius Theorem is extended to log-Lipschitz subbundles with sharp regularity estimates.
result For log-Lipschitz involutive subbundles, there exists a homeomorphism with specific regularity properties.
Study on residual Monge-Ampère mass of complex functions with directional Lipschitz continuity.
problem Estimating the residual Monge-Ampère mass of plurisubharmonic functions.
method General decomposition formula under Sasakian structure, L1-apriori estimate, upper-bound estimate on residual mass. result Upper-bound estimate on residual mass for uniformly directional Lipschitz continuity confirmed.
Scalable method bounds Lipschitz constant of generative models.
problem Bounding the Lipschitz constant of generative models.
method Layerwise convex approximations using zonotopes.
result Efficient and tight bounds on generative models.
Develops new strategy for Hessian estimates in Lagrangian mean curvature equation.
problem Interior Hessian estimates for solutions with prescribed Lipschitz phases.
method Allard-type regularity theorem, geometric measure theory, geometry of Lagrangian graphs, De Giorgi-Nash-Moser iteration.
result Sharp interior Hessian estimates for solutions with critical and supercritical phases.
Proposes a method for obtaining interval bounds in off-policy evaluation.
problem Provides provably correct upper and lower bounds for off-policy evaluation.
method Searches for the maximum and minimum values of the expected reward among Lipschitz Q-functions.
result Introduces a Lipschitz value iteration method to monotonically tighten interval bounds.
Paper investigates Lipschitz constants of self-attention modules in neural networks.
problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.
The paper proves Lipschitz continuity of cut times in spacetimes.
problem Lipschitz continuity of cut times in globally hyperbolic spacetimes.
method Adapted Itoh-Tanaka method to Lorentzian setting.
result Lipschitz continuity of cut times with quantitative estimates.
i-DenseNets improve parameter efficiency and performance in density estimation.
problem Improving parameter efficiency and performance in density estimation models.
method Invertible Dense Networks (i-DenseNets) with learnable weighted concatenation and Concatenated LipSwish activation function.
result i-DenseNets outperform Residual Flows and other flow-based models in bits per dimension.
The paper proves properties of Lipschitz spacetimes with bounded Ricci curvature.
problem Properties of Lipschitz spacetimes with bounded Ricci curvature.
method Globally hyperbolic spacetimes with locally Lipschitz metrics and timelike Ricci curvature.
result New comparison theorems for Lipschitz spacetimes.
Estimates hypergraphons for modeling complex interactions efficiently.
problem Modeling higher-order interactions using hypergraphons.
method Restricted class of Simple Lipschitz Hypergraphons (SLH) for efficient estimation.
result Optimal rates of convergence for SLH estimator.
Defines a new modulus for Lipschitz surfaces and proves a homological duality theorem.
problem Lipschitz homology classes and their moduli.
method Defining a new modulus dModp and proving a homological duality theorem. result Every relative Lipschitz k-homology class has a unique dual class satisfying a specific modulus product equality. Researchers develop a new framework to control neural network sensitivity.
problem Understanding and controlling the behavior of neural networks.
method Direct parameterization of bi-Lipschitzness in convex neural networks.
result A clear and tight control of neural network sensitivity achieved.
HALO uses local Lipschitz constants to optimize functions efficiently.
problem Efficiently solving global optimization problems with complex objective functions.
method Hybrid Adaptive Lipschizian Optimization (HALO) algorithm that estimates local Lipschitz constants and balances global and local information.
result HALO outperforms other global optimization algorithms on numerous test functions.
Considering the Teichmüller space of a surface equipped with Thurston's Lipschitz metric, we study geodesic segments whose endpoints have bounded combinatorics. We show that these geodesics are cobounded, and that the closest-point projection to these geodesics is strongly contracting. Consequently, these geodesics are…
Deep neural networks are notorious for being sensitive to small well-chosen perturbations, and estimating the regularity of such architectures is of utmost importance for safe and robust practical applications. In this paper, we investigate one of the key characteristics to assess the regularity of such methods: the Li…
This paper bounds the Lipschitz constants of neural networks and their gradients.
problem Estimating the Lipschitz constant of complex models like neural networks.
method Local upper and lower bounds on Lipschitz constants computed with respect to network parameters.
result It is impossible to derive global upper bounds for the Lipschitz constants of neural networks.
Techniques known as Nonlinear Set Membership prediction, Kinky Inference or Lipschitz Interpolation are fast and numerically robust approaches to nonparametric machine learning that have been proposed to be utilised in the context of system identification and learning-based control. They utilise presupposed Lipschitz p…
We prove 3-dimensional hyperbolic cone-manifolds are geometrically inflexible: a cone-deformation of a hyperbolic cone-manifold determines a bi-Lipschitz diffeomorphism between initial and terminal manifolds in the deformation in the complement of a standard tubular neighborhood of the cone-locus whose pointwise bi-Lip…
This paper analyzes the Lipschitz constants of deep neural networks with random weights.
problem Estimating the Lipschitz constants of deep neural networks with random parameters.
method High probability upper and lower bounds derived for ReLU neural networks with He initialization.
result The behavior of the Lipschitz constant varies significantly between p∈[1,2) and p∈[2,∞]. ECP optimizes expensive functions without knowing Lipschitz constant.
problem Optimizing expensive, non-convex functions with unknown Lipschitz constants.
method ECP minimizes evaluations by focusing on potentially optimal regions, eliminating Lipschitz constant estimation.
result Guaranteed no-regret performance and minimax-optimal regret bounds.
Proves Hawking's theorem for less smooth spacetime metrics.
problem Proving Hawking's singularity theorem for less smooth spacetime metrics.
method New estimates for Ricci curvature and a segment-type inequality for volume control.
result Proves Hawking's singularity theorem for Lipschitz metrics.
Unified high-probability regret bounds for online convex optimisation with randomised gradient estimators.
problem Online convex optimisation with randomised gradient estimators for ℓq-Lipschitz losses. method FTRL with randomised two-point finite-difference gradient estimators based on cone-measure sampling from ℓr-spheres. result Unified high-probability regret bounds for all p,q,r∈[1,∞]. The paper simplifies arguments for stationary varifolds results.
problem Height bound and Lipschitz approximation for stationary varifolds.
method Simpler arguments to obtain height bound and Lipschitz approximation.
result Excess decay as a consequence of height bound and Lipschitz approximation.
New method estimates Riemannian derivatives from noisy function evaluations.
problem Optimizing functions on Riemannian manifolds with noisy data.
method Riemannian Gaussian smoothing for gradient and Hessian estimation.
result Oracle complexity independent of ambient dimension.
In a series of papers, including the present one, we give a new, shorter proof of Almgren's partial regularity theorem for area minimizing currents in a Riemannian manifold, with a slight improvement on the regularity assumption for the latter. This note establishes a new a priori estimate on the excess measure of an a…
Abstract cone operators prove scalar curvature comparisons on singular manifolds.
problem Proving scalar curvature inequalities on manifolds with cone singularities.
method Using index theory for twisted Dirac operators on Lipschitz bundles.
result Lipschitz rigidity for scalar curvature on odd-dimensional manifolds.