arXiv research

New Transformers maintain Lipschitz continuity for robustness.

Proves existence of curved surfaces in hyperbolic space.

The paper proves Lipschitz continuity of cut times in spacetimes.

GroupSort neural networks can approximate Lipschitz continuous functions.

This research explores principles of Lipschitz continuity in neural networks for robustness and generalization.

We show that the isoperimetric profile $h_{g(t)}(ξ)$ of a compact Riemannian manifold $(M,g)$ is jointly continuous when metrics $g(t)$ vary continuously. We also show that, when $M$ is a compact surface and $g(t)$ evolves under normalized Ricci flow, $h^2_{g(t)}(ξ)$ is uniform Lipschitz continuous and hence $h_{g(t)}(…

We model how Lipschitz continuity changes during neural network training.

In binary classification and regression problems, it is well understood that Lipschitz continuity and smoothness of the loss function play key roles in governing generalization error bounds for empirical risk minimization algorithms. In this paper, we show how these two properties affect generalization error bounds in …

Study shows prior Lipschitz continuity can improve adversarial robustness of Bayesian Neural Networks.

Lipschitz continuity recently becomes popular in generative adversarial networks (GANs). It was observed that the Lipschitz regularized discriminator leads to improved training stability and sample quality. The mainstream implementations of Lipschitz continuity include gradient penalty and spectral normalization. In th…

Investigates Lipschitz continuity in neural networks across various settings.

We consider Aubry-Mather theory for a subclass of class A spacetimes, i.e. compact vicious spacetimes with globally hyperbolic Abelian cover. In this subclass, called class A_1, we obtain improved results on timelike maximizers and Lipschitz continuity of the time separation of the Abelian cover on the i.g. optimal sub…

In this note we prove that reconstruction from magnitudes of frame coefficients (the so called "phase retrieval problem") can be performed using Lipschitz continuous maps. Specifically we show that when the nonlinear analysis map $α:{\mathcal H}\rightarrow\mathbb{R}^m$ is injective, with $(α(x))_k=|<x,f_k>|^2$, where $…

In this paper, we investigate the underlying factor that leads to failure and success in the training of GANs. We study the property of the optimal discriminative function and show that in many GANs, the gradient from the optimal discriminative function is not reliable, which turns out to be the fundamental cause of fa…

We study continuous maps between differential manifolds from a microlocal point of view. In particular, we characterize the Lipschitz continuity of these maps in terms of the microsupport of the constant sheaf on their graph. Furthermore, we give lower and upper bounds on the microsupport of the graph of a continuous m…

Study on residual Monge-Ampère mass of complex functions with directional Lipschitz continuity.

Proposes a weaker version of Strong Cosmic Censorship with curvature bounds.

Lipschitz normalization boosts deep attention models, especially for graph neural networks.

For a bounded domain equipped with a piecewise Lipschitz continuous Riemannian metric g, we consider harmonic map from $(Ω, g)$ to a compact Riemannian manifold $(N,h)\subset\mathbb R^k$ without boundary. We generalize the notion of stationary harmonic map and prove the partial regularity. We also discuss the global Li…

The paper studies Lipschitz bounds for integral kernels under differentiability assumptions.

New methods solve MI problems with locally Lipschitz operators, improving solution efficiency.

Local Lipschitz continuity of sub-elliptic harmonic maps into CAT(0) spaces proved.

The paper quantifies the regularity of attention operations.

We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipschitz models where we quantify the error using the Wasserstein metric. We go on to prove an error bound for the value-function estimate arisi…

A scattering transform defines a signal representation which is invariant to translations and Lipschitz continuous relatively to deformations. It is implemented with a non-linear convolution network that iterates over wavelet and modulus operators. Lipschitz continuity locally linearizes deformations. Complex classes o…

New framework tightens certified robustness gaps in machine learning models.

New method improves optimization algorithms without Lipschitz smoothness.

In 1997, J. Jost [27] and F. H. Lin [39], independently proved that every energy minimizing harmonic map from an Alexandrov space with curvature bounded from below to an Alexandrov space with non-positive curvature is locally Hölder continuous. In [39], F. H. Lin proposed a challenge problem: Can the Hölder continuity …

Efficient algorithm for global optimization of multivariate Lipschitz functions.

This paper presents a novel nonmyopic adaptive Gaussian process planning (GPP) framework endowed with a general class of Lipschitz continuous reward functions that can unify some active learning/sensing and Bayesian optimization criteria and offer practitioners some flexibility to specify their desired choices for defi…

Study examines stability of image-reconstruction algorithms using variational regularization.

New algorithms sample from log concave distributions without gradient Lipschitz continuity.

The paper proves deep learning can be robust with certain loss functions.

LALR adapts learning rate for faster convergence in regression and neural nets.

We prove a Frobenius-type theorem for singular distributions generated by a family of locally Lipschitz continuous vector fields satisfying almost everywhere a quantitative finite type condition.

Efficient binary sampling method for global optimization of univariate functions with low regret.

Invertible DenseNets improve model efficiency and performance.

Study shows limitations of Lie bracket commutation for nonsmooth vector fields.

Transfer learning for bandits with latent Lipschitz continuity.

The curse of dimensionality affects neural network optimization, especially with smooth functions.

Unified framework for analyzing graph neural operators converging to graph limits.

New NN design for nonlinear systems control with guarantees.

Improved robustness of 1D CNNs for heart arrhythmia classification.

Privacy affects fairness in classification models, but not drastically.

Verifying correctness of deep neural networks (DNNs) is challenging. We study a generic reachability problem for feed-forward DNNs which, for a given set of inputs to the network and a Lipschitz-continuous function over its outputs, computes the lower and upper bound on the function values. Because the network and the …

We study online optimization of smoothed piecewise constant functions over the domain [0, 1). This is motivated by the problem of adaptively picking parameters of learning algorithms as in the recently introduced framework by Gupta and Roughgarden (2016). Majority of the machine learning literature has focused on Lipsc…

We provide a new local class-purity theorem for Lipschitz continuous DNN classifiers. In addition, we discuss how to achieve classification margin for training samples. Finally, we describe how to compute margin p-values for test samples.

This paper converts ADMM to proximal gradient for efficient sparse estimation.