arXiv research

Deep neural networks help recover two signals from noisy mixtures.

Study robust learning of Lipschitz functions under corrupted binary signals.

This research explores principles of Lipschitz continuity in neural networks for robustness and generalization.

A scattering transform defines a signal representation which is invariant to translations and Lipschitz continuous relatively to deformations. It is implemented with a non-linear convolution network that iterates over wavelet and modulus operators. Lipschitz continuity locally linearizes deformations. Complex classes o…

The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to generalize the concept of Lipschitz smoothness condition to the relative smoothn…

Study Lipschitz regularity for manifold-constrained ROF model on curved surfaces.

In this work we compute lower Lipschitz bounds of $\ell_p$ pooling operators for $p=1, 2, \infty$ as well as $\ell_p$ pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…

We consider deep feedforward neural networks with rectified linear units from a signal processing perspective. In this view, such representations mark the transition from using a single (data-driven) linear representation to utilizing a large collection of affine linear representations tailored to particular regions of…

Paper introduces a new metric to select optimal Graph Shift Operator for GNNs.

Generative model controls heterophily in graph signals.

Unified framework for uniform signal recovery in nonlinear GCS with 1-bit/quantized measurements.

Study identifies three quantization regimes for ReLU networks.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters which are well localized i…

Stable algebraic filters improve neural network performance.

We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a conv…

CGNNs use wavelets for continuous function generation in infinite-dimensional spaces.

The paper develops a robust signal estimation method for noisy measurements from generative models.

Neural networks are dense among Lipschitz functions with fixed Lipschitz constant.

Constructs a map with prescribed local Lipschitz constants on a subset of a manifold.

Deep convolutional neural networks have led to breakthrough results in numerous practical machine learning tasks such as classification of images in the ImageNet data set, control-policy-learning to play Atari games or the board game Go, and image captioning. Many of these applications first perform feature extraction …

Maps between certain Lipschitz manifolds are isometries if they preserve volume.

Near-optimal sample complexity for phase retrieval with generative priors.

We study a sparse negative binomial regression (NBR) for count data by showing the non-asymptotic advantages of using the elastic-net estimator. Two types of oracle inequalities are derived for the NBR's elastic-net estimates by using the Compatibility Factor Condition and the Stabil Condition. The second type of oracl…

This paper recovers smooth functions from noisy modulo samples using a three-stage strategy.

Abstract: Lipschitz homeomorphisms are deformed using Perelman's methods.

First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made---for the continuous-time case---in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathemati…

We compute the local Lipschitz constant of ReLU networks precisely.

The study examines the limitations of bi-Lipschitz Normalizing Flows in approximating certain distributions.

Novikov conjecture reduced to Lipschitz cohomology of groups.

PGD algorithms solve nonlinear inverse problems with generative priors using noisy measurements.

We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipschitz models where we quantify the error using the Wasserstein metric. We go on to prove an error bound for the value-function estimate arisi…

New MIP formulations for neural network Lipschitz constant estimation.

This paper introduces SS-MAMP to address convergence issues in AMP algorithms.

Framework for designing nonlinearities in neural networks with slope constraints.

New method for differentially private optimization with general Lipschitz conditions.

The paper studies properties of intrinsically Lipschitz constants in metric spaces.

New scalable Lipschitz bounds improve neural network robustness analysis.

The paper studies Lipschitz bounds for integral kernels under differentiability assumptions.

We characterize locally Lipschitz mappings and existence of Lipschitz extensions through a first order nonlinear system of PDEs. We extend this study to graded group-valued Lipschitz mappings defined on compact Riemannian manifolds. Through a simple application, we emphasize the connection between these PDEs and the Ru…

Geodesically complete spaces with curvature bounded above have maps with finite energy that are Lipschitz.

We construct a smooth compact n-dimensional manifold Y with one point singularity such that all its Lipschitz homotopy groups are trivial, but Lipschitz mappings Lip(S^n,Y) are not dense in the Sobolev space W^{1,n}(S^n,Y). On the other hand we show that if a metric space Y is Lipschitz (n-1)-connected, then Lipschitz …

Proves rigidity for maps between manifolds using degree theory and current developments.

Paper investigates Lipschitz constants of self-attention modules in neural networks.

The thesis defines and proves invariants for manifolds of bounded geometry.

Defines a new modulus for Lipschitz surfaces and proves a homological duality theorem.

Investigates Lipschitz continuity in neural networks across various settings.

New Transformers maintain Lipschitz continuity for robustness.