Neural networks with low Lipschitz constants can approximate any Lipschitz function.
problem Adversarial attacks on machine learning models.
method Three-layer neural networks with FullSort activation function.
result Three-layer neural networks using FullSort are Universal Lipschitz Approximators.
ELF simplifies normalizing flows, making them more efficient and universal.
problem Computational inefficiency of normalizing flows.
method ELF introduces a simple, one-layer network with closed-form Lipschitz constants, combining the ease of residual flows with the performance of autoregressive flows.
result ELF is a provably universal density approximator, more efficient computationally and parameter-wise.
Residual flows are shown to approximate MMD well.
problem Lack of theoretical understanding of normalizing flows' expressiveness.
method Proved residual flows are universal approximators in MMD.
result Residual flows can approximate MMD with a bounded number of blocks.
Dense neural networks can't approximate all functions.
problem Approximation capabilities of dense neural networks.
method Model compression approach combining weak regularity lemma and graph neural networks.
result Existence of Lipschitz continuous functions not approximable by dense neural networks.
New Transformers maintain Lipschitz continuity for robustness.
problem Ensuring robustness in Transformers for safety-sensitive applications.
method Introducing gradient-descent-type in-context Transformers with explicit Euler steps of negative gradient flows.
result Universal approximation theorem for Lipschitz continuous Transformers.
Neural operators learn to solve LQ MFGs efficiently in infinite dimensions.
problem Solving many related LQ MFG problems in infinite-dimensional settings.
method Training neural operators to map problem data to equilibrium strategies.
result NOs reliably solve unseen LQ MFG variants with controlled parameters.
Bi-Lipschitz flows approximate a wide range of distributions.
problem Characterizing the expressivity of bi-Lipschitz normalizing flows.
method Linking score regularity to transport map bi-Lipschitzness via probability flow ODE.
result Gaussian pullbacks induced by bi-Lipschitz variance-preserving transport maps are L1-dense among all probability densities. A new machine learning model uses matrix exponentials for universal approximation.
problem Developing a robust and efficient machine learning model.
method Introduces a novel architecture using matrix exponentials as the only nonlinearity.
result The model achieves universal approximation properties and outperforms other models on benchmark tasks.
Training neural networks under a strict Lipschitz constraint is useful for provable adversarial robustness, generalization bounds, interpretable gradients, and Wasserstein distance estimation. By the composition property of Lipschitz functions, it suffices to ensure that each individual affine transformation or nonline…
New proof shows incremental flow models are essential for universal generation.
problem Understanding the universality of flow-based models in generating natural maps.
method Topological-dynamical argument and algebraic properties of flows.
result Incremental generation is necessary and sufficient for universal flow-based generation.
Deep neural operators learn complex probabilistic models efficiently.
problem Learning complex probabilistic models with global Lipschitz conditions.
method Deep neural-operator framework under global Lipschitz conditions.
result Explicit network-size bounds for universal approximation of probabilistic models.
Generalised regularisation equals robustness for exotic function classes.
problem Adversarial examples highlight the need for robust models in exotic function classes.
method Equality result linking distributional robustness and Lipschitz regularisation.
result Certified robustness of Lipschitz-regularised models with mild assumptions.
The paper uses neural networks to learn system dynamics from data with Lipschitz regularization.
problem Learning governing equations from time-sampled data.
method Lipschitz regularized deep neural networks for ODE system identification.
result Lipschitz regularization improves the smoothness and generalization of the learned function.
Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.
problem Estimating unknown Lipschitz functions from noisy observations.
method Extends max-affine methods to Lipschitz setting using nonlinear feature expansion and adaptive partitioning.
result Achieves minimax convergence rate with respect to intrinsic dimension, up to logarithmic factors.
The study examines Lipschitz normally embedded Hölder triangles in 4D space.
problem Comparing ambient and outer Lipschitz geometry of Hölder triangles.
method Analyzes Lipschitz normally embedded Hölder triangles in \(\mathbb{R}^4\).
result Infinitely many equivalence classes of microknots.
Deep neural networks can generate any 2D distribution with high accuracy.
problem Generating accurate high-dimensional distributions from random noise.
method A deep neural network with a space-filling property of sawtooth functions.
result The network can approximate any 2D Lipschitz-continuous distribution arbitrarily closely.
Model approximates market prices and returns without prior market dynamics.
problem Simultaneously approximate market prices and log returns.
method GDN model of Kratsios and Papon (2022) for generalized Ornstein-Uhlenbeck process.
result Universal approximation guarantees for conditional distributions and contingent claims.
Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.
problem Statistical and computational limits of prompt tuning for transformer-based models.
method Investigation of single-head transformers with a single self-attention layer, proving universality and efficiency under SETH.
result Existence of almost-linear time prompt tuning inference algorithms under certain conditions.
Study Gaussian approximation for deep neural networks with random weights.
problem Understanding the distribution of deep neural networks with random weights.
method Established Gaussian approximation bounds in Wasserstein-1 norm.
result Convergence rates of order n−(1/6)L−1+ε for deep networks with proportional layer widths. Graded Transformers embed algebraic structure in neural networks through graded transformations.
problem Efficiently modeling hierarchical and structured data in neural networks.
method Introduces Linearly Graded Transformer (LGT) and Exponentially Graded Transformer (EGT) with graded scaling operators.
result Establishes rigorous guarantees and improved efficiency for structured data.
Paper introduces a neural network training algorithm for noisy data that achieves optimal parameters and replicates real-world behaviors.
problem Theoretical gap between universal approximation theorems and practical machine learning with noisy data.
method Randomized training algorithm for neural networks trained on noisy data samples.
result Trained neural networks achieve optimal parameters and exhibit real-world behaviors like sub-linear complexity and interpolation.
Method approximates Lipschitz domains with smoother shapes.
problem Approximating bounded Lipschitz domains.
method Sequence of smooth, bounded domains with weak curvatures.
result Uniform isocapacitary estimates for approximating sets.
Prototype rules simplify multiclass classification in metric spaces, achieving consistency and reduced complexity.
problem Multiclass classification in metric spaces, focusing on universal consistency and convergence rates.
method Novel Proto-NN and hybrid rules for multiclass classification in metric spaces, analyzing convergence rates.
result Proto-NN is universally consistent and simpler to implement, with similar computational advantages.
The paper proposes a deep learning approach to efficiently approximate diffeomorphisms for shape alignment.
problem Finding optimal reparameterizations of shapes for computing geodesic distances.
method The authors develop a neural network-based algorithm to construct approximations of diffeomorphisms using PyTorch.
result The proposed method achieves universal approximation properties and bounds on Lipschitz constants for the constructed diffeomorphisms.
The study examines the limitations of bi-Lipschitz Normalizing Flows in approximating certain distributions.
problem The expressivity of bi-Lipschitz Normalizing Flows in approximating specific target distributions.
method Characterization of expressivity through lower bounds on Total Variation distance and discussion of potential remedies.
result Several target distributions are difficult to approximate using bi-Lipschitz Normalizing Flows, and lower bounds on their approximation are provided.
Mixtures of neural operators reduce active complexity in operator learning.
problem Reduction of active complexity in operator learning models.
method Constructive comparison between routed mixtures of neural operators (MoNOs) and a fixed single-neural-operator construction.
result Every scalar uniformly continuous nonlinear operator can be approximated by a MoNO whose active expert has smaller depth, width, and rank scaling.
We present sufficient conditions for the cohomology of a closed aspherical manifold to be proper Lipschitz in sense of Connes-Gromov-Moscovici [CGM]. The conditions are stated in terms of the Stone-Čech compactification of the universal cover of a manifold. We show that these conditions are formally weaker than the suf…
GroupSort neural networks can approximate Lipschitz continuous functions.
problem Understanding and improving the expressive power of neural networks with Lipschitz constraints.
method Introduced and studied GroupSort neural networks with constraints on weights, proving their ability to approximate Lipschitz continuous functions.
result GroupSort networks can represent any Lipschitz continuous piecewise linear functions and are well-suited for approximating general Lipschitz continuous functions.
In this paper we prove that every definable set has a definable triangulation which is locally Lipschitz and weakly bi-Lipschitz on the natural simplicial stratification of the simplicial complex. We also distinguish a class T of regularity conditions and give a universal construction of a definable triangulation with …
Almost perimeter-minimizing boundaries in plentiful groups can be approximated by Lipschitz graphs.
problem Regularity of boundaries in plentiful groups.
method Lipschitz approximation of boundaries.
result Boundary of almost minimizers can be approximated by intrinsic Lipschitz graphs.
The paper tightens bounds on distances between Reeb graphs.
problem Certifying quasi-universality of distances between Reeb graphs.
method Establishes tight bi-Lipschitz bounds for various distances.
result Proves strict universality of the functional contortion distance for contour trees and coincides with interleaving distance for merge trees.
We show, as our main theorem, that if a Lipschitz map from a compact Riemannian manifold M to a connected compact Riemannian manifold N, where dimM≥dimN, has no singular points on M in the sense of F.H. Clarke, then the map admits a smooth approximation via Ehresmann fibrations. We also show the Reeb s…
Study shows shallow ReLU networks struggle with high-dimensional Lipschitz functions.
problem Expressing high-dimensional Lipschitz functions with shallow ReLU networks.
method Established lower bounds on shallow network complexity for polynomial approximation.
result Shallow ReLU networks suffer from the curse of dimensionality for Lipschitz functions.
Scalable method bounds Lipschitz constant of generative models.
problem Bounding the Lipschitz constant of generative models.
method Layerwise convex approximations using zonotopes.
result Efficient and tight bounds on generative models.
We introduce a topological invariant, it a type of a graph-manifold, which takes natural values. For a 4-dimensional graph-manifold, whose type does not exceed two, it is proved that its universal cover is bi-Lipschitz equivalent to a universal cover of an orthogonal graph-manifold (for any Riemannian metrics on graph-…
The study provides conditions for approximating Riemannian manifolds with polyhedral metrics.
problem Approximating Riemannian manifolds with polyhedral metrics.
method Conditions on curvature tensors for Lipschitz and local polyhedral approximations.
result Conditions are sufficient for local polyhedral approximations, conjectured to be sufficient for global approximations.
Let us consider a Riemannian manifold M (either separable or non-separable). We prove that, for every ε>0, every Lipschitz function f:M→R can be uniformly approximated by a Lipschitz, C1-smooth function g with $\Lip(g)\le \Lip(f)+ε$. As a consequence, every Riemannian manifold is uniformly …
Softmax attention approximates complex functions and subsumes many known universal approximators.
problem Universal approximation of continuous sequence-to-sequence functions.
method Interpolation-based analysis of attention's internal mechanism, showing its ability to approximate ReLU functions.
result Softmax attention is a universal approximator for continuous sequence-to-sequence functions.
RVFL networks can efficiently approximate Lipschitz functions in L∞ norm.
problem Efficiently approximating Lipschitz continuous functions in L∞ norm.
method Random Vector Functional Link (RVFL) network with ReLU activation functions, proving approximation in L∞ norm.
result An RVFL with ReLU activation functions can approximate Lipschitz continuous functions in L∞ norm.
Techniques known as Nonlinear Set Membership prediction, Lipschitz Interpolation or Kinky Inference are approaches to machine learning that utilise presupposed Lipschitz properties to compute inferences over unobserved function values. Provided a bound on the true best Lipschitz constant of the target function is known…
The paper simplifies arguments for stationary varifolds results.
problem Height bound and Lipschitz approximation for stationary varifolds.
method Simpler arguments to obtain height bound and Lipschitz approximation.
result Excess decay as a consequence of height bound and Lipschitz approximation.
Study shows limits on deep and shallow neural networks for approximating compact sets.
problem Understanding the limitations of deep and shallow neural networks in approximating compact sets.
method Proved Carl's type inequalities for approximation error, using Lipschitz widths.
result Lower bounds on approximation error for neural network outputs.
Study identifies three quantization regimes for ReLU networks.
problem Approximation of Lipschitz functions by ReLU networks with finite-precision weights.
method Established through nonasymptotic tight lower and upper bounds on minimax approximation error.
result Memory-optimality achieved in proper quantization regime for deep networks.
JacNet learns Jacobians to enforce structure on derivatives for invertibility and Lipschitz functions.
problem Enforcing structure on derivatives of neural network mappings.
method Proposes using a neural network to directly learn the Jacobian of the input-output function, allowing control over derivative structure.
result Demonstrates learning invertible approximations to simple and 1-Lipschitz functions.
Smooths metrics on manifolds with curvature bounds and injectivity radius constraints.
problem Smooth metrics on manifolds with curvature and injectivity constraints.
method Bi-Lipschitz smoothing with controlled smoothing and volume lower bounds.
result Proves existence of smooth metrics with curvature bounds and injectivity radius constraints.
Paper provides statistical guarantees for GANs estimating Hölder space densities.
problem Statistical properties and theoretical guarantees for GANs.
method Approximation and statistical guarantees for GANs using Hölder space densities.
result GANs are consistent estimators of data distributions under strong discrepancy metrics.
Optimizes Lipschitz estimates for partitions of unity and characterizes spaces with Assouad-Nagata dimension.
problem Understanding the properties of partitions of unity and their Lipschitz bounds.
method Analyzes the standard partition of unity and its ℓp-generalizations, using the approximate midpoint property and Lebesgue number. result Optimal Lipschitz bounds for partitions of unity and characterizes metric spaces with Assouad-Nagata dimension.
Universal approximation for stochastic processes using Brownian motion.
problem Approximating stochastic processes with linear functionals.
method Establishing Lp-type universal approximation theorems for rough path spaces. result Linear functionals on the signature of time-extended Brownian motion can approximate any p-integrable stochastic process.