Develops a new approach to establish universality for any-dimensional machine learning models.
problem Understanding universality for models with inputs of varying sizes.
method Identifies any-dimensional functions with a unique function in an infinite-dimensional limit space, using symmetries and relations between inputs of different sizes.
result Establishes universality for several existing architectures and proposes modifications to restore it.
Stochastic gradient descent converges to universal limits in high dimensions.
problem Statistical tasks in high dimensions with specific data projections.
method Stochastic gradient descent applied to mixture distributions, proving universality of limits.
result The ODE limits are universal for mixtures of arbitrary product distributions.
We prove that if Y is the Gromov-Hausdorff limit of a sequence of complete manifolds, Min, with a uniform lower bound on Ricci curvature then Y has a universal cover.
Study the topology of Ricci limit spaces using Gromov-Hausdorff limits.
problem Topology of Ricci limit spaces.
method Gromov-Hausdorff limits, slice theorem for isometric pseudo-group actions, uniform diameter bounds.
result Established semi-locally simply connected property and described universal cover.
Study on nodal components of random band-limited functions on surfaces, finding a universal law.
problem Distribution of tangencies of nodal components to a vector field on surfaces.
method Analysis of random band-limited functions on smooth compact Riemannian surfaces with vector fields.
result The distribution of tangencies to a vector field on nodal components of random band-limited functions on surfaces follows a universal deterministic law.
The Basilica Julia set is universally equivalent to other complex dynamics sets.
problem Establishing the universality of the Basilica Julia set.
method Quasiconformal equivalence and geometric finiteness.
result The Basilica Julia set is quasiconformally equivalent to other complex dynamics sets.
Universal spaces for finite topological spaces simplify shape descriptions.
problem Describing shape properties of compact metric spaces.
method Inverse limits of finite spaces and Alexandroff extensions.
result Universal spaces simplify shape descriptions of compact metric spaces.
Sumformer simplifies Transformers to handle long sequences efficiently.
problem Quadratic complexity of Transformers limits their use with long sequences.
method Introducing Sumformer, a simple architecture that universally approximates equivariant sequence-to-sequence functions.
result Sumformer achieves the first universal approximation results for Linformer and Performer.
The study constructs a dense orbit in the universal commensurability augmented Teichmüller space.
problem Understanding the dense orbit in the universal commensurability augmented Teichmüller space.
method Using isometric embeddings and directed limits of augmented Teichmüller and moduli spaces.
result The action of the universal commensurability modular group on the universal commensurability augmented Teichmüller space produces a dense orbit.
New framework explains normalizing flows' power and limitations.
problem Understanding the expressive power and limitations of normalizing flows.
method Theoretical framework for well-conditioned coupling-based normalizing flows and volume-preserving flows.
result RealNVP is distributionally universal, but volume-preserving flows are not.
We extend the notion of canonical measures to all (possibly non-compact) metric graphs. This will allow us to introduce a notion of "hyperbolic measures" on universal covers of metric graphs. Kazhdan's theorem for Riemann surfaces describes the limiting behavior of canonical (Arakelov) measures on finite covers in rela…
Study classifies super vector bundles and proves universality.
problem Homotopy classification of super vector bundles.
method Construction of supergrassmannians, Gauss morphism, multilinear algebra, direct and inverse limits.
result Proves the resulting super vector bundle is universal.
Thurston's boundary to the universal Teichmüller space T(H) is the set of asymptotic rays to the embedding of T(H) in the space of geodesic currents; the boundary is identified with the projective bounded measured laminations PMLbdd(H) of H. We prove that each Teichmüller …
We construct a compact nonpositively curved squared 2-complex whose universal cover contains a flat plane that is not the limit of periodic flat planes.
In this survey article we will consider universal lower bounds on the volume of a Riemannian manifold, given in terms of the volume of lower dimensional objects (primarily the lengths of geodesics). By `universal' we mean without curvature assumptions. The restriction to results with no (or only minimal) curvature assu…
A new method inflates and deflates data manifolds to estimate densities without losing universality.
problem Density estimation on low-dimensional manifolds with non-Euclidean support.
method Inflation-deflation approach using Normalizing Flows with added noise.
result Exact estimation of densities on manifolds with sufficient conditions and Gaussian noise approximation.
In this paper, we study adaptive online convex optimization, and aim to design a universal algorithm that achieves optimal regret bounds for multiple common types of loss functions. Existing universal methods are limited in the sense that they are optimal for only a subclass of loss functions. To address this limitatio…
Derived geometry can be defined as the universal way to adjoin finite homotopical limits to a given category of manifolds compatibly with products and glueing. The point of this paper is to show that a construction closely resembling existing approaches to derived geometry in fact produces a geometry with this universa…
Study shows perceptrons with random labels perform similarly to Gaussian data.
problem The assumption of Gaussian input data is often seen as a limitation in machine learning.
method Analyzed generalized linear classification (perceptron model) with random labels.
result Minimum training loss is independent of data covariance for high-dimensional input data.
Learning three data points can generate all types of periodic orbits in a neural network.
problem Can learning three data points generate all types of periodic orbits in a neural network?
method Investigated a continuous one-dimensional map with period three in a random neural network in its thermodynamic limit.
result Almost all learned periods are unstable, and each network has its own characteristic attractors.
Study shows limitations and universality of equivariant QNNs with Sn-equivariant gates.
problem Understanding the expressiveness of Sn-equivariant QNNs with k-body gates. method Investigated the interplay between symmetry and k-bodyness in Sn-equivariant QNN generators. result QNNs are semi-universal but not universal with one- and two-body Sn-equivariant gates. Paper generalizes Gaussian universality and CGMT to dependent data, impacting data augmentation in high-dimensional logistic regression.
problem Limitation of Gaussian universality and CGMT in handling dependent data.
method Generalizes Gaussian universality and CGMT to dependent data (block dependence, m-dependence, mixing). Establishes a novel CGMT framework.
result Gaussian universality holds for high-dimensional logistic regression under various types of dependence.
If p:Y→X is an unramified covering map between two compact oriented surfaces of genus at least two, then it is proved that the embedding map, corresponding to p, from the Teichmüller space T(X), for X, to T(Y) actually extends to an embedding between the Thurston compactification of the tw…
Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.
problem Statistical and computational limits of prompt tuning for transformer-based models.
method Investigation of single-head transformers with a single self-attention layer, proving universality and efficiency under SETH.
result Existence of almost-linear time prompt tuning inference algorithms under certain conditions.
Graph Neural Networks struggle on random graphs without node identifiers.
problem Graph Neural Networks' limitations on random graphs without node identifiers.
method Study of Graph Neural Networks and Structural Graph Neural Networks convergence on large random graphs.
result Structural Graph Neural Networks are more powerful and universal than Graph Neural Networks on random graphs.
In an earlier paper [Acta Mathematica, v. 176, 1996, 145-169, alg-geom/9505024 ] the present authors and Dennis Sullivan constructed the universal direct system of the classical Teichmüller spaces of Riemann surfaces of varying genus. The direct limit, which we called the universal commensurability Teichmüller space, $…
The main goal of the paper is to prove the existence of the universal cover for RCD∗(K,N)-spaces. This generalizes earlier work of C. Sormani and the second named author on the existence of universal covers for Ricci limit spaces. As a result, we also obtain several structure results on the (revised) fundamental gro…
Constructs universal link invariants from intersections in configuration spaces.
problem Globalise topologically all coloured Jones polynomials and ADO polynomials.
method Defines new link invariants from graded intersections in configuration spaces.
result Recover all coloured Jones polynomials and ADO polynomials for links.
A taut foliation of a hyperbolic 3-manifold has the continuous extension property for leaves in almost every direction; that is, for each leaf of the universal cover of the foliation and almost every geodesic ray in the leaf, the limit of the ray in the universal cover of the 3-manifold is a well-defined point in the i…
A practical limitation of deep neural networks is their high degree of specialization to a single task and visual domain. Recently, inspired by the successes of transfer learning, several authors have proposed to learn instead universal, fixed feature extractors that, used as the first stage of any deep network, work w…
This work establishes universality for deep equivariant networks, overcoming limitations of previous approaches.
problem Rarity of universality results for equivariant neural networks, especially in high-dimensional settings.
method Develops a more general account of universality for equivariant networks, introducing entry-wise separability and readout layers.
result Deep equivariant networks achieve universality under entry-wise separability, with or without readout layers.
This paper explores the limits of deep learning in poly-time.
problem Characterizing function distributions that deep learning can or cannot learn efficiently.
method Analysis of SGD and GD-based deep learning approaches, proving universality and non-universality results.
result SGD-based deep learning is efficiently universal, while GD-based is not, especially with large batches.
The study characterizes quasiperiodic surfaces in pseudo-hyperbolic spaces with curvature conditions.
problem Characterizing quasiperiodic surfaces in pseudo-hyperbolic spaces.
method Curvature conditions, Gromov hyperbolicity, conformal hyperbolicity.
result Limit curves of quasiperiodic surfaces in the Einstein Universe have canonical quasisymmetric parametrizations.
Unified framework for accelerating DNNs on resource-limited platforms.
problem Accelerating DNN execution on resource-limited platforms.
method Block-based pruning framework with reweighted regularization.
result First universal framework for both CNNs and RNNs with real-time acceleration and no accuracy compromise.
We study the volume distribution of nodal domains of random band-limited functions on generic manifolds, and find that in the high energy limit a typical instance obeys a deterministic universal law, independent of the manifold. Some of the basic qualitative properties of this law, such as its support, monotonicity and…
Study of convergence of point-object configurations to a charged dust continuum.
problem Understanding the convergence of discretized point-object configurations to a charged dust continuum.
method Establishing existence and uniqueness of horizons/minimal surfaces, studying geometries of regions exterior to minimal surfaces, and discussing limits.
result Examples of scalar curvature jumps upon taking Gromov-Hausdorff and intrinsic flat limits.
We study the spherical cap packing problem with a probabilistic approach. Such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector. When the set of unit vectors …
This paper gives a quantitative version of Thurston's hyperbolic Dehn surgery theorem. Applications include the first universal bounds on the number of non-hyperbolic Dehn fillings on a cusped hyperbolic 3-manifold, and estimates on the changes in volume and core geodesic length during hyperbolic Dehn filling. The proo…
The paper analyzes thin-shell limits for viscous operators on Riemannian hypersurfaces.
problem Analyzing boundary conditions and thin-shell limits for viscous operators on arbitrary smooth hypersurfaces.
method Decomposing the ambient Bochner Laplacian into intrinsic and radial pieces, proving results for stress-free and Hodge boundary conditions.
result Universal thin-shell limits for viscous operators on arbitrary smooth hypersurfaces, including stress-free and Hodge boundary conditions.
Thurston's boundary to the universal Teichmüller space T(D) is the space PMLbdd(D) of projective bounded measured laminations of D. A geodesic ray in T(D) is of Teichmüller type if it shrinks vertical foliation of an integrable holomorphic quadratic differential. In a prio…
We briefly review data analysis of the Island order book, part of NASDAQ, which suggests a framework to which all limit order markets should comply. Using a simple exclusion particle model, we argue that short-time price over-diffusion in limit order markets is due to the non-equilibrium of order placement, cancellatio…
We present evidence, that if a large enough set of high resolution stock market data is analyzed, certain analogies with physics -- such as scaling and universality -- fail to capture the full complexity of such data. Despite earlier expectations, the mean value per trade, the mean number of trades per minute and the m…
The paper sets limits on neural network sizes based on dataset shapes.
problem Understanding the size of neural networks needed for accurate predictions.
method Examined how the shape of data influences neural network complexity.
result Established upper limits on neural network width based on dataset topology.
We consider compact, aspherical solenoids obtained as the inverse limit of a system of CW~complexes and covering maps. This includes P-adic solenoids, as well as the universal hyperbolic solenoid of Teichmüller theory. Using ideas from shape theory, we classify maps between such solenoids up to homotopy, and we prove…
Exact distribution of split conformal prediction coverage found.
problem Determining the reliability of prediction sets in batch mode.
method Analysis of exchangeable data to find universal distribution of empirical coverage.
result Exact distribution of empirical coverage is universal and determined by nominal miscoverage level and calibration sample size.
Research shows RCD* spaces are semi-locally simply connected.
problem Understanding the topological properties of RCD* spaces.
method Proving semi-locally simply connected property for any point and radius.
result RCD* spaces are semi-locally simply connected.
PLN-Nets with two linear layers and parallel LN achieve universal approximation.
problem Limitations of standard neural network architectures in universal approximation.
method Introduced PLN-Nets combining two linear layers with parallel LN.
result PLN-Nets achieve universal approximation, while standard LN has limited power.
New geometric invariant from disc intersections captures all coloured Jones polynomials.
problem Constructing a universal knot invariant from configuration spaces.
method Defining a new local system and Lagrangian submanifolds in the disc.
result The new invariant recovers Habiro's universal invariant and more.