arXiv research

MLPs can approximate any function in context, challenging the importance of in-context universality.

Sumformer simplifies Transformers to handle long sequences efficiently.

Transformers enable in-context learning with guarantees for a wide range of tasks.

The paper explores simple and relatively simple transformation groups and their universal coverings.

Transformers can approximate any sequence-to-sequence function, surprising given their complexity.

Sig-Splines model uses signatures and splines for time series data, achieving universality and convexity.

New analysis shows RPE-based Transformers can't approximate all functions.

Adversarially trained transformers can learn robustly across tasks with minimal tuning.

Classifies conformal transformations in spacetimes without observer horizons.

Deep neural networks are proven universally powerful using Koopman operator.

Study presents a method to induce a generalized neural network from joint group invariant functions.

Unified theorem for deep and shallow joint-equivariant machines.

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

Softmax attention approximates complex functions and subsumes many known universal approximators.

Sparse Transformers can approximate dense Transformers with only O(n) connections.

OptFormer learns universal HPO from diverse datasets.

Improved BERT model with latent persona and topic variables.

Single-layer Transformer can approximate any sequence mapping.

Automorphisms of Kodaira surfaces are shown to be affine transformations.

Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.

We apply Cartan's method of equivalence to find a Bäcklund autotransformation for the tangent covering of the universal hierarchy equation. The transformation provides a recursion operator for symmetries of this equation.

Building on the universal covering group of the general linear group, we introduce the composite spinor bundle whose subbundles are Lorentz spin structures associated with different gravitational fields. General covariant transformations of this composite spinor bundle are canonically defined.

Transformer pretraining yields strong EB performance without explicit adaptation.

The paper proves a transformation theorem under a monotone property of almost Euclidean factors of geodesic balls.

In the present paper, we study the finite type invariants of Gauss words. In the Polyak algebra techniques, we reduce the determination of the group structure to transformation of a matrix into its Smith normal form and we give the simplified form of a universal finite type invariant by means of the isomorphism of this…

Transformers can predict new tokens based on any number of context tokens, approximating continuous mappings with fixed resources.

URT layer improves few-shot image classification across diverse domains.

Transformers use ReLUs to approximate softmax efficiently.

The geometry of an admissible Bäcklund transformation for an exterior differential system is described by an admissible Cartan connection for a geometric structure on a tower with infinite--dimensional skeleton which is the universal prolongation of a $|1|$--graded semi-simple Lie algebra.

A theorem transforms Lorentzian to signature-changing metrics.

The Akbulut cork cannot transform all exotic 4-manifolds.

Transformers can emulate various algorithms by prompting, proving universality.

Graded Transformers embed algebraic structure in neural networks through graded transformations.

Transformers are explained as infinite-dimensional kernel machines.

Normalizing flows are shown to be equivalent to Bayesian networks, revealing new insights.

Paper proves CFlows can approximate any diffeomorphism and applies it in Bayesian optimization.

We construct a certain cross product of two copies of the braided dual $\tilde H$ of a quasitriangular Hopf algebra $H$, which we call the elliptic double $E_H$, and which we use to construct representations of the punctured elliptic braid group extending the well-known representations of the planar braid group attache…

Paper proposes universally consistent K-sample tests using any dependence measure.

GCNNs gain rotation invariance with more training augmentation, making SVD-Universal more effective.

Unique domain found in Einstein universe, simplifying manifold classification.

Sum-of-Squares flow improves autoregressive and flow-based density estimation.

Paper establishes rates of universal approximation for neural tangent kernels using transport mappings.

Gaussianization flows transform any random vector into a Gaussian, enabling efficient computation and sample generation.

CF-INNs can approximate any invertible function, resolving a long-standing problem.

Deep learning approximates geometric measures of planar curves.

In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine t…

UGformer uses transformers to learn graph representations.

Model approximates continuous functions in 1-Wasserstein space.