arXiv research

URT layer improves few-shot image classification across diverse domains.

Study presents a method to induce a generalized neural network from joint group invariant functions.

Unified theorem for deep and shallow joint-equivariant machines.

Deep neural networks are proven universally powerful using Koopman operator.

Transformers enable in-context learning with guarantees for a wide range of tasks.

This paper describes a language representation model which combines the Bidirectional Encoder Representations from Transformers (BERT) learning mechanism described in Devlin et al. (2018) with a generalization of the Universal Transformer model described in Dehghani et al. (2018). We further improve this model by addin…

We construct a certain cross product of two copies of the braided dual $\tilde H$ of a quasitriangular Hopf algebra $H$, which we call the elliptic double $E_H$, and which we use to construct representations of the punctured elliptic braid group extending the well-known representations of the planar braid group attache…

CF-INNs can approximate any invertible function, resolving a long-standing problem.

Analyzes the differential expansion of knot polynomials, focusing on its applicability and modifications.

In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine t…

A framework for learning disentangled representations of symmetric environments.

We introduce a transformer-based GNN model, named UGformer, to learn graph representations. In particular, we present two UGformer variants, wherein the first variant (publicized in September 2019) is to leverage the transformer on a set of sampled neighbors for each input node, while the second (publicized in May 2021…

MLPs can approximate any function in context, challenging the importance of in-context universality.

Sumformer simplifies Transformers to handle long sequences efficiently.

Graded Transformers embed algebraic structure in neural networks through graded transformations.

SNNs can represent complex functions efficiently.

Transformers can emulate various algorithms by prompting, proving universality.

The paper explores simple and relatively simple transformation groups and their universal coverings.

Transformers can approximate any sequence-to-sequence function, surprising given their complexity.

Transformers are explained as infinite-dimensional kernel machines.

Many learning algorithms require categorical data to be transformed into real vectors before it can be used as input. Often, categorical variables are encoded as one-hot (or dummy) vectors. However, this mode of representation can be wasteful since it adds many low-signal regressors, especially when the number of uniqu…

A new KAN variant uses sinusoidal activations to approximate functions.

Sig-Splines model uses signatures and splines for time series data, achieving universality and convexity.

We give a characterization of flat affine connections on manifolds by means of a natural affine representation of the universal covering of the Lie group of diffeomorphisms preserving the connection. From the infinitesimal point of view, this representation is determined by the 1-connection form and the fundamental for…

Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data such as 2D and 3D images, video, and other data where symmetries are present. In geometrical terms, regular G-CNNs represent data in terms of s…

Symmetry principles help in creating better AI representations.

New analysis shows RPE-based Transformers can't approximate all functions.

Drinfel'd used associators to construct families of universal representations of braid groups. We consider semi-associators (i.e., we drop the pentagonal axiom and impose a normalization in degree one). We show that the process may be reversed, to obtain semi-associators from universal representations of 3-braids. We v…

Adversarially trained transformers can learn robustly across tasks with minimal tuning.

TSLANet improves time series models by capturing long-term and short-term interactions.

Classifies conformal transformations in spacetimes without observer horizons.

Triangular map is a recent construct in probability theory that allows one to transform any source probability density function to any target density function. Based on triangular maps, we propose a general framework for high-dimensional density estimation, by specifying one-dimensional transformations (equivalently co…

Study extends Vogel's universality to torus knots in adjoint representation.

Skew parallelogram nets factorize, encompassing discrete differential geometry.

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

We present a universal knot polynomials for 2- and 3-strand torus knots in adjoint representation, by universalization of appropriate Rosso-Jones formula. According to universality, these polynomials coincide with adjoined colored HOMFLY and Kauffman polynomials at SL and SO/Sp lines on Vogel's plane, and give their ex…

MultiRocket boosts TSC speed and accuracy with pooling and transformations.

An orbifold is a topological space modeled on quotient spaces of a finite group actions. We can define the universal cover of an orbifold and the fundamental group as the deck transformation group. Let $G$ be a Lie group acting on a space $X$. We show that the space of isotopy-equivalence classes of $(G,X)$-structures …

Softmax attention approximates complex functions and subsumes many known universal approximators.

Based on the analogies between knot theory and number theory, we study a deformation theory for SL_2-representations of knot groups, following after Mazur's deformation theory of Galois representations. Firstly, by employing the pseudo-SL_2-representations, we prove the existence of the universal deformation of a given…

Sparse Transformers can approximate dense Transformers with only O(n) connections.

OptFormer learns universal HPO from diverse datasets.

Subspace clustering (SC) refers to the problem of clustering high-dimensional data into a union of low-dimensional subspaces. Based on spectral clustering, state-of-the-art approaches solve SC problem within a two-stage framework. In the first stage, data representation techniques are applied to draw an affinity matrix…

STRING improves 2D and 3D position encodings for better performance.

Single-layer Transformer can approximate any sequence mapping.

Automorphisms of Kodaira surfaces are shown to be affine transformations.

Efficient neural network invariant to symmetry subgroups.

Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.