A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Transformation Equivariant Representations (TERs) aim to capture the intrinsic visual structures that equivary to various transformations by expanding the notion of {\em translation} equivariance underlying the success of Convolutional Neural Networks (CNNs). For this purpose, we present both deterministic AutoEncoding…
New Feedback Transformer architecture improves model performance by exposing past representations to future.
Unified framework for learning function representations using INRs and Transformers.
Transformer learns graph structure better with subgraph info.
The Weyl transform is introduced as a rich framework for data representation. Transform coefficients are connected to the Walsh-Hadamard transform of multiscale autocorrelations, and different forms of dyadic periodicity in a signal are shown to appear as different features in its Weyl coefficients. The Weyl transform …
Relational data representations have become an increasingly important topic due to the recent proliferation of network datasets (e.g., social, biological, information networks) and a corresponding increase in the application of statistical relational learning (SRL) algorithms to these domains. In this article, we exami…
PatchGT uses non-trainable graph patches to improve graph representation learning.
We present a signal representation framework called the sparse manifold transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maint…
Improves contrastive learning invariance with novel training objectives and feature averaging.
Transformer learns representations from time series data for money laundering detection.
Researchers analyze the geometric and statistical properties of transformer model representations.
A neural scene representation framework enforcing 3D transformations.
A framework uses free probability to analyze Transformer models.
We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure. Our Tensor-Product Transformer (TP-Transformer) sets a new state of the art on the recently-introduced Mathematics Dataset containing 56 categories of free-form math word-pr…
Transforms solutions of Davey-Stewartson II equation geometrically.
Researchers present and compare different representations of dissipative Hamiltonian DAE systems.
Representations in the auditory cortex might be based on mechanisms similar to the visual ventral stream; modules for building invariance to transformations and multiple layers for compositionality and selectivity. In this paper we propose the use of such computational modules for extracting invariant and discriminativ…
The key to success in machine learning (ML) is the use of effective data representations. Traditionally, data representations were hand-crafted. Recently it has been demonstrated that, given sufficient data, deep neural networks can learn effective implicit representations from simple input representations. However, fo…
This study investigates abrupt learning dynamics in Transformers, revealing plateau formation and internal representation collapse.
How can intelligent agents solve a diverse set of tasks in a data-efficient manner? The disentangled representation learning approach posits that such an agent would benefit from separating out (disentangling) the underlying structure of the world into disjoint parts of its representation. However, there is no generall…
Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformation-invariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invari…
The paper reinterprets knot group invariants using affine transformations.
CRATE-MAE learns structured representations from unlabeled data.
IGT learns graph representations without supervision.
We present a 2x2 Lax representation for discrete circular nets of constant negative Gauß curvature. It is tightly linked to the 4D consistency of the Lax representation of discrete K-nets (in asymptotic line parametrization). The description gives rise to Bäcklund transformations and an associated family. All the membe…
New representations for discrete surfaces derived from dual transforms.
Normality equations describe Newtonian dynamical systems admitting normal shift of hypersurfaces. These equations were first derived in Euclidean geometry. Then very soon they were rederived in Riemannian and in Finslerian geometry. Recently I have found that normality equations can be derived in geometry given by clas…
Vision Transformers show different internal representations compared to CNNs.
Subspace clustering assumes that the data is sepa-rable into separate subspaces. Such a simple as-sumption, does not always hold. We assume that, even if the raw data is not separable into subspac-es, one can learn a representation (transform coef-ficients) such that the learnt representation is sep-arable into subspac…
Paper recovers latent causal structure and linear transformation from indirect observations.
We construct a certain cross product of two copies of the braided dual of a quasitriangular Hopf algebra , which we call the elliptic double , and which we use to construct representations of the punctured elliptic braid group extending the well-known representations of the planar braid group attache…
Transformers reduce redundancy by focusing on invariant relational quantities.
CascadeXML improves multi-resolution learning for XMC with transformer features.
Sensory data are often comprised of independent content and transformation factors. For example, face images may have shapes as content and poses as transformation. To infer separately these factors from given data, various ``disentangling'' models have been proposed. However, many of these are supervised or semi-super…
Transformer-M learns molecular data in 2D or 3D formats.
The Interaction-Transformation (IT) is a new representation for Symbolic Regression that restricts the search space into simpler, but expressive, function forms. This representation has the advantage of creating a smoother search space unlike the space generated by Expression Trees, the common representation used in Ge…
Study geometric and representation theory of statistical transformation models.
Transformer model improves source code summarization.
This paper tackles causal representation learning with linear and general transformations.
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-orde…
A new model improves CT image quality from low-dose scans.
New unsupervised learning technique learns independent kernels for better machine learning tasks.
Multimodalities provide promising performance than unimodality in most tasks. However, learning the semantic of the representations from multimodalities efficiently is extremely challenging. To tackle this, we propose the Transformer based Cross-modal Translator (TCT) to learn unimodal sequence representations by trans…
Weierstrass-type representations have been used extensively in surface theory to create surfaces with special curvature properties. In this paper we give a unified description of these representations in terms of classical transformation theory of -surfaces.
We extend the framework of variational autoencoders to represent transformations explicitly in the latent space. In the family of hierarchical graphical models that emerges, the latent space is populated by higher order objects that are inferred jointly with the latent representations they act on. To explicitly demonst…
This paper compares Transformers and RNNs in various tasks, showing size differences.
Study of SL(2,R) representations on a once-punctured torus, showing Cantor set spectrum.
Enhances GNNs with text features for better fake news detection.