We discuss natural transformations in the context of Lie groupoids, and their infinitesimal counterpart. Our main result is an integration procedure that provides smooth natural transformations between Lie groupoid morphisms.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “natural transformation”
Transforms Aronszajn to Sikorski subcartesian spaces.
problem Mapping between subcategories of differential spaces.
method Constructs a natural transformation.
result Establishes a relationship between Aronszajn and Sikorski subcartesian spaces.
ETC improves Transformer models for long and structured inputs.
problem Scaling input length and encoding structured inputs in Transformers.
method Introduces global-local attention, relative position encodings, and CPC pre-training.
result Achieves state-of-the-art results on four natural language datasets.
Enhanced transformer converts whispered speech to natural speech.
problem Machine recognition of whispered speech is challenging.
method Proposes an enhanced transformer architecture trained end-to-end using supervised learning.
result Similar formant distributions of converted speech to groundtruth.
We present a signal representation framework called the sparse manifold transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maint…
In this paper, we generalize the famous Hasimoto's transformation by showing that the dynamics of a closed unidimensional vortex filament embedded in a three-dimensional manifold of constant curvature gives rise under Hasimoto's transformation to the non-linear Schrodinger equation. We also give a natural interpretatio…
The Transformer architecture has become increasingly popular over the past two years, owing to its impressive performance on a number of natural language processing (NLP) tasks. However, all Transformer computations occur at the level of word representations and therefore, it may be argued that Transformer models do no…
Paper shows invertibility of tensor X-ray transform on certain manifolds.
problem Invertibility of tensor X-ray transform on asymptotically conic manifolds.
method Used 1-cusp pseudodifferential operator algebra and modified solenoidal gauge condition.
result Invertibility of tensor X-ray transform up to natural obstruction.
Given a Lorentzian manifold, the light ray transform of a function is its integrals along null geodesics. This paper is concerned with the injectivity of the light ray transform on functions and tensors, up to the natural gauge for the problem. First, we study the injectivity of the light ray transform of a scalar func…
We define a Fourier-Mukai transform for a triple consisting of two holomorphic vector bundles over an elliptic curve and a homomorphism between them. We prove that in some cases the transform preserves the natural stability condition for a triple. We also define a Nahm transform for solutions to natural gauge-theoretic…
3D models vulnerable to adversarial attacks, new method improves success rate and naturalness.
problem Vulnerability of 3D deep learning models to adversarial examples in the physical world.
method ε-isometric (-ISO) attack considering geometric properties and invariance to physical transformations.
result Significantly improved attack success rate and naturalness of 3D adversarial examples.
In this paper we study the local magnetic ray transform of symmetric tensor fields up to rank two on a Riemannian manifold of dimension with boundary. In particular, we consider the magnetic ray transform of the combinations of tensors of different orders due to the nature of magnetic flows. We show that such …
Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
problem Understanding the expressivity of Transformers for function approximation.
method Theoretical analysis and experimental validation of Transformer's ability to approximate smooth functions.
result Transformers cannot reliably approximate smooth functions, relying on piecewise constant approximations.
In trying to generalize Bianchi's Bäcklund transformation of quadrics to Bäcklund transformations of isometric deformations of other (classes of) surfaces, we investigate basic features of the isometric deformation of surfaces via the Bäcklund transformation with isometric correspondence of leaves of a general nature (…
Linformer reduces transformer complexity to linear, improving efficiency.
problem High cost of training and deploying large transformer models for long sequences.
method Approximates self-attention with low-rank matrix, proposing Linformer with complexity.
result Linformer performs similarly to standard transformers but is more memory- and time-efficient.
Conditions for Penrose-Ward transformation on specific manifolds.
problem Conditions for Penrose-Ward transformation on almost -manifolds with almost twistorial structures.
method Necessary and sufficient conditions derived through Penrose-Ward transformation.
result Conditions for Penrose-Ward transformation on almost -manifolds with almost twistorial structures.
For many tasks and data types, there are natural transformations to which the data should be invariant or insensitive. For instance, in visual recognition, natural images should be insensitive to rotation and translation. This requirement and its implications have been important in many machine learning applications, a…
Analyzes how BPE tokenisation affects corpus statistics and model entropy in transformer models.
problem Understanding how natural language properties relate to tokenisation schemes in transformer models.
method Analyzes Shannon entropy of corpora under Zipfian distribution, investigates BPE transformations, trains language models, and uses attention diagnostics.
result Transformer models trained on BPE-tokenised corpora increasingly agree with Zipfian predictions as BPE depth increases, indicating reduced local token dependencies.
OLS is a special case of Transformer, revealing its linear nature.
problem Understanding the statistical essence of Transformer architecture.
method Algebraic proof and spectral decomposition of covariance matrix.
result Attention mechanism in Transformers is mathematically equivalent to OLS.
An audiovisual speaker conversion method is presented for simultaneously transforming the facial expressions and voice of a source speaker into those of a target speaker. Transforming the facial and acoustic features together makes it possible for the converted voice and facial expressions to be highly correlated and f…
Transformers improve time series modeling by capturing long-range dependencies.
problem Capturing long-range dependencies in time series data.
method Summarized and reviewed adaptations of Transformers for time series analysis.
result Transformers enhance time series forecasting, anomaly detection, and classification.
Fusion of transformer networks using optimal transport for improved performance.
problem Improving performance of transformer-based models through fusion.
method Exploiting optimal transport for soft alignment of transformer components.
result Consistently outperforms vanilla fusion and individual parent models.
Predictive Sparse Manifold Transform learns dynamic video sequences.
problem Learning and predicting natural dynamics in video sequences.
method Two-layer framework: sparse coding and manifold learning.
result PSMT with dynamic embedding space outperforms static baselines in future frame prediction.
We study Cheeger-Simons differential characters and provide geometric descriptions of the ring structure and of the fiber integration map. The uniqueness of differential cohomology (up to unique natural transformation) is proved by deriving an explicit formula for any natural transformation between a differential cohom…
This study compresses BERT to make it suitable for low-capability devices.
problem Large Transformer models are resource-intensive.
method Survey and analysis of model compression techniques for BERT.
result Insights into best practices for compressing large-scale Transformer models.
The concept of the Ricci soliton was introduced by Hamilton. Ricci soliton is defined by vector field and it's a natural generalization of Einstein metric. We have shown earlier that the vector field of Ricci soliton is an infinitesimal harmonic transformation. In our paper, we survey Ricci solitons geometry as an appl…
Local invertibility of ray transforms on convex manifolds.
problem Invertibility of ray transforms on compact Riemannian manifolds with strictly convex boundary.
method Local invertibility results for transverse and mixed ray transforms of 1 and 1+1 tensors.
result Local invertibility of ray transforms near boundary points, leading to global results.
We prove a categorified version of the Poincaré lemma. The natural setting for our result is that of -local systems. More precisely, we show that any smooth homotopy between maps and induces an -natural transformation between the corresponding pullback functors. This transformation is…
We give a construction of a Poisson transform mapping density valued differential forms on generalized flag manifolds to differential forms on the corresponding Riemannian symmetric spaces, which can be described entirely in terms of finite dimensional representations of reductive Lie groups. Moreover, we will explicit…
In this paper we consider the geodesic X-ray transform with attenuation coefficient as a combination of smooth complex function and 1-form. We show that attenuated X-ray transform applied to the pair of tensors is injective modulo the natural obstruction.
We study the geometric properties of Darboux transforms of constant mean curvature (CMC) surfaces and use these transforms to obtain an algebro-geometric representation of constant mean curvature tori. We find that the space of all Darboux transforms of a CMC torus has a natural subset which is an algebraic curve (call…
A reductive structure is associated here with Lagrangian canonically defined conserved quantities on gauge-natural bundles. Parametrized transformations defined by the gauge-natural lift of infinitesimal principal automorphisms induce a variational sequence such that the generalized Jacobi morphism is naturally self-ad…
The classical Liouville Theorem on conformal transformations determines local conformal transformations on the Euclidean space of dimension . Its natural adaptation to the general framework of Riemannian structures is the 2-rigidity of conformal transformations, that is such a transformation is fully determined…
The study analyzes the convergence rate of a large transformer model.
problem Understanding the convergence rate of over-parameterized transformer models.
method Theoretical analysis focusing on gradient descent optimization.
result Theoretical upper bound on missclassification probability.
Many common sequential data sources, such as source code and natural language, have a natural tree-structured representation. These trees can be generated by fitting a sequence to a grammar, yielding a hierarchical ordering of the tokens in the sequence. This structure encodes a high degree of syntactic information, ma…
New pruning method retains model expressiveness for NLP tasks.
problem Pruning large pretrained transformer models for real-world deployment.
method Mixture Gaussian Prior Pruning (MGPP) algorithm.
result MGPP outperforms existing pruning methods in high sparsity settings.
We introduce the Koenigs lattice, which is a new integrable reduction of the quadrilateral lattice (discrete conjugate net) and provides natural integrable discrete analogue of the Koenigs net. We construct the Darboux-type transformations of the Koenigs lattice and we show permutability of superpositions of such trans…
Transformers improve logical reasoning on longer proofs but struggle with length.
problem Understanding systematic generalization in neural proof generation.
method Soft theorem-proving using Transformer models, evaluating logical consistency and inference accuracy.
result Transformers improve generalization with longer proofs but have difficulty with length.
New method learns identity-preserving transformations on data manifolds without labels.
problem Learning identity-preserving transformations on natural variations without supervision.
method Introduces a learning strategy that does not require transformation labels and learns local regions for operators.
result Trains on MNIST and Fashion MNIST, and CelebA, learning transformations without labels.
New method eliminates domain size restrictions for X-ray transform inversion.
problem Injectivity and stability of X-ray transform in convex domains.
method Semiclassical analysis to invert X-ray transform without small domain assumptions.
result Elimination of domain size restrictions for injectivity and stability.
We show that for a simple surface with boundary the attenuated ray transform in the presence of a unitary connection and a skew-Hermitian Higgs field is injective modulo the natural obstruction for functions and vector fields. We also show that the connection and the Higgs field are uniquely determined by the scatterin…
Natural gradient descent is a robust optimization method for machine learning.
problem Training poorly parameterized networks efficiently.
method Optimization algorithms with natural transformation properties.
result Optimization algorithms with natural transformation properties are more efficient for poorly parameterized networks.
Weil algebra morphism induce natural transformations between Weil bundles. In some well known cases, a natural transformation is endowed with a canonical structure of affine bundle. We show that this structure arises only when the Weil algebra morphism is surjective and its kernel has null square. Moreover, in some cas…
Intelligent Momentum Transformer outperforms traditional trading strategies.
problem Improving time-series momentum and mean-reversion trading strategies.
method Attention-based deep-learning architecture (Momentum Transformer) combining attention and LSTM.
result Momentum Transformer outperforms benchmarks and adapts to new market regimes.
Geometric structures on quaternionic unit ball for slice regular Möbius transformations.
problem No new problem introduced.
method Introducing Hermitian, Riemannian, and Kähler-like structures on quaternionic unit ball using regular Möbius transformations.
result Geometric structures are natural generalizations of complex setup and solve problems not achieved by other geometries.
Study reveals Transformer's expressive power and mechanisms.
problem Understanding the approximation properties of Transformer for sequence modeling.
method Systematic study of Transformer's components and their combined effects, establishing approximation rates.
result Reveals roles of critical parameters in Transformer, such as number of layers and attention heads.
This PHD thesis is concerned with uncertainty relations in quantum probability theory, state estimation in quantum stochastics, and natural bundles in differential geometry. After some comments on the nature and necessity of decoherence in open systems and its absence in closed ones, we prove sharp, state-independent i…
We study natural differential operators transforming two tensor fields into a tensor field. First, it is proved that all bilinear operators are of order one, and then we give the full classification of such operators in several concrete situations.