LLMs struggle with arithmetic tasks unless they use high numerical precision.
problem Improving arithmetical reasoning capabilities of LLMs.
method Theoretical analysis and empirical experiments on numerical precision.
result LLMs require high numerical precision to efficiently handle arithmetic tasks.
Transformer models improve arithmetic accuracy with number decomposition.
problem Transformer models struggle with arithmetic operations without decomposition.
method Fine-tuning models with a pipeline that decomposes numbers into units, tens, etc.
result Accuracy increased by 63% in five-digit addition tasks.
Recursive Feature Machines show grokking in modular arithmetic without neural networks.
problem Grokking in modular arithmetic tasks.
method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).
result RFM and neural networks learn block-circulant features to solve modular arithmetic.
Transformers learn to solve modular arithmetic tasks by in-context learning and skill composition.
problem Understanding how large language models generalize to unseen tasks in modular arithmetic.
method Pre-training on a set of modular arithmetic tasks and evaluating out-of-distribution performance.
result Transformers require two transformer blocks for out-of-distribution generalization, and deeper models exhibit transient out-of-distribution performance.
Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to car…
Transformer struggles with arithmetic length but improves with explicit structure encoding.
problem Transformers fail to generalize length in arithmetic tasks.
method Explicitly encoding structural symmetries via modified number formatting and custom positional encodings.
result Transformer can generalize up to 50-digit numbers without additional data.
Better neural arithmetic logic units improve cell counting model generalization.
problem Neural networks struggle with high cell counts outside training data range.
method Introduced Neural Arithmetic Logic Units (NALU) for arithmetic operations in existing architectures.
result Improved cell counting accuracy for higher numeric ranges with better generalization.
Neural networks learn modular arithmetic but not all, extending known solutions to generalize.
problem Neural networks struggle with modular arithmetic, especially for polynomials.
method Developed analytical solutions for MLP networks to learn modular addition and multiplication, then combined these solutions to generalize on arbitrary modular polynomials.
result Neural networks can learn and generalize solutions to modular polynomials, supporting the hypothesis that some polynomials are learnable.
Over-parameterized models can memorize noisy labels and still generalize well, revealing a hidden structure.
problem Understanding how over-parameterized models can simultaneously memorize noisy labels and generalize well.
method Investigated through modular arithmetic tasks with label noise using two-layer neural networks.
result Over-parameterized models can achieve near-perfect test accuracy with 80% label noise by extracting an internal generalization structure.
A fake quadric is a smooth projective surface that has the same rational cohomology as a smooth quadric surface but is not biholomorphic to one. We provide an explicit classification of all irreducible fake quadrics according to the commensurability class of their fundamental group. To accomplish this task, we develop …
CUTS removes corruption from models without clean data, improving utility and security.
problem Removing corruption from models without access to clean training data.
method CUTS uses a proxy set to amplify corruption and subtract it from model weights.
result CUTS recovers a large fraction of lost utility and nearly eliminates attacks with minimal damage.
Paper tackles division difficulty, proposing new methods to improve accuracy.
problem Division is the most challenging arithmetic operation for both humans and computers.
method Proposes two novel approaches: Neural Reciprocal Unit (NRU) and Neural Multiplicative Reciprocal Unit (NMRU), and improves an existing division module.
result Improves division accuracy from 70.2% to 91.6%.
Deep neural networks (DNN) are powerful models for many pattern recognition tasks, yet their high computational complexity and memory requirement limit them to applications on high-performance computing platforms. In this paper, we propose a new method to evaluate DNNs trained with 32bit floating point (float32) accura…
Study explores how neural networks and Transformers learn modular arithmetic with multiple inputs.
problem Understanding how neural networks and Transformers learn modular arithmetic with multiple inputs.
method Analytical characterization of features learned by neural networks and Transformers, focusing on margin maximization and Fourier spectra.
result Neural networks and Transformers require a minimum neuron count of \( m \geq 2^{2k-2} \cdot (p-1) \) to solve modular addition problems with \( k \) inputs and modulus \( p \).
Autonomous agents situated in real-world environments must be able to master large repertoires of skills. While a single short skill can be learned quickly, it would be impractical to learn every task independently. Instead, the agent should share knowledge across behaviors such that each task can be learned efficientl…
In this article, we investigate when the set of primitive geodesic lengths on a Riemannian manifold have arbitrarily long arithmetic progressions. We prove that in the space of negatively curved metrics, a metric having such arithmetic progressions is quite rare. We introduce almost arithmetic progressions, a coarsific…
Develops arithmetic PDE geometry concepts like curvature and cohomology.
problem Creating a geometry framework for arithmetic PDEs.
method Introducing arithmetic analogues of Levi-Civita and Chern connections, then developing curvature and characteristic classes.
result Arithmetic analogues of curvature and characteristic classes have been developed.
New geometric invariant limits the number of semi-arithmetic groups.
problem Understanding the structure of semi-arithmetic Fuchsian groups.
method Introducing a new geometric invariant called stretch and using the arithmetic Margulis lemma.
result There exist only finitely many conjugacy classes of semi-arithmetic groups with bounded arithmetic dimension, stretch, and coarea.
Course on arithmetic lattices at EPFL.
problem Understanding arithmetic lattices.
method Introductory course on arithmetic lattices.
result Introduction to arithmetic lattices.
MC-LSTM extends LSTM to conserve mass in neural networks.
problem Conservation laws in real-world systems.
method Extending LSTM's inductive bias to conserve mass.
result MC-LSTM sets new state-of-the-art for predicting peak flows.
Paper shows non-arithmetic surface with unique geometric property.
problem Non-arithmetic surfaces with unique geometric properties.
method Example of a non-arithmetic surface with marked length variety rigidity.
result Found a non-arithmetic surface with marked length variety rigidity.
New classification of hyperbolic Coxeter prisms.
problem Classifying hyperbolic Coxeter prisms.
method Determine which prisms are quasi-arithmetic or arithmetic.
result New insights into commensurability and systoles of associated orbifolds.
Ordering examples in modular arithmetic training affects model performance.
problem Training efficiency and model performance in modular arithmetic.
method Varying example ordering while keeping other conditions constant.
result Adversarially structured ordering suppresses learning, while fixed-ordering strategies achieve high accuracy.
Arithmetic Dijkgraaf-Witten theory constructs analogues in Chern-Simons TQFT.
problem Developing arithmetic analogues in Chern-Simons TQFT.
method Constructing arithmetic analogues of Chern-Simons 1-cocycle, prequantization bundle, and Chern-Simons functional.
result Decomposition and gluing formulas for arithmetic Chern-Simons invariants and arithmetic Dijkgraaf-Witten partition functions.
In this article, we prove that every arithmetic locally symmetric orbifold of classical type without Euclidean or compact factors has arbitrarily long arithmetic progressions in its primitive length spectrum. Moreover, we show the stronger property that every primitive length occurs in arbitrarily long arithmetic progr…
New method constructs non-arithmetic hyperbolic orbifolds from complex arithmetic ball quotients.
problem Creating non-arithmetic lattices in projective orthogonal groups.
method Using anti-holomorphic involutions on complex arithmetic ball quotients, gluing fixed loci along geodesic subspaces.
result Explicit calculation of the volume of constructed non-arithmetic orbifolds.
We show that the non-arithmetic lattices in PO(n,1) of Belolipetsky and Thomson (2011), obtained as fundamental groups of closed hyperbolic manifolds with short systole, are quasi-arithmetic in the sense of Vinberg, and, by contrast, the well-known non-arithmetic lattices of Gromov and Piatetski-Shapiro are not quasi-a…
The paper explores subspaces in hyperbolic lattices and their arithmetic properties.
problem Arithmeticity criterion for hyperbolic lattices and suborbifolds.
method Analysis of totally geodesic suborbifolds and Vinberg's commensurability invariants.
result Arithmeticity of hyperbolic orbifolds is linked to the existence of infinitely many fc-subspaces.
Convolutional neural networks (CNNs) have achieved remarkable performance in many applications, especially in image recognition tasks. As a crucial component of CNNs, sub-sampling plays an important role for efficient training or invariance property, and max-pooling and arithmetic average-pooling are commonly used sub-…
Define an arithmetic variety to be the quotient of a bounded symmetric domain by an arithmetic group. An arithmetic variety is algebraic, and the theorem in question states that when one applies an automorphism of the field of complex numbers to the coefficients of an arithmetic variety the resulting variety is again a…
Geodesics on modular surface yield arithmetic 3-manifolds.
problem Understanding arithmetic properties of modular surfaces.
method Constructing geodesics and analyzing their lifts.
result Complements of canonical lifts are arithmetic 3-manifolds.
New property identifies arithmetic lattices from nonuniform lattices.
problem Characterizing arithmetic lattices among nonuniform lattices.
method Introduced Bounded Clustering (B-C) property.
result B-C property uniquely identifies arithmetic lattices.
New proof shows maximal arithmetic groups are finite.
problem Finiteness of maximal arithmetic reflection groups.
method Arithmetic Margulis lemma without automorphic forms.
result Finiteness of maximal arithmetic reflection groups proven.
We study the arithmeticity of the Couwenberg-Heckman-Looijenga lattices in PU(n,1), and show that they contain a non-arithmetic lattice in PU(3,1) which is not commensurable to the non-arithmetic Deligne-Mostow lattice in PU(3,1).
Develops arithmetic PDE geometry using Fermat quotients.
problem Creating an arithmetic PDE analogue of Riemannian geometry.
method Using Fermat quotients and Frobenius elements in the absolute Galois group of a p-adic field. result Existence and uniqueness of geodesics and connections proved.
Study general hyperbolic gluings, proving quasi-arithmeticity of building blocks.
problem Proving quasi-arithmeticity of building blocks in hyperbolic gluings.
method Generalized gluings of hyperbolic orbifolds, proving quasi-arithmeticity.
result Building blocks of quasi-arithmetic gluings must also be quasi-arithmetic.
The study of systoles in arithmetic hyperbolic manifolds.
problem Understanding the systoles of arithmetic hyperbolic manifolds.
method Construction and analysis of arithmetic hyperbolic manifolds.
result Explicit bounds on volumes and systoles of arithmetic hyperbolic manifolds.
We prove that each lower-dimensional face of a quasi-arithmetic Coxeter polytope, which happens to be itself a Coxeter polytope, is also quasi-arithmetic. We also provide a sufficient condition for a codimension 1 face to be actually arithmetic, as well as a few computed examples.
We explore hybrid subgroups of certain non-arithmetic lattices in PU(2,1). We show that all of Mostow's lattices are virtually hybrids; moreover, we show that some of these non-arithmetic lattices are hybrids of two non-commensurable arithmetic lattices in PU(1,1).
New research shows certain arithmetic lattices can't be LERF.
problem Determining if arithmetic lattices are LERF.
method Analyzing trialitarian arithmetic lattices in PSO7,1(R). result Trialitarian arithmetic lattices in PSO7,1(R) are not LERF. Arithmetic spaces' thin parts are negligible, impacting Betti numbers.
problem Understanding the structure of arithmetic locally symmetric spaces.
method Analyzing thin parts and deducing asymptotic results on Betti numbers.
result Arithmetic spaces' thin parts are negligible, impacting Betti numbers.
We apply G. Prasad's volume formula for the arithmetic quotients of semi-simple groups and Bruhat-Tits theory to study the covolumes of arithmetic subgroups of SO(1,n). As a result we prove that for any even dimension n there exists a unique compact arithmetic hyperbolic n-orbifold of the smallest volume. We give a for…
Develops Kleinian Sphere Packings and Bugs, proving their arithmetic origins.
problem Understanding sphere packings and their arithmetic origins in various dimensions.
method Introduces Kleinian Sphere Packings and Bugs, extending Arithmeticity Theorem.
result Kleinian packings and Bugs come from Q-arithmetic lattices of simplest type.
Conditions for flat manifolds as cusp cross-sections in arithmetic hyperbolic manifolds.
problem Determining when a flat manifold can be a cusp cross-section in arithmetic hyperbolic manifolds.
method Analyzing rational representations of holonomy groups and quasi-arithmetic manifolds.
result Conditions for a flat manifold to appear as a cusp cross-section in every commensurability class of arithmetic hyperbolic manifolds.
Study on Alexander polynomials in braids, linking number theory and topology.
problem Distribution of Alexander polynomials in specific families of braids.
method Exploration of arithmetic invariants and analogies with number theory.
result New directions in arithmetic topology and statistics.
Study finds bounds for systole length on arithmetic punctured spheres.
problem Finding the shortest essential curve on arithmetic punctured spheres.
method Correspondence between surfaces and planar triangulations to bound systole length.
result Arithmetic surfaces do not achieve maximal systole length for n=7,10,11. The study finds infinitely many semi-arithmetic Riemann surfaces with dense systoles and distinct invariant trace fields.
problem Existence and properties of semi-arithmetic Riemann surfaces.
method Combining number theory and hyperbolic geometry to prove existence and properties of semi-arithmetic Riemann surfaces.
result Existence of infinitely many semi-arithmetic Riemann surfaces with dense systoles and distinct invariant trace fields.
Neural Power Unit (NPU) learns arbitrary power functions on real numbers.
problem Neural Networks struggle with generalizing beyond seen data and arithmetic operations.
method Introduces Neural Power Unit (NPU) that operates on real numbers and learns arbitrary power functions.
result NPU outperforms competitors in accuracy and sparsity on arithmetic datasets and discovers governing equations from data.