Neural Power Unit (NPU) learns arbitrary power functions on real numbers.
problem Neural Networks struggle with generalizing beyond seen data and arithmetic operations.
method Introduces Neural Power Unit (NPU) that operates on real numbers and learns arbitrary power functions.
result NPU outperforms competitors in accuracy and sparsity on arithmetic datasets and discovers governing equations from data.
The use of low-precision fixed-point arithmetic along with stochastic rounding has been proposed as a promising alternative to the commonly used 32-bit floating point arithmetic to enhance training neural networks training in terms of performance and energy efficiency. In the first part of this paper, the behaviour of …
Paper tackles division difficulty, proposing new methods to improve accuracy.
problem Division is the most challenging arithmetic operation for both humans and computers.
method Proposes two novel approaches: Neural Reciprocal Unit (NRU) and Neural Multiplicative Reciprocal Unit (NMRU), and improves an existing division module.
result Improves division accuracy from 70.2% to 91.6%.
Recursive Feature Machines show grokking in modular arithmetic without neural networks.
problem Grokking in modular arithmetic tasks.
method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).
result RFM and neural networks learn block-circulant features to solve modular arithmetic.
Neural networks learn modular arithmetic but not all, extending known solutions to generalize.
problem Neural networks struggle with modular arithmetic, especially for polynomials.
method Developed analytical solutions for MLP networks to learn modular addition and multiplication, then combined these solutions to generalize on arbitrary modular polynomials.
result Neural networks can learn and generalize solutions to modular polynomials, supporting the hypothesis that some polynomials are learnable.
Better neural arithmetic logic units improve cell counting model generalization.
problem Neural networks struggle with high cell counts outside training data range.
method Introduced Neural Arithmetic Logic Units (NALU) for arithmetic operations in existing architectures.
result Improved cell counting accuracy for higher numeric ranges with better generalization.
Deep neural networks (DNN) are powerful models for many pattern recognition tasks, yet their high computational complexity and memory requirement limit them to applications on high-performance computing platforms. In this paper, we propose a new method to evaluate DNNs trained with 32bit floating point (float32) accura…
Study explores how neural networks and Transformers learn modular arithmetic with multiple inputs.
problem Understanding how neural networks and Transformers learn modular arithmetic with multiple inputs.
method Analytical characterization of features learned by neural networks and Transformers, focusing on margin maximization and Fourier spectra.
result Neural networks and Transformers require a minimum neuron count of \( m \geq 2^{2k-2} \cdot (p-1) \) to solve modular addition problems with \( k \) inputs and modulus \( p \).
Constructing accurate and automatic solvers of math word problems has proven to be quite challenging. Prior attempts using machine learning have been trained on corpora specific to math word problems to produce arithmetic expressions in infix notation before answer computation. We find that custom-built neural networks…
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
problem Reducing multiplication complexity in low-resolution neural networks.
method Adapting neural networks to use low-resolution (8-bit) additions in accumulators, with a cyclic activation layer and overflow penalty regularizer.
result Achieves comparable classification accuracy to 32-bit counterparts using low-resolution additions.
Efficient Winograd convolution for INT8 networks using RNS.
problem Difficulty in applying Winograd algorithm to low-precision quantized networks.
method Extends Winograd algorithm to Residue Number System (RNS) for efficient INT8 convolution.
result Arithmetic complexity reduction up to 7.03x with performance improvement up to 2.30x-4.69x.
Deep neural networks have achieved impressive supervised classification performance in many tasks including image recognition, speech recognition, and sequence to sequence learning. However, this success has not been translated to applications like question answering that may involve complex arithmetic and logic reason…
Degree-Quant improves GNN efficiency by quantizing them without losing accuracy.
problem Efficiency of graph neural networks at inference time.
method Architecturally-agnostic method, Degree-Quant, for quantizing GNNs.
result Degree-Quant trained models perform as well as full-precision models and achieve up to 26% gains.
Stock prediction is a topic undergoing intense study for many years. Finance experts and mathematicians have been working on a way to predict the future stock price so as to decide to buy the stock or sell it to make profit. Stock experts or economists, usually analyze on the previous stock values using technical indic…
MC-LSTM extends LSTM to conserve mass in neural networks.
problem Conservation laws in real-world systems.
method Extending LSTM's inductive bias to conserve mass.
result MC-LSTM sets new state-of-the-art for predicting peak flows.
Deep neural networks struggle with numerical instability during training.
problem Numerical instability in gradient descent training of deep neural networks.
method Analysis of floating-point arithmetic and gradient descent in ReLU neural networks.
result It is highly unlikely for ReLU networks to maintain a superlinear number of affine pieces during training.
In this article, we investigate when the set of primitive geodesic lengths on a Riemannian manifold have arbitrarily long arithmetic progressions. We prove that in the space of negatively curved metrics, a metric having such arithmetic progressions is quite rare. We introduce almost arithmetic progressions, a coarsific…
Develops arithmetic PDE geometry concepts like curvature and cohomology.
problem Creating a geometry framework for arithmetic PDEs.
method Introducing arithmetic analogues of Levi-Civita and Chern connections, then developing curvature and characteristic classes.
result Arithmetic analogues of curvature and characteristic classes have been developed.
Quantized neural networks can represent all fixed-point functions under certain conditions.
problem Expressive power of quantized neural networks under fixed-point arithmetic.
method Analyzing necessary and sufficient conditions for quantized networks to represent all fixed-point functions.
result Various popular activation functions satisfy the sufficient condition for representing all fixed-point functions.
New geometric invariant limits the number of semi-arithmetic groups.
problem Understanding the structure of semi-arithmetic Fuchsian groups.
method Introducing a new geometric invariant called stretch and using the arithmetic Margulis lemma.
result There exist only finitely many conjugacy classes of semi-arithmetic groups with bounded arithmetic dimension, stretch, and coarea.
Course on arithmetic lattices at EPFL.
problem Understanding arithmetic lattices.
method Introductory course on arithmetic lattices.
result Introduction to arithmetic lattices.
Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to car…
Over-parameterized models can memorize noisy labels and still generalize well, revealing a hidden structure.
problem Understanding how over-parameterized models can simultaneously memorize noisy labels and generalize well.
method Investigated through modular arithmetic tasks with label noise using two-layer neural networks.
result Over-parameterized models can achieve near-perfect test accuracy with 80% label noise by extracting an internal generalization structure.
Paper shows non-arithmetic surface with unique geometric property.
problem Non-arithmetic surfaces with unique geometric properties.
method Example of a non-arithmetic surface with marked length variety rigidity.
result Found a non-arithmetic surface with marked length variety rigidity.
New classification of hyperbolic Coxeter prisms.
problem Classifying hyperbolic Coxeter prisms.
method Determine which prisms are quasi-arithmetic or arithmetic.
result New insights into commensurability and systoles of associated orbifolds.
This paper shows neural networks can solve complex graph problems efficiently.
problem Solving exact maximum flow computation and minimum spanning tree problems.
method Introduces Max-Affine Arithmetic Programs and shows equivalence to neural networks.
result Two combinatorial optimization problems can be solved with polynomial-size neural networks.
Arithmetic Dijkgraaf-Witten theory constructs analogues in Chern-Simons TQFT.
problem Developing arithmetic analogues in Chern-Simons TQFT.
method Constructing arithmetic analogues of Chern-Simons 1-cocycle, prequantization bundle, and Chern-Simons functional.
result Decomposition and gluing formulas for arithmetic Chern-Simons invariants and arithmetic Dijkgraaf-Witten partition functions.
In this article, we prove that every arithmetic locally symmetric orbifold of classical type without Euclidean or compact factors has arbitrarily long arithmetic progressions in its primitive length spectrum. Moreover, we show the stronger property that every primitive length occurs in arbitrarily long arithmetic progr…
New method constructs non-arithmetic hyperbolic orbifolds from complex arithmetic ball quotients.
problem Creating non-arithmetic lattices in projective orthogonal groups.
method Using anti-holomorphic involutions on complex arithmetic ball quotients, gluing fixed loci along geodesic subspaces.
result Explicit calculation of the volume of constructed non-arithmetic orbifolds.
We show that the non-arithmetic lattices in PO(n,1) of Belolipetsky and Thomson (2011), obtained as fundamental groups of closed hyperbolic manifolds with short systole, are quasi-arithmetic in the sense of Vinberg, and, by contrast, the well-known non-arithmetic lattices of Gromov and Piatetski-Shapiro are not quasi-a…
The paper explores subspaces in hyperbolic lattices and their arithmetic properties.
problem Arithmeticity criterion for hyperbolic lattices and suborbifolds.
method Analysis of totally geodesic suborbifolds and Vinberg's commensurability invariants.
result Arithmeticity of hyperbolic orbifolds is linked to the existence of infinitely many fc-subspaces.
Define an arithmetic variety to be the quotient of a bounded symmetric domain by an arithmetic group. An arithmetic variety is algebraic, and the theorem in question states that when one applies an automorphism of the field of complex numbers to the coefficients of an arithmetic variety the resulting variety is again a…
Geodesics on modular surface yield arithmetic 3-manifolds.
problem Understanding arithmetic properties of modular surfaces.
method Constructing geodesics and analyzing their lifts.
result Complements of canonical lifts are arithmetic 3-manifolds.
New property identifies arithmetic lattices from nonuniform lattices.
problem Characterizing arithmetic lattices among nonuniform lattices.
method Introduced Bounded Clustering (B-C) property.
result B-C property uniquely identifies arithmetic lattices.
New proof shows maximal arithmetic groups are finite.
problem Finiteness of maximal arithmetic reflection groups.
method Arithmetic Margulis lemma without automorphic forms.
result Finiteness of maximal arithmetic reflection groups proven.
We study the arithmeticity of the Couwenberg-Heckman-Looijenga lattices in PU(n,1), and show that they contain a non-arithmetic lattice in PU(3,1) which is not commensurable to the non-arithmetic Deligne-Mostow lattice in PU(3,1).
Develops arithmetic PDE geometry using Fermat quotients.
problem Creating an arithmetic PDE analogue of Riemannian geometry.
method Using Fermat quotients and Frobenius elements in the absolute Galois group of a p-adic field. result Existence and uniqueness of geodesics and connections proved.
Study general hyperbolic gluings, proving quasi-arithmeticity of building blocks.
problem Proving quasi-arithmeticity of building blocks in hyperbolic gluings.
method Generalized gluings of hyperbolic orbifolds, proving quasi-arithmeticity.
result Building blocks of quasi-arithmetic gluings must also be quasi-arithmetic.
The study of systoles in arithmetic hyperbolic manifolds.
problem Understanding the systoles of arithmetic hyperbolic manifolds.
method Construction and analysis of arithmetic hyperbolic manifolds.
result Explicit bounds on volumes and systoles of arithmetic hyperbolic manifolds.
We prove that each lower-dimensional face of a quasi-arithmetic Coxeter polytope, which happens to be itself a Coxeter polytope, is also quasi-arithmetic. We also provide a sufficient condition for a codimension 1 face to be actually arithmetic, as well as a few computed examples.
A new deep learning method using Boolean logic reduces training and inference energy.
problem High computational and energy costs in deep learning training and inference.
method Introduces Boolean weights and inputs for efficient training using Boolean logic.
result Achieves full-precision accuracy in ImageNet classification and surpasses state-of-the-art results in semantic segmentation.
We explore hybrid subgroups of certain non-arithmetic lattices in PU(2,1). We show that all of Mostow's lattices are virtually hybrids; moreover, we show that some of these non-arithmetic lattices are hybrids of two non-commensurable arithmetic lattices in PU(1,1).
New research shows certain arithmetic lattices can't be LERF.
problem Determining if arithmetic lattices are LERF.
method Analyzing trialitarian arithmetic lattices in PSO7,1(R). result Trialitarian arithmetic lattices in PSO7,1(R) are not LERF. Arithmetic spaces' thin parts are negligible, impacting Betti numbers.
problem Understanding the structure of arithmetic locally symmetric spaces.
method Analyzing thin parts and deducing asymptotic results on Betti numbers.
result Arithmetic spaces' thin parts are negligible, impacting Betti numbers.
We apply G. Prasad's volume formula for the arithmetic quotients of semi-simple groups and Bruhat-Tits theory to study the covolumes of arithmetic subgroups of SO(1,n). As a result we prove that for any even dimension n there exists a unique compact arithmetic hyperbolic n-orbifold of the smallest volume. We give a for…
Develops Kleinian Sphere Packings and Bugs, proving their arithmetic origins.
problem Understanding sphere packings and their arithmetic origins in various dimensions.
method Introduces Kleinian Sphere Packings and Bugs, extending Arithmeticity Theorem.
result Kleinian packings and Bugs come from Q-arithmetic lattices of simplest type.
Conditions for flat manifolds as cusp cross-sections in arithmetic hyperbolic manifolds.
problem Determining when a flat manifold can be a cusp cross-section in arithmetic hyperbolic manifolds.
method Analyzing rational representations of holonomy groups and quasi-arithmetic manifolds.
result Conditions for a flat manifold to appear as a cusp cross-section in every commensurability class of arithmetic hyperbolic manifolds.
Persistent homology reveals a topological signature of grokking in neural networks.
problem Understanding how neural networks learn and generalize from modular arithmetic tasks.
method Persistent homology on point clouds derived from embedding matrices of models trained on modular arithmetic.
result A sharp increase in first homology persistence indicates grokking, with a dominant long-lived topological feature and structured secondary features.