Neural Power Unit (NPU) learns arbitrary power functions on real numbers.
problem Neural Networks struggle with generalizing beyond seen data and arithmetic operations.
method Introduces Neural Power Unit (NPU) that operates on real numbers and learns arbitrary power functions.
result NPU outperforms competitors in accuracy and sparsity on arithmetic datasets and discovers governing equations from data.
Paper tackles division difficulty, proposing new methods to improve accuracy.
problem Division is the most challenging arithmetic operation for both humans and computers.
method Proposes two novel approaches: Neural Reciprocal Unit (NRU) and Neural Multiplicative Reciprocal Unit (NMRU), and improves an existing division module.
result Improves division accuracy from 70.2% to 91.6%.
Better neural arithmetic logic units improve cell counting model generalization.
problem Neural networks struggle with high cell counts outside training data range.
method Introduced Neural Arithmetic Logic Units (NALU) for arithmetic operations in existing architectures.
result Improved cell counting accuracy for higher numeric ranges with better generalization.
Geodesics on modular surface yield arithmetic 3-manifolds.
problem Understanding arithmetic properties of modular surfaces.
method Constructing geodesics and analyzing their lifts.
result Complements of canonical lifts are arithmetic 3-manifolds.
Stock prediction is a topic undergoing intense study for many years. Finance experts and mathematicians have been working on a way to predict the future stock price so as to decide to buy the stock or sell it to make profit. Stock experts or economists, usually analyze on the previous stock values using technical indic…
Rodent identifies ODEs from trajectories without needing basis functions.
problem Identifying the generating ODE from observed system trajectories.
method Uses Neural Arithmetic Units and sparsification techniques (VAE and ARD) to minimize state size and non-zero parameters.
result Learned models represent a manifold of ODEs including harmonic signals and Lotka-Volterra systems.
Study C-Fuchsian subgroups of non-arithmetic lattices.
problem Understand structure and fundamental domains of C-Fuchsian subgroups. method General procedure to analyze structure and show fundamental domains lie on a complex geodesic.
result Fundamental domains of C-Fuchsian subgroups lie on a complex geodesic homeomorphic to the unit disk. MC-LSTM extends LSTM to conserve mass in neural networks.
problem Conservation laws in real-world systems.
method Extending LSTM's inductive bias to conserve mass.
result MC-LSTM sets new state-of-the-art for predicting peak flows.
Survey of Weber's class number problem and related topics.
problem Weber's class number problem and its variants.
method Arithmetic topology, units, generalized Pell's equation, p-adic limits of class numbers. result Numerical investigation of class numbers in p-adic towers for knots and elliptic curves. This paper shows neural networks can solve complex graph problems efficiently.
problem Solving exact maximum flow computation and minimum spanning tree problems.
method Introduces Max-Affine Arithmetic Programs and shows equivalence to neural networks.
result Two combinatorial optimization problems can be solved with polynomial-size neural networks.
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
The use of low-precision fixed-point arithmetic along with stochastic rounding has been proposed as a promising alternative to the commonly used 32-bit floating point arithmetic to enhance training neural networks training in terms of performance and energy efficiency. In the first part of this paper, the behaviour of …
New bounds on diameters and generators for specific lattices and graphs.
problem Finding bounds on diameters and generators for arithmetic lattices and Ramanujan graphs.
method Analyzing arithmetic lattices from Eichler orders in quaternion algebras, applying techniques to definite quaternion algebras.
result Bounds on diameters and generators for arithmetic lattices and Ramanujan graphs.
Transformer models improve arithmetic accuracy with number decomposition.
problem Transformer models struggle with arithmetic operations without decomposition.
method Fine-tuning models with a pipeline that decomposes numbers into units, tens, etc.
result Accuracy increased by 63% in five-digit addition tasks.
Recursive Feature Machines show grokking in modular arithmetic without neural networks.
problem Grokking in modular arithmetic tasks.
method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).
result RFM and neural networks learn block-circulant features to solve modular arithmetic.
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
Paper proves existence of a CMC hypertorus in 4D sphere using numerical methods.
problem Proving the existence of a constant mean curvature (CMC) hypertorus in \(S^4\).
method Employed the round Taylor method with rational arithmetic and the Poincare-Miranda theorem.
result Existence of a constant mean curvature (CMC) hypertorus in \(S^4\).
We consider the analogue of Hurwitz curves, smooth projective curves C of genus g≥2 that realize equality in the Hurwitz bound ∣Aut(C)∣≤84(g−1), to smooth compact quotients S of the unit ball in C2. When S is arithmetic, we show that ∣Aut(S)∣≤288e(S), where $e(S…
There is a recent interest in neural network (NN)-based communication algorithms which have shown to achieve (beyond) state-of-the-art performance for a variety of problems or lead to reduced implementation complexity. However, most work on this topic is simulation based and implementation on specialized hardware for f…
Study of Alexander polynomials of torus knots and links, showing zeros equidistribute on unit circle.
problem Analyzing asymptotic behavior and distribution of zeros of Alexander polynomials of torus knots.
method Equidistribution analysis, moment sequence, Iwasawa theory, logarithmic Mahler measure.
result Zeros of Alexander polynomials of torus knots and links become equidistributed on the unit circle as p, q → ∞.
Neural networks learn modular arithmetic but not all, extending known solutions to generalize.
problem Neural networks struggle with modular arithmetic, especially for polynomials.
method Developed analytical solutions for MLP networks to learn modular addition and multiplication, then combined these solutions to generalize on arbitrary modular polynomials.
result Neural networks can learn and generalize solutions to modular polynomials, supporting the hypothesis that some polynomials are learnable.
Set risk measures extend traditional risk measures to handle sets of positions.
problem Handling sets of positions with a single capital requirement.
method Developed an axiomatic framework for set risk measures, dual representation through topology and measures.
result Characterized worst-case set risk measures and provided examples.
We prove that every homomorphism from the elementary Chevalley group over a finitely generated unital commutative ring associated with reduced irreducible classical root system of rank at least 2, and ME analogues of such groups, into acylindrically hyperbolic groups has an absolutely elliptic image. This result provid…
Improved math problem solvers using Transformer networks and diverse notations.
problem Challenges in constructing accurate and automatic solvers for math word problems.
method Transformer networks trained to translate math word problems to arithmetic expressions in infix, prefix, and postfix notations. Pre-training on general text corpus to improve performance.
result Significant improvements in accuracy, up to 10% over previous state of the art.
The purpose of this paper is to give presentations for projective S-unit groups of the Hurwitz order in Hamilton's quaternions over the rational field Q. To our knowledge, this provides the first explicit presentations of an S-arithmetic lattice in a semisimple Lie group with S large. In particular, we…
Deep neural networks (DNN) are powerful models for many pattern recognition tasks, yet their high computational complexity and memory requirement limit them to applications on high-performance computing platforms. In this paper, we propose a new method to evaluate DNNs trained with 32bit floating point (float32) accura…
Study explores how neural networks and Transformers learn modular arithmetic with multiple inputs.
problem Understanding how neural networks and Transformers learn modular arithmetic with multiple inputs.
method Analytical characterization of features learned by neural networks and Transformers, focusing on margin maximization and Fourier spectra.
result Neural networks and Transformers require a minimum neuron count of \( m \geq 2^{2k-2} \cdot (p-1) \) to solve modular addition problems with \( k \) inputs and modulus \( p \).
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
problem Reducing multiplication complexity in low-resolution neural networks.
method Adapting neural networks to use low-resolution (8-bit) additions in accumulators, with a cyclic activation layer and overflow penalty regularizer.
result Achieves comparable classification accuracy to 32-bit counterparts using low-resolution additions.
Efficient Winograd convolution for INT8 networks using RNS.
problem Difficulty in applying Winograd algorithm to low-precision quantized networks.
method Extends Winograd algorithm to Residue Number System (RNS) for efficient INT8 convolution.
result Arithmetic complexity reduction up to 7.03x with performance improvement up to 2.30x-4.69x.
Deep neural networks have achieved impressive supervised classification performance in many tasks including image recognition, speech recognition, and sequence to sequence learning. However, this success has not been translated to applications like question answering that may involve complex arithmetic and logic reason…
Degree-Quant improves GNN efficiency by quantizing them without losing accuracy.
problem Efficiency of graph neural networks at inference time.
method Architecturally-agnostic method, Degree-Quant, for quantizing GNNs.
result Degree-Quant trained models perform as well as full-precision models and achieve up to 26% gains.
Study on hidden units in finite Bayesian neural networks and their tail properties.
problem Understanding the behavior of hidden units in finite Bayesian neural networks.
method Introduced a generalized Weibull-tail property to describe hidden units tails.
result Unit priors become heavier-tailed going deeper, providing insights into finite Bayesian neural networks.
Study examines dependence properties of Bayesian neural network units in finite-width networks.
problem Understanding dependence properties of hidden units in practical finite-width Bayesian neural networks.
method Theoretical analysis and empirical evaluation of depth and width impacts.
result Hidden units in finite-width Bayesian neural networks are dependent, contrary to the infinite-width limit assumption.
Deep neural networks struggle with numerical instability during training.
problem Numerical instability in gradient descent training of deep neural networks.
method Analysis of floating-point arithmetic and gradient descent in ReLU neural networks.
result It is highly unlikely for ReLU networks to maintain a superlinear number of affine pieces during training.
We establish basic geometric and topological properties of Thurston's Master Teapot and the Thurston set for superattracting unimodal self-maps of intervals. In particular, the Master Teapot is connected, contains the unit cylinder, and its intersection with a set D×{c} grows monotonically with c.…
Faces of quasi-arithmetic Coxeter polytopes are also quasi-arithmetic.
problem Characterizing faces of quasi-arithmetic Coxeter polytopes.
method Proof of quasi-arithmetic property of faces and sufficient condition for arithmetic faces.
result Lower-dimensional faces of quasi-arithmetic Coxeter polytopes are quasi-arithmetic.
In this article, we investigate when the set of primitive geodesic lengths on a Riemannian manifold have arbitrarily long arithmetic progressions. We prove that in the space of negatively curved metrics, a metric having such arithmetic progressions is quite rare. We introduce almost arithmetic progressions, a coarsific…
Develops arithmetic PDE geometry concepts like curvature and cohomology.
problem Creating a geometry framework for arithmetic PDEs.
method Introducing arithmetic analogues of Levi-Civita and Chern connections, then developing curvature and characteristic classes.
result Arithmetic analogues of curvature and characteristic classes have been developed.
We investigate deep Bayesian neural networks with Gaussian weight priors and a class of ReLU-like nonlinearities. Bayesian neural networks with Gaussian priors are well known to induce an L2, "weight decay", regularization. Our results characterize a more intricate regularization effect at the level of the unit activat…
Quantized neural networks can represent all fixed-point functions under certain conditions.
problem Expressive power of quantized neural networks under fixed-point arithmetic.
method Analyzing necessary and sufficient conditions for quantized networks to represent all fixed-point functions.
result Various popular activation functions satisfy the sufficient condition for representing all fixed-point functions.
New geometric invariant limits the number of semi-arithmetic groups.
problem Understanding the structure of semi-arithmetic Fuchsian groups.
method Introducing a new geometric invariant called stretch and using the arithmetic Margulis lemma.
result There exist only finitely many conjugacy classes of semi-arithmetic groups with bounded arithmetic dimension, stretch, and coarea.
Neural networks approximate unit spheres as polytopes.
problem Approximating unit spheres with neural networks.
method Using ReLU activation in neural networks to generate polytopes.
result Neural networks can approximate unit spheres as polytopes.
Course on arithmetic lattices at EPFL.
problem Understanding arithmetic lattices.
method Introductory course on arithmetic lattices.
result Introduction to arithmetic lattices.
DPA aligns LLMs with multi-objective rewards for diverse user preferences.
problem Fine-grained control over LLMs for diverse user needs.
method Integrates multi-objective reward modeling and directional preference control.
result DPA offers better performance trade-offs and intuitive user control over LLM generation.
Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to car…
Over-parameterized models can memorize noisy labels and still generalize well, revealing a hidden structure.
problem Understanding how over-parameterized models can simultaneously memorize noisy labels and generalize well.
method Investigated through modular arithmetic tasks with label noise using two-layer neural networks.
result Over-parameterized models can achieve near-perfect test accuracy with 80% label noise by extracting an internal generalization structure.
Paper shows non-arithmetic surface with unique geometric property.
problem Non-arithmetic surfaces with unique geometric properties.
method Example of a non-arithmetic surface with marked length variety rigidity.
result Found a non-arithmetic surface with marked length variety rigidity.
New classification of hyperbolic Coxeter prisms.
problem Classifying hyperbolic Coxeter prisms.
method Determine which prisms are quasi-arithmetic or arithmetic.
result New insights into commensurability and systoles of associated orbifolds.