On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “width optimization”

Urysohn Width and Surgeriesmath.MG

Study bounds Urysohn width of manifolds under surgeries.

problem Bounding Urysohn width of manifolds after surgeries.

method Analyzes connected sums and universal covers, applies to general surgeries.

result Optimal constants in estimates of width bounds are shown.

Urysohn width manifolds surgeries connected sums universal covers

2026-02-17abs ↗pdf ↗

On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfercs.LG

New optimizers control network width scaling, improving stability and transfer across different model sizes.

problem Designing stable optimizers for networks of varying widths.

method Interpreting optimizers as steepest descent under mean-normalized operator norms, enabling layerwise composability and width-independent bounds.

result New optimizers like row normalization and column normalization provide stable learning-rate transfer across different model widths.

Neural Optimizers Width Scaling Operator Norms Steepest Descent Layerwise Composability

2026-03-10abs ↗pdf ↗

Joslim: Joint Widths and Weights Optimization for Slimmable Neural Networkscs.LG

Joslim optimizes both width and weight configurations for slimmable neural networks, improving model efficiency.

problem Optimizing both width and weight configurations for slimmable neural networks to improve efficiency.

method Proposes a general framework for joint optimization of width configurations and weights, and introduces Joslim algorithm.

result Improves model efficiency by up to 1.7% in top-1 accuracy on the ImageNet dataset.

slimmable networks width optimization weight optimization neural networks model efficiency

2020-07-23abs ↗pdf ↗

The Depth-to-Width Interplay in Self-Attentioncs.LG

Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.

problem Understanding the optimal balance between depth and width in self-attention models.

method Theoretical predictions and empirical ablations on networks of varying depths and widths.

result An optimal width of 30K is recommended for a 1-Trillion parameter network, marking a significant width for self-attention models.

self-attention depth width network efficiency parameter size

2020-06-22abs ↗pdf ↗

Spectral Condition for

μ

P under Width-Depth Scalingcs.LG

Unified spectral framework for μP under joint width-depth scaling.

problem Challenges in stable feature learning and HP transfer for width-depth scaled models.

method Developed a simple and unified spectral framework for μP under joint width-depth scaling.

result Unified and generalized μP formulation for practical architectures with multi-transformation branches.

μP spectral framework width-depth scaling residual networks language models

2026-02-28abs ↗pdf ↗

The p-widths of a surfacemath.DG

New insights into $p$ -widths of surfaces, proving optimality and calculating constants.

problem Understanding

p

-widths of surfaces and their relationship to geodesics.

method Analyzing min-max sequences of minimal submanifolds and sweepouts of surfaces.

result Optimal sweepouts of the round two-sphere by

\lfloor \sqrt{p} floor

great circles.

$p$-widths Riemannian manifolds min-max sequences geodesics sweepouts

2021-07-24abs ↗pdf ↗

Sensitivity-Aware Mixed-Precision Quantization and Width Optimization of Deep Neural Networks Through Cluster-Based Tree-Structured Parzen Estimationcs.LG

This work optimizes neural network bit-width and layer-width for efficiency.

problem Efficient optimization of deep neural networks for reduced size and computational demands.

method Cluster-based tree-structured Parzen estimator for surrogate modeling, Hessian-based pruning for parameter reduction.

result 20% decrease in model size with 12x reduction in search time compared to existing methods.

deep learning neural networks quantization width optimization surrogate modeling

2023-08-12abs ↗pdf ↗

Bayesian Bits: Unifying Quantization and Pruningcs.LG

Bayesian Bits unifies quantization and pruning through gradient optimization.

problem Joint mixed precision quantization and pruning for efficient neural networks.

method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.

result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.

Bayesian Bits Quantization Pruning Gradient Optimization Mixed Precision

2020-05-14abs ↗pdf ↗

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scalingcs.LG

Large learning rates work surprisingly well in standard parameterization, contrary to theory.

problem Theoretical limits of large learning rates do not match practical network behavior.

method Fine-grained analysis of learning rates and network behavior under cross-entropy loss.

result There are two distinct sub-regimes of unstable learning rates, with a controlled divergence regime where features continue to evolve.

large learning rates standard parameterization cross-entropy loss finite-width phenomena infinite-width limits

2025-05-28abs ↗pdf ↗

A Proof of Learning Rate Transfer under

μ

Pstat.ML

Proof of learning rate transfer in MLPs with $μ$ P parameterization.

problem Understanding and optimizing learning rates in neural networks with different parameterizations.

method Theoretical analysis and empirical validation of learning rate transfer in MLPs with

μ

P, SP, and NTP parameterizations.

result The optimal learning rate converges to a non-zero constant as width goes to infinity under

μ

P, explaining learning rate transfer.

learning rate transfer neural networks parameterization infinite-width limit learning rate optimization

2025-11-03abs ↗pdf ↗

Asymptotics of Wide Convolutional Neural Networkscs.LG

Wide CNNs outperform infinite width networks, revealing scaling laws.

problem Understanding the performance difference between finite and infinite width convolutional networks.

method Diagrammatic approach to derive asymptotic width dependence for various quantities.

result The difference in performance between finite and infinite width models vanishes at a definite rate with respect to model width.

wide neural networks convolutional neural networks asymptotics scaling laws skip connections

2020-08-19abs ↗pdf ↗

The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Studycs.LG

We investigate how the final parameters found by stochastic gradient descent are influenced by over-parameterization. We generate families of models by increasing the number of channels in a base network, and then perform a large hyper-parameter search to study how the test error depends on learning rate, batch size, a…

2019-05-09abs ↗pdf ↗

Band Width Estimates and Rigidity of Manifolds with Negative Curvaturemath.DG

Establish optimal Lipschitz lower bounds for functions on manifolds with negative curvature, revealing interplay between width, boundary area, and topology.

problem Width estimates and rigidity of manifolds with negative curvature

method Gromov's μ-bubble method

result Sharp lower bound for boundary area in hyperbolic bands

width rigidity negative curvature Gromov's μ-bubble hyperbolic band

2026-06-23abs ↗pdf ↗

Fisher Width: A Geometric Measure of Complexity on Statistical Manifoldscs.LG

Fisher width is a geometric measure of complexity on statistical manifolds.

problem Complexity measures on statistical manifolds

method Introducing Fisher width as a Fisher-geometric analogue of Gaussian width

result Fisher width retains key structural features of Gaussian width while capturing anisotropic geometric effects

Fisher width statistical manifolds complexity measures curvature learning theory

2026-06-16abs ↗pdf ↗

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limitstat.ML

Residual networks with depthwise hyperparameter scaling transfer optimal hyperparameters across width and depth.

problem The challenge of hyperparameter tuning in deep learning, especially for large models.

method Combining

μ

P parameterization with residual networks having a residual branch scale of

1/\sqrt{ ext{depth}}

result Optimal hyperparameters transfer across width and depth in residual networks trained with this parameterization.

hyperparameter tuning residual networks depthwise scaling mean field theory neural network dynamics

2023-09-28abs ↗pdf ↗

Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networksstat.ML

Sharp feature transitions revealed in extensive-width networks.

problem Learning hierarchical features from noisy queries in large networks.

method Information-theoretic analysis and heuristic decoupling argument.

result Sequential phase transitions in feature learnability and effective width.

feature learning Bayes-optimal scaling laws high-dimensional networks

2026-05-11abs ↗pdf ↗

Robust Layerwise Scaling Rules by Proper Weight Decay Tuningcs.LG

We introduce a new weight-decay scaling rule to maintain sublayer gains across different widths in modern scale-invariant architectures.

problem In modern scale-invariant architectures, training quickly enters a steady state where normalization layers create backward scale sensitivity, degrading learning-rate transfer.

method We introduce a weight-decay scaling rule for AdamW that preserves sublayer gain across widths by equalizing the effective learning rate.

result Our empirical weight-decay scaling rule

λ_2\propto \sqrt{d}

approximately keeps sublayer gains width invariant, enabling zero-shot transfer of learning rate and weight decay.

weight decay sublayer gain learning rate scale-invariant architectures AdamW

2025-10-17abs ↗pdf ↗

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Sizecs.LG

New analysis shows optimal embedding learning rate depends on vocabulary size, not just model width.

problem Optimal learning rate for language model embeddings is not well understood, especially with large vocabularies.

method Theoretical analysis of training dynamics, interpolation between

μ

P and LV regimes.

result Optimal embedding learning rate scales as

Θ(\sqrt{width})

in the LV regime, not

Θ(width)

μ

P predicts.

language models learning rate vocabulary size embedding dimension training dynamics

2025-06-17abs ↗pdf ↗

Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcs.LG

This paper optimizes ReLU networks for approximating Hölder continuous functions.

problem Optimizing the approximation rate of ReLU networks in terms of width and depth.

method Constructive proof of ReLU networks' approximation power with specific width and depth constraints.

result Optimal approximation rate of ReLU networks with width and depth constraints.

ReLU networks approximation rate width and depth Hölder continuous functions logarithmic factor

2021-02-28abs ↗pdf ↗

Deep Network Approximation for Smooth Functionscs.LG

Deep ReLU networks can approximate smooth functions nearly optimally.

problem Approximating smooth functions with deep neural networks.

method Using Taylor expansions and deep ReLU network approximations, the paper establishes optimal approximation error bounds.

result Deep ReLU networks of width and depth

\mathcal{O}(N\ln N)

and

\mathcal{O}(L\ln L)

can approximate

f\in C^s([0,1]^d)

with an error

\mathcal{O}(\|f\|_{C^s([0,1]^d)}N^{-2s/d}L^{-2s/d})

Deep Learning Recurrent Neural Networks Polynomial Approximation Smooth Functions Approximation Error

2020-01-09abs ↗pdf ↗

On the Benefit of Width for Neural Networks: Disappearance of Bad Basinscs.LG

Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the difference between wide and narrow networks. In this work, we prove that from narrow to wide networks, there is a phase transition from havi…

2018-12-28abs ↗pdf ↗

Provable Benefits of Sinusoidal Activation for Modular Additioncs.LG

Sine activation functions enable two-layer neural networks to learn modular addition more efficiently.

problem Learning modular addition with two-layer neural networks.

method Introduced and analyzed sine activation functions, providing theoretical and empirical evidence.

result Sine activation functions allow for constant-width network realizations of modular addition, whereas ReLU networks require linear width scaling.

modular addition two-layer neural networks sine activation expressivity gap generalization bound

2025-11-28abs ↗pdf ↗

How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterizationcs.LG

Wide neural networks can benefit from multi-task learning in their infinite-width limit.

problem The generalization behavior of wide neural networks in multi-task learning settings.

method Optimizing wide ReLU neural networks with L2-regularization promotes multi-task learning in the infinite-width limit.

result An exact quantitative characterization of multi-task learning in the infinite-width limit of wide ReLU neural networks.

multi-task learning wide neural networks infinite-width limit ReLU networks L2 regularization

2021-12-31abs ↗pdf ↗

Bayes-optimal Learning of Deep Random Networks of Extensive-widthstat.ML

Bayes-optimal learning of deep random networks with Gaussian weights is studied.

problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.

method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.

result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.

Bayes-optimal learning deep neural networks random Gaussian weights ridge regression kernel regression

2023-02-01abs ↗pdf ↗

Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamicscs.LG

New framework analyzes deep learning optimization with finite width networks, revealing generalization gaps and excess risks.

problem Analyzing generalization error of deep learning with finite width networks.

method Formulating neural network training as transportation map estimation and analyzing via infinite dimensional Langevin dynamics.

result Achieves fast learning rate and minimax optimal rates for classification and regression problems.

deep learning generalization error transportation map Langevin dynamics neural networks

2020-07-11abs ↗pdf ↗

Minimum Width for Deep, Narrow MLP: A Diffeomorphism Approachcs.LG

New approach finds minimum width for deep, narrow MLPs.

problem Finding the minimum width for deep, narrow MLPs to approximate continuous functions.

method Proposes a framework to simplify finding minimum width into determining a geometrical function

w(d_x, d_y)

based on input and output dimensions.

result Proves that

w(d_x, d_y)

equals the optimal minimum width for deep, narrow MLPs to achieve universality.

Deep Learning MLP Minimum Width Diffeomorphism Universal Approximation

2023-08-30abs ↗pdf ↗

A comparison of the Almgren-Pitts and the Allen-Cahn min-max theorymath.DG

This research proves that two min-max theories for hypersurfaces are equivalent.

problem Comparing two min-max theories for hypersurfaces.

method Developed and proved the equivalence of Almgren-Pitts and Allen-Cahn min-max theories.

result The Almgren-Pitts widths and Allen-Cahn widths are equivalent.

min-max theory hypersurfaces Almgren-Pitts Allen-Cahn equivalence

2020-04-10abs ↗pdf ↗

Understanding the Mechanisms of Fast Hyperparameter Transfercs.LG

New framework explains fast transfer of hyperparameters across model scales.

problem Understanding and optimizing hyperparameters for large-scale models.

method Developed a conceptual framework for HP transfer across scale, showing fast transfer is equivalent to useful transfer for compute-optimal grid search.

result Fast transfer of hyperparameters is equivalent to useful transfer for compute-optimal grid search, offering asymptotic computational advantage.

hyperparameter optimization scale-aware HPs fast transfer compute efficiency Maximal Update Parameterization

2025-12-28abs ↗pdf ↗

Global Optimality of Elman-type RNN in the Mean-Field Regimestat.ML

Elman-type RNNs converge to globally optimal solutions in the mean-field regime.

problem Optimizing feature learning in wide RNNs.

method Analysis of gradient descent dynamics and mean-field limits.

result Fixed points of infinite-width dynamics are globally optimal.

Elman RNN Recurrent Neural Networks Gradient Descent Mean-Field Optimality

2023-03-12abs ↗pdf ↗

Stochastic Neural Networks with Infinite Width are Deterministiccs.LG

Stochastic neural networks with infinite width become deterministic, reducing training variance.

problem Understanding how stochasticity in neural networks affects learning and regularization.

method Theoretical analysis of stochastic neural networks with infinite width.

result As the width of an optimized stochastic neural network increases, its predictive variance on the training set decreases to zero.

stochastic neural networks infinite width predictive variance regularization dropout

2022-01-30abs ↗pdf ↗

Maximum Likelihood Bounded Tree-Width Markov Networkscs.LG

Chow and Liu (1968) studied the problem of learning a maximumlikelihood Markov tree. We generalize their work to more complexMarkov networks by considering the problem of learning a maximumlikelihood Markov network of bounded complexity. We discuss howtree-width is in many ways the appropriate measure of complexity and…

2013-01-10abs ↗pdf ↗

Quantitative CLTs in Deep Neural Networkscs.LG

Quantitative CLTs show neural network distributions converge to Gaussian as width increases.

problem Understanding the distribution of fully connected neural networks with random weights and biases.

method Analyzing the distribution of a fully connected neural network with random Gaussian weights and biases, proving quantitative bounds on normal approximations.

result The distance between a random fully connected network and the corresponding infinite width Gaussian process scales like

n^{-γ}

for

γ>0

quantitative CLTs deep neural networks random Gaussian weights finite width Gaussian process

2023-07-12abs ↗pdf ↗

What graph neural networks cannot learn: depth vs widthcs.LG

This paper studies the expressive power of graph neural networks falling within the message-passing framework (GNNmp). Two results are presented. First, GNNmp are shown to be Turing universal under sufficient conditions on their depth, width, node attributes, and layer expressiveness. Second, it is discovered that GNNm…

2019-07-06abs ↗pdf ↗

Bayesian Interpolation with Deep Linear Networksstat.ML

Bayesian linear networks reveal optimal depth and width trade-offs.

problem Understanding how depth, width, and dataset size affect model quality in linear networks.

method Zero noise Bayesian inference with Gaussian weight priors and mean squared error.

result Optimal predictions at infinite depth and maximized Bayesian model evidence at infinite depth.

Bayesian inference Linear networks Depth-width trade-offs Model selection Effective depth

2022-12-29abs ↗pdf ↗

An Algorithmic Definition of Gabai Widthmath.GT

We define the Wirtinger width of a knot. Then we prove the Wirtinger width of a knot equals its Gabai width. The algorithmic nature of the Wirtinger width leads to an efficient technique for establishing upper bounds on Gabai width. As an application, we use this technique to calculate the Gabai width of approximately …

2019-12-04abs ↗pdf ↗

Mapping the Schedule x Bit-Width Boundary in Sub-100M Quantisation-Aware Trainingcs.LG

Study finds optimal learning rate schedules for sub-100M quantization-aware training across bit-widths.

problem Optimal learning rate schedules for quantization-aware training depend on bit-width.

method Factorial grid testing over bit-width, warmdown fraction, LR magnitude, model size, and seed.

result INT6 QAT requires a different schedule than higher-precision training, falsifying the primary hypothesis.

quantization-aware training learning rate schedules sub-100M models bit-width warmdown fraction

2026-05-25abs ↗pdf ↗

An Empirical Analysis of the Advantages of Finite- v.s. Infinite-Width Bayesian Neural Networksstat.ML

Empirical study compares finite- and infinite-width BNNs, revealing performance differences under model mismatch.

problem Comparing BNNs with different widths due to conflicting model properties and inference intractability.

method Empirical comparison of finite- and infinite-width BNNs, analyzing performance under model mismatch.

result Increasing width can hurt BNN performance when the model is mis-specified, and finite-width BNNs generalize better under model mismatch.

Bayesian neural networks finite-width infinite-width model mismatch frequency spectrum

2022-11-16abs ↗pdf ↗

Optimal Neural Network Approximation for High-Dimensional Continuous Functionscs.LG

Optimized neural network approximates high-dimensional functions with minimal parameters.

problem Achieving optimal approximation of high-dimensional continuous functions with minimal parameters.

method Developed a neural network with a specific activation function and architecture to achieve super approximation property.

result A composed network with at most 10889d + 10887 nonzero parameters achieves super approximation property, suggesting optimality in parameter growth.

Neural Networks Continuous Functions Super Approximation Elementary Universal Activation Kolmogorov Superposition Theorem

2024-09-04abs ↗pdf ↗

Bayesian Deep Learning Hyperparameter Search for Robust Function Mapping to Polynomials with Noisecs.LG

This paper explores BDL hyperparameters for robust polynomial mapping with noise.

problem Designing BDL hyperparameters for robust function mapping with uncertainty quantification.

method Mapping Bayesian connectionist representations to polynomials of varying orders and noise types.

result Optimal network depth and ensemble size for prediction and uncertainty quantification.

Bayesian Deep Learning Hyperparameter Search Polynomial Mapping Noise Contamination Uncertainty Quantification

2021-06-23abs ↗pdf ↗

The universal approximation power of finite-width deep ReLU networkscs.LG

We show that finite-width deep ReLU neural networks yield rate-distortion optimal approximation (Bölcskei et al., 2018) of polynomials, windowed sinusoidal functions, one-dimensional oscillatory textures, and the Weierstrass function, a fractal function which is continuous but nowhere differentiable. Together with thei…

2018-06-05abs ↗pdf ↗

The Isospectral Problem for p-widths: An Application of Zoll Metricsmath.DG

The isospectral problem for p-widths is solved using Zoll metrics on S^2.

problem Determine if a Riemannian manifold is uniquely determined by its p-widths.

method Construct counterexamples on S^2 using Zoll metrics and properties of geodesic p-widths.

result Many counterexamples exist on S^2, showing uniqueness is not guaranteed.

isospectral problem p-widths Zoll metrics geodesic widths S^2

2024-05-30abs ↗pdf ↗

Links, bridge number, and width treesmath.GT

Width trees link link invariants and bridge number.

problem Understanding link invariants through geometric structures.

method Associate width trees to links and use their geometric properties to bound link invariants.

result Width trees uniquely realize certain link invariants under specific conditions.

width trees link invariants bridge number geometric structures

2020-05-25abs ↗pdf ↗

Les Houches Lectures on Deep Learning at Large & Infinite Widthstat.ML

Lectures on deep learning properties in infinite and large-width networks.

problem Understanding deep neural networks in extreme width conditions.

method Analysis of random deep neural networks, connections to linear models, kernels, and Gaussian processes, perturbative and non-perturbative treatments.

result Properties and behaviors of deep neural networks in the infinite-width limit and large-width regime.

deep learning neural networks infinite width large width statistical physics

2023-09-04abs ↗pdf ↗

The p-widths of the Hemispheremath.DG

Computed p-widths for hemisphere, first for manifolds with boundary.

problem Finding p-widths for manifolds with boundary.

method Computed p-widths for the hemisphere.

result First known p-widths for a manifold with boundary.

p-widths hemisphere manifolds boundary

2026-03-18abs ↗pdf ↗

The p-widths of a polygonmath.DG

Polygon $p$ -widths are found via billiard trajectories.

problem Finding

p

-widths of polygons.

method Proved via billiard trajectories and computed specific cases.

result Polygon

p

-widths are achieved by billiard trajectories.

polygon billiard trajectories p-widths spectrum of Laplacian

2025-05-05abs ↗pdf ↗

On C

^2

-smooth Surfaces of Constant Widthmath.DG

A number of results for C $^2$ -smooth surfaces of constant width in Euclidean 3-space ${\mathbb{E}}^3$ are obtained. In particular, an integral inequality for constant width surfaces is established. This is used to prove that the ratio of volume to cubed width of a constant width surface is reduced by shrinking it along…

2007-04-24abs ↗pdf ↗

The p-widths of

RP^2

math.DG

Computed p-widths for real projective plane.

problem Calculating p-widths for real projective plane.

method Standard metric used to compute p-widths.

result Computed p-widths for real projective plane.

p-widths real projective plane standard metric

2025-01-23abs ↗pdf ↗

Adaptive kernel predictors from feature-learning infinite limits of neural networkscs.LG

Adaptive kernels from neural networks improve model performance.

problem Improving neural network performance through adaptive kernels.

method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.

result Adaptive kernels achieve lower test loss compared to traditional kernels.

neural networks kernel predictors infinite width feature learning gradient flow

2025-02-11abs ↗pdf ↗