On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “large width”

Les Houches Lectures on Deep Learning at Large & Infinite Widthstat.ML

Lectures on deep learning properties in infinite and large-width networks.

problem Understanding deep neural networks in extreme width conditions.

method Analysis of random deep neural networks, connections to linear models, kernels, and Gaussian processes, perturbative and non-perturbative treatments.

result Properties and behaviors of deep neural networks in the infinite-width limit and large-width regime.

deep learning neural networks infinite width large width statistical physics

2023-09-04abs ↗pdf ↗

Large-width asymptotics for ReLU neural networks with

α

-Stable initializationscs.LG

This paper studies large-width asymptotics for ReLU neural networks with α-Stable initializations.

problem Characterizing the large-width behavior of ReLU neural networks with α-Stable initializations.

method Analysis of the large-width distributions and training dynamics of ReLU neural networks initialized with α-Stable distributions.

result For ReLU neural networks with α-Stable initializations, the large-width training dynamics achieve zero training error at a linear rate, characterized by a random kernel.

ReLU neural networks α-Stable distributions large-width asymptotics training dynamics random kernel

2022-06-16abs ↗pdf ↗

Width of codimension two knotsmath.GT

We extend the classical definition of {\it width} to higher dimensional, smooth codimension 2 knots and show in each dimension there are knots of arbitrarily large width.

2019-02-19abs ↗pdf ↗

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scalingcs.LG

Large learning rates work surprisingly well in standard parameterization, contrary to theory.

problem Theoretical limits of large learning rates do not match practical network behavior.

method Fine-grained analysis of learning rates and network behavior under cross-entropy loss.

result There are two distinct sub-regimes of unstable learning rates, with a controlled divergence regime where features continue to evolve.

large learning rates standard parameterization cross-entropy loss finite-width phenomena infinite-width limits

2025-05-28abs ↗pdf ↗

The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecs.LG

Wide neural networks can degrade performance, contrary to conventional wisdom.

problem Understanding the limitations of increasing network width in neural networks.

method Using Deep Gaussian Processes to decouple capacity and width, analyzing their effects on representational power and non-Gaussianity.

result Wide neural networks can become less adaptable and more Gaussian, leading to performance degradation.

neural networks width capacity non-Gaussianity Gaussian processes

2021-06-11abs ↗pdf ↗

Urysohn width and macroscopic scalar curvaturemath.DG

The macroscopic version of Urysohn width for scalar curvature is disproven in high dimensions.

problem Disproving the macroscopic version of Gromov's Urysohn width conjecture for scalar curvature.

method Novel estimate on Urysohn width of circle bundles and a new notion of ruling for Riemannian manifolds.

result The macroscopic version of Gromov's Urysohn width conjecture for scalar curvature is false in dimensions four and above.

Urysohn width scalar curvature circle bundles macroscopic Riemannian manifolds

2026-01-21abs ↗pdf ↗

Revisiting the Neural Tangent Kernel: the role of large width and depthcs.LG

This work challenges the Neural Tangent Kernel's role in overparameterized neural networks, especially with large width and depth.

problem The Neural Tangent Kernel's behavior in overparameterized neural networks with large width and depth is unclear.

method Experimental and theoretical analysis of ReLU networks with large width and depth.

result The aggregate norm of hidden neuron deviations does not vanish in infinitely-wide ReLU networks, indicating non-trivial behavior.

Neural Tangent Kernel ReLU networks large width large depth overparameterization

2025-11-10abs ↗pdf ↗

The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initializationstat.ML

Study on neural network initialization with shaped infinite depth-and-width networks.

problem Understanding the distribution of random covariance matrices in shaped infinite-depth-and-width networks.

method Introduced the Neural Covariance SDE to model the distribution of the random covariance matrix.

result Identified the precise scaling of the activation function necessary for a non-trivial limit.

neural networks infinite depth infinite width covariance matrix stochastic differential equation

2022-06-06abs ↗pdf ↗

On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfercs.LG

New optimizers control network width scaling, improving stability and transfer across different model sizes.

problem Designing stable optimizers for networks of varying widths.

method Interpreting optimizers as steepest descent under mean-normalized operator norms, enabling layerwise composability and width-independent bounds.

result New optimizers like row normalization and column normalization provide stable learning-rate transfer across different model widths.

Neural Optimizers Width Scaling Operator Norms Steepest Descent Layerwise Composability

2026-03-10abs ↗pdf ↗

1-Uryson width and coversmath.MG

Study on 1-Uryson width of polyhedra and their covers.

problem Existence of Riemannian polyhedra with bounded 1-Uryson width of covers but unbounded in the polyhedron itself.

method Investigated specific cases of virtually cyclic fundamental groups and Riemannian surfaces, showing bounds on 1-Uryson width.

result For compact polyhedra with virtually cyclic fundamental groups, 1-Uryson width of polyhedron is bounded by that of its universal cover.

1-Uryson width polyhedra universal covers virtual cyclic groups Riemannian surfaces

2025-05-27abs ↗pdf ↗

Quantitative CLTs in Deep Neural Networkscs.LG

Quantitative CLTs show neural network distributions converge to Gaussian as width increases.

problem Understanding the distribution of fully connected neural networks with random weights and biases.

method Analyzing the distribution of a fully connected neural network with random Gaussian weights and biases, proving quantitative bounds on normal approximations.

result The distance between a random fully connected network and the corresponding infinite width Gaussian process scales like

n^{-γ}

for

γ>0

quantitative CLTs deep neural networks random Gaussian weights finite width Gaussian process

2023-07-12abs ↗pdf ↗

Width, Largeness and Index Theorymath.DG

The paper explores index theory for Dirac operators to understand scalar curvature properties.

problem Understanding scalar curvature properties in Riemannian bands and manifolds.

method Index theory for Dirac operators, focusing on scalar curvature and width.

result A quantitative negative upper bound on the infimum of scalar curvature for complete metrics.

index theory scalar curvature Dirac operators width Riemannian bands

2020-08-31abs ↗pdf ↗

On the twin-width of smooth manifoldsmath.GT

Smooth manifolds can be triangulated with graphs of bounded twin-width.

problem Understanding the structure of triangulations of smooth manifolds.

method Using Whitney's triangulation method and bounding the twin-width of specific graphs.

result Compact smooth manifolds have triangulations with graphs of bounded twin-width.

twin-width triangulation smooth manifolds graph theory manifold topology

2024-07-14abs ↗pdf ↗

Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networksstat.ML

Study on fluctuations in neural network kernels and predictions, focusing on finite width effects.

problem Characterizing fluctuations in finite width neural networks.

method Dynamical mean field theory analysis of wide but finite feature learning neural networks.

result Fluctuations in kernels and predictions are dynamically coupled, leading to reduced variance in feature learning regimes.

neural networks finite width mean field theory fluctuations prediction variance

2023-04-06abs ↗pdf ↗

Finite Depth and Width Corrections to the Neural Tangent Kernelcs.LG

We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation is exponential in the ratio of network depth to width. Thus, even in the limit of infinite overparameterization, the NTK is not determinist…

2019-09-13abs ↗pdf ↗

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfercond-mat.dis-nn

The study examines spectral dynamics in deep neural networks, predicting how outliers evolve during training.

problem Understanding spectral evolution in deep neural networks during training.

method Developed a two-level dynamical mean-field theory (DMFT) to track spectral dynamics.

result The theory predicts how outliers evolve with training time, width, output scale, and initialization variance.

spectral dynamics deep neural networks outliers mean-field theory stochastic gradient descent

2026-05-08abs ↗pdf ↗

Finite Versus Infinite Neural Networks: an Empirical Studycs.LG

Empirical study compares wide neural networks to kernel methods, resolving open questions.

problem Understanding the relationship between wide neural networks and kernel methods.

method Large-scale empirical study using various neural network architectures and kernel methods.

result Wide neural networks outperform fully-connected finite-width networks in some cases, but underperform convolutional finite-width networks.

neural networks kernel methods finite width infinite width CIFAR-10

2020-07-31abs ↗pdf ↗

Metrics of positive scalar curvature and unbounded widthsmath.DG

In this work we construct a sequence of Riemannian metrics on the three-sphere with scalar curvature greater than or equal to $6$ and arbitrarily large widths. Our procedure is based on the connected sum construction of positive scalar curvature metrics due to Gromov and Lawson. We develop analogies between the area of…

2015-03-08abs ↗pdf ↗

Urysohn width of hypersurfaces and positive macroscopic scalar curvaturemath.DG

The paper proves conditions for the existence of small Urysohn width hypersurfaces in manifolds with positive scalar curvature.

problem Conditions for the existence of small Urysohn width hypersurfaces in manifolds with positive scalar curvature.

method Adaptation of Guth's macroscopic version of the Schoen-Yau descent argument.

result A complete Riemannian manifold with positive macroscopic scalar curvature contains a non-nullhomologous hypersurface of small Urysohn width.

Urysohn width hypersurface positive scalar curvature macroscopic homology

2025-04-09abs ↗pdf ↗

Width-Robust Learnability in Mean-Field Bayesian Neural Networksstat.ML

Bayesian neural networks learn efficiently at infinite width, matching polynomial-width performance.

problem Understanding the inductive bias of infinite-width neural networks.

method Analyzing the reduced entropy and using subsampling techniques.

result The Bayesian mean-field learner generalizes exactly on polynomially-bounded targets.

Bayesian neural networks mean-field infinite width reduced entropy subsampling

2026-07-07abs ↗pdf ↗

Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luckcs.LG

This work explores feature learning tradeoffs in neural networks.

problem Resource tradeoffs in neural feature learning.

method Theoretical and experimental investigation of offline sparse parity learning.

result Width improves sample efficiency in sparse feature learning.

Neural Networks Feature Learning Tradeoffs Sparse Parity Sample Efficiency

2023-09-07abs ↗pdf ↗

Asymptotics of Wide Networks from Feynman Diagramscs.LG

Understanding the asymptotic behavior of wide networks is of considerable interest. In this work, we present a general method for analyzing this large width behavior. The method is an adaptation of Feynman diagrams, a standard tool for computing multivariate Gaussian integrals. We apply our method to study training dyn…

2019-09-25abs ↗pdf ↗

Explaining Neural Scaling Lawscs.LG

Theory explains neural network scaling with dataset and model size.

problem Neural network scaling laws with dataset and model size.

method Identified variance-limited and resolution-limited scaling behaviors.

result Four scaling regimes explained: infinite data, infinite width, resolution-limited, and large width.

neural networks scaling laws dataset size model size infinite data

2021-02-12abs ↗pdf ↗

On the min-max width of unit volume three-spheresmath.DG

How large can be the width of Riemannian three-spheres of the same volume in the same conformal class? If a maximum value is attained, how does a maximising metric look like? What happens as the conformal class changes? In this paper, we investigate these and other related questions, focusing on the context of Simon-Sm…

2018-09-10abs ↗pdf ↗

Minimum width for universal approximation using ReLU networks on compact domaincs.LG

Minimum width for ReLU networks on compact domain is exactly max{d_x, d_y, 2}

problem Characterizing the minimum width for ReLU networks to approximate functions on compact domains

method Analyzing the minimum width for

L^p

approximation of

L^p

functions from

[0,1]^d

\mathbb R^{d_y}

using ReLU-like activation functions

result The minimum width for

L^p

approximation on a compact domain is exactly max{d_x, d_y, 2} for ReLU-like activation functions

ReLU networks compact domain minimum width universal approximation activation functions

2023-09-19abs ↗pdf ↗

Surfaces of small diameter with large widthmath.DG

Given a 2-dimensional surface M and a constant C we construct a Riemannian metric g, so that diameter diam(M,g)=1 and every 1-cycle dividing M into two regions of equal area has length >C. It follows that there exists no universal inequality bounding 1-width of M in terms of its diameter. This answers a question of Ste…

2013-07-08abs ↗pdf ↗

The Effect of Network Width on the Performance of Large-batch Trainingstat.ML

Distributed implementations of mini-batch stochastic gradient descent (SGD) suffer from communication overheads, attributed to the high frequency of gradient updates inherent in small-batch training. Training with large batches can reduce these overheads; however, large batches can affect the convergence properties and…

2018-06-11abs ↗pdf ↗

On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspectivecs.LG

NTK theory fails to predict practical behavior of large-width neural networks.

problem Theoretical limits of NTK do not match practical neural network architectures.

method Empirical investigation of NTK's applicability to large-width architectures.

result Practically relevant behavior of large-width architectures differs from NTK theory.

neural tangent kernel neural networks large-width architectures theoretical limits practical behavior

2023-09-29abs ↗pdf ↗

Kernel Renormalization in Bayesian Deep Neural Networks: the Equivalent Wishart Ansatz in the Proportional Regimecs.LG

New approach predicts generalization of deep neural networks in proportional-width regime.

problem Predicting generalization of deep neural networks in proportional-width regime.

method Equivalent Wishart Ansatz for hierarchical empirical kernels, renormalized NNGP kernel.

result Renormalized NNGP kernel captures dominant stochastic fluctuations in deep neural networks.

Bayesian deep learning deep neural networks generalization Wishart Ansatz renormalization

2026-05-28abs ↗pdf ↗

Adversarial Examples in Multi-Layer Random ReLU Networkscs.LG

Adversarial examples in deep ReLU networks with constant depth.

problem Adversarial perturbations lead to large output changes in deep ReLU networks.

method Analyzes the phenomenon in networks with independent Gaussian parameters and constant depth.

result Adversarial examples arise due to functions being close to linear.

ReLU networks adversarial examples deep learning random parameters constant depth

2021-06-23abs ↗pdf ↗

Large-width functional asymptotics for deep Gaussian neural networksmath.PR

The paper studies deep neural networks with Gaussian weights and finds their asymptotic behavior.

problem Understanding the behavior of deep neural networks with large width.

method Function-space perspective, Gaussian process analysis, weak convergence in large-width limit.

result Deep neural networks with large width converge to a continuous Gaussian process.

deep neural networks Gaussian processes functional asymptotics large-width limit weak convergence

2021-02-20abs ↗pdf ↗

Propagation of Chaos in One-hidden-layer Neural Networks beyond Logarithmic Timestat.ML

Study shows polynomial-width neural networks can closely approximate infinite-width networks in polynomial time.

problem Approximating dynamics of polynomial-width neural networks with infinite-width networks.

method Bounding approximation gap through a differential equation governed by mean-field dynamics, considering local Hessian.

result Polynomially many neurons are sufficient to closely approximate mean-field dynamics.

Neural Networks Polynomial Width Infinite Width Mean-Field Dynamics Local Hessian

2025-04-17abs ↗pdf ↗

Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methodscs.LG

Gradient methods improve deep network training with tighter bounds and faster convergence.

problem Improving convergence and generalization of gradient methods for neural networks.

method Algorithmic stability analysis and novel bounds on excess risk.

result Gradient descent achieves optimal excess risk for deep nets with polynomial width conditions.

gradient methods neural networks excess risk stability analysis test error rate

2024-10-13abs ↗pdf ↗

Proportional infinite-width infinite-depth limit for deep linear neural networksstat.ML

Study of deep linear neural networks with proportional width and depth.

problem Lack of descriptive power in Gaussian limit of deep linear neural networks.

method Proportional infinite-width infinite-depth limit for deep linear neural networks.

result Characterization of limiting distribution as a nontrivial mixture of Gaussians.

deep learning neural networks infinite-width infinite-depth Gaussian process

2024-11-22abs ↗pdf ↗

Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networksstat.ML

Study shows how large neural networks avoid overfitting through decoupling of feature learning and complexity growth.

problem Understanding inductive bias and generalization in large neural networks.

method Dynamical mean field theory applied to large two-layer networks.

result Training dynamics of large networks exhibit a separation of timescales, decoupling feature learning and overfitting.

neural networks generalization overfitting dynamical mean field theory two-layer networks

2025-02-28abs ↗pdf ↗

On the asymptotics of wide networks with polynomial activationscs.LG

Wide networks with polynomial activations have proven asymptotic behavior.

problem Understanding the behavior of neural networks in the large width limit.

method Proving a conjecture for deep networks with polynomial activation functions.

result Tight bounds on the behavior of wide networks during stochastic gradient descent and derivation of their finite-width dynamics.

neural networks asymptotics stochastic gradient descent polynomial activations finite-width dynamics

2020-06-11abs ↗pdf ↗

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Sizecs.LG

New analysis shows optimal embedding learning rate depends on vocabulary size, not just model width.

problem Optimal learning rate for language model embeddings is not well understood, especially with large vocabularies.

method Theoretical analysis of training dynamics, interpolation between

μ

P and LV regimes.

result Optimal embedding learning rate scales as

Θ(\sqrt{width})

in the LV regime, not

Θ(width)

μ

P predicts.

language models learning rate vocabulary size embedding dimension training dynamics

2025-06-17abs ↗pdf ↗

LoRA+: Efficient Low Rank Adaptation of Large Modelscs.LG

Improved LoRA+ adapts large models more efficiently.

problem Suboptimal feature learning in LoRA for large models.

method Different learning rates for LoRA adapter matrices A and B.

result Improves performance and speed by 1-2% and up to 2X, respectively.

LoRA Low Rank Adaptation Large Models Feature Learning Efficient Adaptation

2024-02-19abs ↗pdf ↗

On the spherical Blaschke-Lebesgue problemmath.MG

Study on spherical bodies of constant width on the unit sphere, proving bounds on their relative effective radius.

problem Understanding the smallest spherical bodies of constant width on the unit sphere.

method Analyzing spherical bodies of constant width on the unit sphere, constructing examples and applying geometric arguments.

result Proved non-trivial bounds on the relative effective radius of spherical bodies of constant width.

spherical geometry constant width unit sphere relative effective radius spherical bodies

2026-06-29abs ↗pdf ↗

Maximal Initial Learning Rates in Deep ReLU Networksstat.ML

Maximal initial learning rate for deep ReLU networks identified.

problem Finding the optimal initial learning rate for deep neural networks.

method Simple approach to estimate maximal initial learning rate

η^{\ast}

, analyzing its behavior in constant-width fully-connected ReLU networks.

result Maximal initial learning rate

η^{\ast}

is well predicted as a power of depth × width, with specific conditions for network width and input layer training.

deep learning ReLU networks learning rate initialization maximal learning rate

2022-12-14abs ↗pdf ↗

The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limitstat.ML

New Transformer architecture prevents rank degeneracy in deep attention models.

problem Rank degeneracy in deep attention models.

method Modified Softmax-based attention model with skip connections, centered at identity, and scaled logits.

result Existence of a stable SDE implies well-behaved covariance structure, preventing rank degeneracy.

Transformer attention models infinite depth-and-width covariance matrix stochastic differential equation

2023-06-30abs ↗pdf ↗

Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfercs.LG

The study analyzes deep linear networks from random initialization, capturing dynamics and hyperparameter effects.

problem Understanding training dynamics in deep linear networks from random initialization.

method Theoretical analysis of gradient descent dynamics in deep linear networks with random initialization and large data.

result Captures the 'wider is better' effect and hyperparameter transfer effects, contrasting with neural-tangent parameterization.

deep linear networks random initialization gradient descent hyperparameter transfer mean-field

2025-02-04abs ↗pdf ↗

Analysis of aggregated tick returns: evidence for anomalous diffusionphysics.soc-ph

In order to investigate the origin of large price fluctuations, we analyze stock price changes of ten frequently traded NASDAQ stocks in the year 2002. Though the influence of the trading frequency on the aggregate return in a certain time interval is important, it cannot alone explain the heavy tailed distribution of …

2006-06-18abs ↗pdf ↗

Deep Networks and the Multiple Manifold Problemstat.ML

Deep networks can perfectly classify two low-dimensional manifolds on a sphere with large depth and width.

problem Binary classification of two low-dimensional submanifolds on a sphere.

method Analysis of a deep fully-connected neural network trained to separate two submanifolds of the unit sphere.

result Randomly-initialized gradient descent can perfectly classify the two manifolds with high probability when the network depth is large relative to certain geometric and statistical properties of the data.

deep learning neural networks manifold learning machine vision neural tangent kernel

2020-08-25abs ↗pdf ↗

Predicting the outputs of finite deep neural networks trained with noisy gradientsstat.ML

Analyzes DNNs trained with noisy gradients, finding FWCs negligible for large n.

problem Analyzing DNNs trained with noisy gradients.

method Introduced analytical framework to analyze non-Gaussian stochastic process.

result FWCs negligible for large n, improving CNN performance.

finite deep neural networks noisy gradients finite width correction finite convolutional neural networks image classification

2020-04-02abs ↗pdf ↗

Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field trainingcs.LG

This work studies fluctuation in multilayer neural networks using mean field theory.

problem Understanding fluctuation in multilayer neural networks with mean field training.

method Developed a second-order mean field limit to capture fluctuation, demonstrating stability of gradient descent training.

result Gradient descent training in multilayer networks biases towards minimal fluctuation, even after convergence.

multilayer neural networks mean field theory fluctuation gradient descent stability

2021-10-29abs ↗pdf ↗

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limitstat.ML

Residual networks with depthwise hyperparameter scaling transfer optimal hyperparameters across width and depth.

problem The challenge of hyperparameter tuning in deep learning, especially for large models.

method Combining

μ

P parameterization with residual networks having a residual branch scale of

1/\sqrt{ ext{depth}}

result Optimal hyperparameters transfer across width and depth in residual networks trained with this parameterization.

hyperparameter tuning residual networks depthwise scaling mean field theory neural network dynamics

2023-09-28abs ↗pdf ↗

Deep Stable neural networks: large-width asymptotics and convergence ratescs.LG

This paper analyzes deep Stable neural networks, showing convergence rates under different growth settings.

problem Analyzing the behavior of deep Stable neural networks as width increases.

method Large-width asymptotic analysis and convergence rates for fully connected feed-forward deep Stable NNs.

result The rescaled deep Stable NN converges weakly to a Stable SP under joint growth, with sup-norm convergence rates established.

Deep Learning Stable Distributions Convergence Rates Asymptotic Analysis Neural Networks

2021-08-02abs ↗pdf ↗