On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “finite depth”

On the infinite-depth limit of finite-width neural networksstat.ML

Study infinite-depth limits of neural networks with fixed width.

problem Understanding the behavior of neural networks as depth increases with fixed width.

method Analyzing finite-width residual networks with random Gaussian weights, focusing on the infinite-depth limit.

result The pre-activations converge to a zero-drift diffusion process, differing from the infinite-width limit.

neural networks infinite-depth finite-width residual networks random weights

2022-10-03abs ↗pdf ↗

On Random Kernels of Residual Architecturescs.LG

ResNets and DenseNets converge to NTK with depth and width, offering advantages for kernel regression.

problem Understanding convergence of ResNets and DenseNets to Neural Tangent Kernel (NTK).

method Analysis of finite width and depth corrections for NTK of ResNets and DenseNets.

result ResNets and DenseNets can converge to NTK with depth and width, unlike vanilla networks.

ResNets DenseNets Neural Tangent Kernel Finite width Finite depth

2020-01-28abs ↗pdf ↗

Finite Depth and Width Corrections to the Neural Tangent Kernelcs.LG

We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation is exponential in the ratio of network depth to width. Thus, even in the limit of infinite overparameterization, the NTK is not determinist…

2019-09-13abs ↗pdf ↗

Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examplesstat.ML

Generates infinite-depth hierarchical clusters from few examples.

problem Inadequate finite-sample clustering methods for fine-scale hierarchical structures.

method Classification fields generated by a local refinement rule, approximated by predictors.

result Learned predictors can approximate infinite-depth hierarchical structures.

Hierarchical Clustering Infinite Depth Local Refinement Finite-Sample Exponential Convergence

2026-05-08abs ↗pdf ↗

Rectangular diagrams of foliationsmath.GT

Rectangular diagrams help analyze foliations in 3-sphere.

problem Analyzing foliations in 3-sphere complements.

method Introduced rectangular diagrams for foliations and links.

result Any co-orientable finite depth foliation can be presented by a compatible rectangular diagram.

foliations three-sphere rectangular diagrams co-orientable finite depth

2025-08-11abs ↗pdf ↗

Feature Learning Dynamics in Infinite-Depth Neural Networkscs.LG

Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.

problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.

method Conditional Gaussian representations and SDE system with decoupled backward weights.

result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.

Feature Learning Neural Networks Infinite Depth ResNets SDE System

2025-12-24abs ↗pdf ↗

Quasi-actions on trees II: Finite depth Bass-Serre treesmath.GR

This paper addresses questions of quasi-isometric rigidity and classification for fundamental groups of finite graphs of groups, under the assumption that the Bass-Serre tree of the graph of groups has finite depth. The main example of a finite depth graph of groups is one whose vertex and edge groups are coarse Poinca…

2004-05-13abs ↗pdf ↗

Depth induces scale-averaging in overparameterized linear Bayesian neural networkscs.LG

Study how depth affects inference in deep Bayesian neural networks.

problem Understanding how depth impacts inference in overparameterized linear Bayesian neural networks.

method Interpreting finite deep linear Bayesian neural networks as scale mixtures of Gaussian process predictors.

result Advances analytical understanding of how depth affects inference in a simple class of Bayesian neural networks.

Bayesian neural networks scale mixtures Gaussian processes inference depth effects

2021-11-23abs ↗pdf ↗

seMCD: Sequentially implemented Monte Carlo depth computation with statistical guaranteesstat.ME

seMCD computes depth functions with statistical guarantees using sequential Monte Carlo.

problem Computing depth functions is computationally challenging, especially in high dimensions.

method Sequential Monte Carlo methodology with theoretical and empirical guarantees.

result The seMCD method provides accurate depth approximations with fewer samples than traditional methods.

Monte Carlo depth functions sequential algorithms statistical guarantees outlier detection

2025-07-08abs ↗pdf ↗

Layer-wise Geometric Approximation Rates for Deep Networkscs.LG

Deep networks improve by progressively refining approximations at each layer.

problem Standard approximation theory doesn't explain the role of intermediate layers in deep neural networks.

method Developed a mixed-activation architecture with a geometric scale interpretation of depth.

result Each intermediate layer approximates the target function with a geometric rate.

deep learning neural networks approximation theory geometric rate multigrade deep learning

2026-04-22abs ↗pdf ↗

Polar Depth for Potentially Heavy-Tailed Datamath.ST

Introduces Polar Depth for analyzing multivariate heavy-tailed data extremes.

problem Analyzing the behavior of extremes from multivariate heavy-tailed distributions.

method Introduces Polar Depth, a novel statistical depth function expressed in polar coordinates.

result The polar depth of the largest observations converges to the polar depth of the limiting distribution as the threshold increases.

Polar Depth Heavy-tailed Data Multivariate Extremes Anomaly Detection Regularly Varying Random Variables

2026-05-29abs ↗pdf ↗

Open Saturated Sets Without Holonomymath.GT

Open, connected, saturated sets W without holonomy in codimension one foliations play key roles as fundamental building blocks. Here, for the case of foliated 3-manifolds, we produce a finite system of closed, convex, non-overlapping polyhedral cones in the first cohomology of W with real coefficients such that the iso…

2011-08-03abs ↗pdf ↗

Model-Free Kernel Conformal Depth Measures Algorithm for Uncertainty Quantification in Regression Models in Separable Hilbert Spacesstat.ML

New algorithm quantifies uncertainty in regression models for complex data types.

problem Uncertainty quantification in regression models for complex data types.

method Model-free uncertainty quantification algorithm based on conditional depth measures and kernel mean embeddings.

result Provides faster convergence rates and non-asymptotic guarantees for prediction regions.

uncertainty quantification regression models conditional depth measures kernel mean embeddings conformal prediction

2025-06-10abs ↗pdf ↗

Affine-Invariant Integrated Rank-Weighted Depth: Definition, Properties and Finite Sample Analysisstat.ML

A new depth function improves multivariate data analysis by considering variability directions.

problem Developing a depth function that respects quantile properties and is affine-invariant.

method Integrating rank-weighted depth with affine-invariance and covariance matrices.

result The AI-IRW depth function provides accurate quantile estimates and is robust to data variability.

depth function affine-invariance multivariate data statistical depth anomaly detection

2021-06-21abs ↗pdf ↗

Differentially private multivariate mediansmath.ST

Develops privacy-preserving multivariate median estimation methods.

problem Lack of rigorous privacy guarantees for robust multivariate location estimation.

method Novel finite-sample performance guarantees for differentially private multivariate depth-based medians.

result Sharp performance guarantees for multivariate depth-based medians under differential privacy.

differential privacy multivariate medians robust statistics privacy-preserving depth functions

2022-10-12abs ↗pdf ↗

Transformers Learn Shortcuts to Automatacs.LG

Transformers simulate finite-state automata with fewer layers.

problem How do shallow, non-recurrent Transformers simulate complex computations?

method Hierarchical reparameterization of recurrent dynamics to simulate automata.

result Polynomial-sized,

O(\log T)

-depth solutions exist and are common.

Transformers Finite-state automata Shortcut solutions Circuit complexity Brittleness

2022-10-19abs ↗pdf ↗

A First-Order Mean Field Control Analysis of Transformer Layers under Cross-Entropy Trainingmath.OC

Study Transformer layers under cross-entropy training using mean field control.

problem Understanding the behavior of Transformer layers in cross-entropy training.

method Continuous-depth mean field control analysis, treating depth as time and layer parameters as controls.

result Derivation of a Pontryagin condition for the limiting population problem, involving the softmax residual.

Transformer layers cross-entropy training mean field control Pontryagin condition hidden-state flow

2026-06-22abs ↗pdf ↗

Foliation Conesmath.GT

David Gabai showed that disk decomposable knot and link complements carry taut foliations of depth one. In an arbitrary sutured 3-manifold M, such foliations F, if they exist at all, are determined up to isotopy by an associated ray [F] issuing from the origin in H^1(M;R) and meeting points of the integer lattice H^1(M…

1998-09-18abs ↗pdf ↗

The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initializationstat.ML

Study on neural network initialization with shaped infinite depth-and-width networks.

problem Understanding the distribution of random covariance matrices in shaped infinite-depth-and-width networks.

method Introduced the Neural Covariance SDE to model the distribution of the random covariance matrix.

result Identified the precise scaling of the activation function necessary for a non-trivial limit.

neural networks infinite depth infinite width covariance matrix stochastic differential equation

2022-06-06abs ↗pdf ↗

Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initializationstat.ML

The paper examines how the angle between inputs in ReLU networks decreases with depth, impacting training.

problem Depth degeneracy in neural networks, leading to constant function behavior on initialization.

method Combinatorial expansions and Monte Carlo experiments to analyze the angle between inputs in ReLU networks of increasing depth.

result The angle between inputs in ReLU networks decreases exponentially with depth, leading to constant function behavior on initialization.

depth degeneracy ReLU networks angle between inputs Monte Carlo experiments Bessel numbers

2023-02-20abs ↗pdf ↗

Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Modelscs.LG

Wide neural networks converge to Gaussian processes, improving generalization.

problem Understanding the generalization of wide neural networks, especially deep equilibrium models.

method Investigation of deep equilibrium models (DEQs) with infinite-depth layers, focusing on their convergence to Gaussian processes as width and depth approach infinity.

result Wide DEQs converge to Gaussian processes, maintaining generalization performance.

deep equilibrium models neural networks Gaussian processes benign overfitting infinite-depth networks

2023-10-16abs ↗pdf ↗

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limitstat.ML

Residual networks with depthwise hyperparameter scaling transfer optimal hyperparameters across width and depth.

problem The challenge of hyperparameter tuning in deep learning, especially for large models.

method Combining

μ

P parameterization with residual networks having a residual branch scale of

1/\sqrt{ ext{depth}}

result Optimal hyperparameters transfer across width and depth in residual networks trained with this parameterization.

hyperparameter tuning residual networks depthwise scaling mean field theory neural network dynamics

2023-09-28abs ↗pdf ↗

A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networkscs.LG

Unified method for deriving ridgelet transforms for various neural network architectures.

problem Deriving closed-form expressions for ridgelet transforms in modern neural network architectures.

method Unified Fourier slice method to derive ridgelet transforms for diverse neural network types.

result Systematic method to derive ridgelet transforms for various neural network architectures.

ridgelet transform neural networks Fourier slice depth-2 networks finite fields

2024-02-25abs ↗pdf ↗

A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regionsstat.ML

A new pseudo-metric uses data depth to compare probability distributions.

problem Designing a metric between probability distributions for machine learning applications.

method Extension of univariate quantiles to multivariate spaces, using data depth and Hausdorff distance.

result The pseudo-metric is robust, factorizes translations, and has good behavior under transformations.

pseudo-metric probability distributions data depth Hausdorff distance machine learning

2021-03-23abs ↗pdf ↗

Statistical Depth Functions for Ranking Distributions: Definitions, Statistical Learning and Applicationscs.LG

This paper introduces depth functions for ranking data, improving statistical summaries.

problem Lack of comprehensive statistical summaries for ranking data.

method Metric-based depth functions on symmetric group to define rankings, depths, and procedures.

result Novel depth functions provide a more informative summary of ranking data.

ranking data depth functions symmetric group statistical summaries metric-based

2022-01-20abs ↗pdf ↗

Graph Neural Networks Do Not Always Oversmoothstat.ML

Graphs avoid oversmoothing with properly initialized weights.

problem Oversmoothing in graph convolutional networks (GCNs) limits their depth.

method Used Gaussian process equivalence to analyze GCNs, identifying a non-oversmoothing phase.

result Properly initialized GCNs can be deep and expressive.

Graph Neural Networks Oversmoothing Gaussian Process Non-Oversmoothing Phase Initialization

2024-06-04abs ↗pdf ↗

On groups with linear sci growthmath.GT

We prove that the semistability growth of hyperbolic groups is linear, which implies that hyperbolic groups which are sci (simply connected at infinity) have linear sci growth. Based on the linearity of the end-depth of finitely presented groups we show that the linear sci is preserved under amalgamated products over f…

2013-12-03abs ↗pdf ↗

Is deeper better? It depends on locality of relevant featurescs.LG

Deeper networks are better for local labels, but shallower for global labels.

problem Understanding the effect of depth in overparameterized neural networks.

method Introduced local and global labels to investigate the advantage of depth.

result Deeper networks are better for local labels, shallower for global labels.

neural networks overparameterization depth locality generalization

2020-05-26abs ↗pdf ↗

Collective Kernel EFT for Pre-activation ResNetscs.LG

Develops EFT for ResNets, revealing limitations of kernel-only approach.

problem Limitations of kernel-only approach in deep neural networks.

method Collective kernel EFT for pre-activation ResNets based on

G

-only closure hierarchy.

result Numerical findings show

V_4

equation residual accumulates to an

O(1)

error.

ResNets Deep Learning Kernel EFT Finite-width Networks Gaussian Approximations

2026-04-17abs ↗pdf ↗

Feature Learning in Wide Neural Networks under

μ

P: Identifiability and Sparse-Dictionary Decomposition of the Mean-Field Limitcs.LG

Wide neural networks learn features under $μ$ P, identifying weights and decomposing support.

problem Feature learning in wide neural networks under

μ

method Proving mean-field limit, characterizing identifiability, sparse-dictionary decomposition, and feature-learning-error decomposition.

result The triple

(w^*, D^*_{\mathrm{orb}}, S^*)

identifies the natural learning cell of the architecture-data pair

(σ, ρ)

wide neural networks feature learning $μ$P mean-field limit identifiability

2026-05-23abs ↗pdf ↗

The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limitstat.ML

New Transformer architecture prevents rank degeneracy in deep attention models.

problem Rank degeneracy in deep attention models.

method Modified Softmax-based attention model with skip connections, centered at identity, and scaled logits.

result Existence of a stable SDE implies well-behaved covariance structure, preventing rank degeneracy.

Transformer attention models infinite depth-and-width covariance matrix stochastic differential equation

2023-06-30abs ↗pdf ↗

Feature Learning and Generalization in Deep Networks with Orthogonal Weightscs.LG

Deep networks with orthogonal weights show stable fluctuations, improving generalization and training speed.

problem Fluctuations in deep networks with Gaussian weights can impair training, especially in networks with depth comparable to width.

method Analytical and numerical studies of fully-connected networks with orthogonal weight initialization and tanh activations.

result Rectangular networks with orthogonal weights have stable fluctuations independent of network depth, leading to better generalization and training speed.

deep learning orthogonal weights fluctuations generalization training speed

2023-10-11abs ↗pdf ↗

Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Widthstat.ML

This paper improves neural network approximation for analytic functions with adjustable depth and width.

problem Approximating analytic functions using neural networks with depth and width parameters.

method Characterizes approximation rates as a joint function of width (N) and depth (L) for ReLU networks.

result Establishes upper bounds for analytic function approximation rates of O(N^(-CL^τ)) with τ influenced by N and L.

Neural Networks ReLU Analytic Functions Approximation Rates Depth

2026-07-12abs ↗pdf ↗

Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscs.LG

CoT enhances transformer accuracy on serial tasks by enabling serial computation.

problem Improving accuracy of large language models on inherently serial problems.

method Integrating a chain of thought (CoT) into decoder-only transformers to enable serial computation.

result Constant-depth transformers with CoT can solve problems in AC^0, surpassing TC^0 without CoT.

transformers chain of thought serial computation arithmetic symbolic reasoning

2024-02-20abs ↗pdf ↗

A stochastic partial differential equation model for limit order book dynamicsq-fin.TR

We propose an analytically tractable class of models for the dynamics of a limit order book, described through a stochastic partial differential equation (SPDE) with multiplicative noise for the order book centered at the mid-price, along with stochastic dynamics for the mid-price which is consistent with the order flo…

2019-04-05abs ↗pdf ↗

Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory?cs.LG

The paper examines when NTK theory applies to real finite-width neural networks.

problem Understanding when NTK theory accurately predicts the behavior of finite-width neural networks.

method Empirical study of fully-connected ReLU and sigmoid DNNs with various hyperparameters and depths.

result NTK theory does not always apply to sufficiently deep networks with exploding gradients, and the kernel changes significantly during training.

Neural Tangent Kernel Finite Neural Networks Gradient Descent Generalization Deep Learning

2020-12-08abs ↗pdf ↗

Geometry of foliations and flows I: Almost transverse pseudo-Anosov flows and asymptotic behavior of foliationsmath.GT

Let F be a foliation in a closed 3-manifold with negatively curved fundamental group and suppose that F is almost transverse to a quasigeodesic pseudo-Anosov flow. We show that the leaves of the foliation in the universal cover extend continuously to the sphere at infinity, hence the limit sets are continuous images of…

2005-02-15abs ↗pdf ↗

Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamicscs.LG

Attention-only transformers learn from context via two stages of inference.

problem Learning from corrupted token sequences in minimal transformers.

method Two-stage empirical Bayes interpretation: kernel-weighted posterior mean and particle dynamics.

result Effective denoising without explicit noise schedules, showing posterior-mean recovery under asymptotic conditions.

attention transformers empirical Bayes particle dynamics denoising

2026-05-28abs ↗pdf ↗

A Note on the Global Convergence of Multilayer Neural Networks in the Mean Field Regimecs.LG

Global convergence of multilayer neural networks proven for any depth.

problem Global convergence of multilayer neural networks in the mean field regime.

method Mean field limit framework, neuronal embedding, bidirectional diversity condition.

result Global convergence for multilayer networks of any depths, including correlated initializations.

global convergence multilayer neural networks mean field regime bidirectional diversity neuronal embedding

2020-06-16abs ↗pdf ↗

Bottleneck Structure in Learned Features: Low-Dimension vs Regularity Tradeoffcs.LG

Large neural networks learn low-dimensional representations that balance complexity and regularity.

problem Understanding the tradeoff between low-dimensional representations and complexity in deep neural networks.

method Computed finite depth corrections to reveal a measure of regularity that bounds the pseudo-determinant of the Jacobian.

result Proved the conjectured bottleneck structure in learned features as network depth increases, showing almost all hidden representations are approximately low-dimensional and weight matrices have singular values close to 1.

Deep Learning Neural Networks Bottleneck Structure Regularization Rank

2023-05-30abs ↗pdf ↗

Coxeter polytopes and Benjamini--Schramm convergencemath.GT

Study on hyperbolic polyhedra and their volume, proving finiteness of arithmetic groups.

problem Finiteness of arithmetic maximal reflection groups in hyperbolic polyhedra.

method Observation of volume distribution and recent work with M. Fraczyk and S. Hurtado.

result Proof of finiteness of arithmetic maximal reflection groups.

hyperbolic polyhedra volume distribution arithmetic groups reflection groups Benjamini-Schramm convergence

2022-09-07abs ↗pdf ↗

Smooth Fractal Trees: Analytic Generators and Discrete Equivalencemath.DS

Smooth fractal trees via analytic generators, preserving combinatorial and geometric properties.

problem Constructing smooth fractal trees from discrete models.

method Using analytic generator fields to integrate smooth vector fields in an internal state space, generating geometric curves as projections of generator trajectories.

result Analytic generators can represent any discrete tree specification and preserve the asymptotic limit geometry.

fractal trees analytic generators discrete equivalence smooth curves branching

2026-01-24abs ↗pdf ↗

On cycles and coverings associated to a knotmath.GT

We consider the space of all representations of the commutator subgroup of a knot group into a finite abelian group Σ, together with a shift map σ_x. This is a finite dynamical system, introduced by D.Silver and S. Williams. We describe the lengths of its cycles in terms of the roots of the Alexander polynomial of the …

2013-01-10abs ↗pdf ↗

Deep imagination is a close to optimal policy for planning in large decision trees under limited resourcesstat.ML

Deep imagination optimizes decision-making in large trees with limited resources.

problem Optimal planning in large decision trees with limited resources and time.

method Analytical solutions and numerical analysis of sampling capacity allocation.

result Optimal policy is to allocate few samples per level for deep exploration, favoring depth over breadth.

decision trees planning resource allocation finite capacity deep imagination

2021-04-13abs ↗pdf ↗

Fast kernel half-space depth for data with non-convex supportsstat.ML

A new depth measure for non-convex data supports, faster than halfspace depth.

problem Non-convex data supports in multivariate statistics.

method Extending halfspace depth to Reproducing Kernel Hilbert Space (RKHS).

result The new depth measure is consistent and can be computed faster.

data depth non-convex data halfspace depth Reproducing Kernel Hilbert Space multimodality

2023-12-21abs ↗pdf ↗

The universal approximation power of finite-width deep ReLU networkscs.LG

We show that finite-width deep ReLU neural networks yield rate-distortion optimal approximation (Bölcskei et al., 2018) of polynomials, windowed sinusoidal functions, one-dimensional oscillatory textures, and the Weierstrass function, a fractal function which is continuous but nowhere differentiable. Together with thei…

2018-06-05abs ↗pdf ↗

On the quality of randomized approximations of Tukey's depthstat.ML

The paper studies randomized approximations of Tukey's depth for log-concave isotropic data.

problem The challenge of approximating Tukey's depth in high dimensions.

method The study examines randomized algorithms for approximating Tukey's depth for log-concave isotropic data.

result Randomized algorithms correctly approximate maximal depth and close to zero depths but not intermediate depths.

Tukey's depth log-concave distribution isotropic data randomized approximation polynomial time

2023-09-11abs ↗pdf ↗

Statistical Depth Meets Machine Learning: Kernel Mean Embeddings and Depth in Functional Data Analysismath.ST

This paper connects functional data analysis with machine learning techniques.

problem Lack of theoretical analysis for functional depths.

method Viewing functional depths as kernel mean embeddings in machine learning.

result Facilitates answers to open questions about functional depths.

functional data analysis kernel mean embedding statistical depth machine learning

2021-05-26abs ↗pdf ↗