On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “finite deep neural networks”

Finite-time Lyapunov exponents of deep neural networkscond-mat.dis-nn

Study of deep neural networks using finite-time Lyapunov exponents.

problem Understanding the geometric structures in input space formed by deep neural networks.

method Analogy with dynamical systems, computing finite-time Lyapunov exponents.

result Ridges of large positive exponents divide input space into regions associated with different classes.

deep neural networks finite-time Lyapunov exponents input space coherent structures learning capabilities

2023-06-21abs ↗pdf ↗

A priori guarantees of finite-time convergence for Deep Neural Networkscs.LG

The paper analyzes deep neural networks using control theory to set a time limit for their convergence.

problem Understanding the finite-time convergence of deep neural networks.

method Lyapunov based analysis of the loss function, control theory framework, finite-time control of non-linear systems.

result A priori guarantees of finite-time convergence for deep neural networks are provided.

deep neural networks finite-time convergence control theory Lyapunov analysis non-linear systems

2020-09-16abs ↗pdf ↗

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightsstat.ML

Study Gaussian approximation for deep neural networks with random weights.

problem Understanding the distribution of deep neural networks with random weights.

method Established Gaussian approximation bounds in Wasserstein-1 norm.

result Convergence rates of order

n^{-({1}/{6})^{L-1} + ε}

for deep networks with proportional layer widths.

Gaussian approximation deep neural networks random weights Wasserstein-1 norm activation function

2025-07-16abs ↗pdf ↗

The universal approximation power of finite-width deep ReLU networkscs.LG

We show that finite-width deep ReLU neural networks yield rate-distortion optimal approximation (Bölcskei et al., 2018) of polynomials, windowed sinusoidal functions, one-dimensional oscillatory textures, and the Weierstrass function, a fractal function which is continuous but nowhere differentiable. Together with thei…

2018-06-05abs ↗pdf ↗

Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Widthcs.LG

Taylorized training improves neural network training at finite width.

problem Understanding and improving neural network training at finite width.

method Training the k-th order Taylor expansion of the neural network at initialization.

result Taylorized training agrees with full neural network training better as k increases and can significantly close the performance gap.

neural networks training Taylor expansion finite width approximation error

2020-02-10abs ↗pdf ↗

Bayesian inference with finitely wide neural networkscond-mat.dis-nn

Bayesian inference for wide neural networks using Edgeworth expansion.

problem Analyzing the non-Gaussian behavior of wide neural networks in Bayesian inference.

method Proposed a non-Gaussian distribution using multivariate Edgeworth expansion for finite-width neural networks.

result Derived non-Gaussian posterior distribution in Bayesian regression tasks.

Bayesian inference wide neural networks Edgeworth expansion non-Gaussian distribution marginal properties

2023-03-06abs ↗pdf ↗

Confident Neural Network Regression with Bootstrapped Deep Ensemblesstat.ML

Improved neural network regression uncertainty estimation.

problem Neural networks lack classical uncertainty due to finite data.

method Bootstrapped Deep Ensembles, incorporating parametric bootstrap.

result Significantly improved uncertainty estimation compared to standard Deep Ensembles.

neural networks regression uncertainty estimation parametric bootstrap Deep Ensembles

2022-02-22abs ↗pdf ↗

Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory?cs.LG

The paper examines when NTK theory applies to real finite-width neural networks.

problem Understanding when NTK theory accurately predicts the behavior of finite-width neural networks.

method Empirical study of fully-connected ReLU and sigmoid DNNs with various hyperparameters and depths.

result NTK theory does not always apply to sufficiently deep networks with exploding gradients, and the kernel changes significantly during training.

Neural Tangent Kernel Finite Neural Networks Gradient Descent Generalization Deep Learning

2020-12-08abs ↗pdf ↗

Dynamics of Deep Neural Networks and Neural Tangent Hierarchycs.LG

The evolution of a deep neural network trained by the gradient descent can be described by its neural tangent kernel (NTK) as introduced in [20], where it was proven that in the infinite width limit the NTK converges to an explicit limiting kernel and it stays constant during training. The NTK was also implicit in some…

2019-09-18abs ↗pdf ↗

Deep Networks are Reproducing Kernel Chainscs.LG

Deep networks are shown to be equivalent to a new type of kernel chain.

problem Identifying an appropriate function space for deep neural networks.

method Extending Reproducing Kernel Banach Spaces (RKBS) to chain RKBS (cRKBS), which composes kernels rather than functions.

result Any deep neural network function is a neural cRKBS function, and conversely, any neural cRKBS function corresponds to a deep neural network.

deep networks Reproducing Kernel Chain neural cRKBS sparse solution empirical risk minimization

2025-01-07abs ↗pdf ↗

Predicting the outputs of finite deep neural networks trained with noisy gradientsstat.ML

Analyzes DNNs trained with noisy gradients, finding FWCs negligible for large n.

problem Analyzing DNNs trained with noisy gradients.

method Introduced analytical framework to analyze non-Gaussian stochastic process.

result FWCs negligible for large n, improving CNN performance.

finite deep neural networks noisy gradients finite width correction finite convolutional neural networks image classification

2020-04-02abs ↗pdf ↗

Deep Maxout Network Gaussian Processstat.ML

Study deep maxout networks and their equivalence to Gaussian processes.

problem Understanding neural networks with infinite width.

method Derive equivalence between deep maxout networks and Gaussian processes, characterize maxout kernel, and provide efficient numerical implementation.

result Bayesian inference based on deep maxout network kernel leads to competitive results compared to finite-width counterparts and deep neural network kernels.

maxout networks Gaussian processes infinite width neural networks Bayesian inference

2022-08-08abs ↗pdf ↗

Les Houches Lectures on Deep Learning at Large & Infinite Widthstat.ML

Lectures on deep learning properties in infinite and large-width networks.

problem Understanding deep neural networks in extreme width conditions.

method Analysis of random deep neural networks, connections to linear models, kernels, and Gaussian processes, perturbative and non-perturbative treatments.

result Properties and behaviors of deep neural networks in the infinite-width limit and large-width regime.

deep learning neural networks infinite width large width statistical physics

2023-09-04abs ↗pdf ↗

A Finite-Time Analysis of Q-Learning with Neural Network Function Approximationcs.LG

Q-learning with neural network function approximation (neural Q-learning for short) is among the most prevalent deep reinforcement learning algorithms. Despite its empirical success, the non-asymptotic convergence rate of neural Q-learning remains virtually unknown. In this paper, we present a finite-time analysis of a…

2019-12-10abs ↗pdf ↗

Learning reduced systems via deep neural networks with memorystat.ML

Deep neural networks with memory learn reduced equations from partial data.

problem Constructing governing equations for unknown dynamical systems from limited data.

method Formulate a discrete approximation of memory integrals, use deep neural networks to incorporate history terms.

result Deep neural networks can learn reduced equations with memory from partial data.

deep learning neural networks memory governing equations partial data

2020-03-20abs ↗pdf ↗

Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networkscs.LG

This paper improves causal inference using deep neural networks for low-dimensional covariates.

problem Improving causal inference with deep learning for high-dimensional covariates.

method Doubly robust off-policy learning with deep neural networks on low-dimensional manifolds.

result Nonasymptotic regret bounds for finite- and continuous-action scenarios, converging at a fast rate depending on intrinsic manifold dimension.

causal inference deep learning doubly robust off-policy learning low-dimensional manifolds

2020-11-03abs ↗pdf ↗

Stable behaviour of infinitely wide deep neural networksstat.ML

Stable processes emerge as limits of deep neural networks with symmetric stable distributions.

problem Understanding the behavior of deep neural networks as they become infinitely wide.

method Analyzing fully connected feed-forward deep neural networks with symmetric stable distributions and showing the limit as a stable process.

result The infinite wide limit of the network is a stable process with multivariate stable distributions.

deep neural networks stable distributions infinite width stochastic processes symmetric distributions

2020-03-01abs ↗pdf ↗

Structures of Neural Network Effective Theorieshep-th

Simplified neural network EFTs reveal a single critical condition.

problem Understanding neuron statistics in neural networks at initialization.

method Diagrammatic approach to effective field theories (EFTs).

result A single condition governs criticality of all neuron preactivations.

neural networks effective field theories deep learning field theory criticality

2023-05-03abs ↗pdf ↗

Asymptotics of representation learning in finite Bayesian neural networkscs.LG

We study how finite Bayesian neural networks adapt their hidden representations.

problem Understanding how finite Bayesian neural networks differ from infinite ones.

method We analyze the asymptotics of learned feature kernels for various network architectures.

result The leading finite-width corrections to feature kernels have a universal form.

Bayesian neural networks representation learning asymptotics finite-width corrections feature kernels

2021-06-01abs ↗pdf ↗

Infinite-channel deep stable convolutional neural networksstat.ML

This paper removes the finite variance assumption for deep convolutional neural networks.

problem Removing the finite variance assumption for deep convolutional neural networks.

method Assuming iid parameters distributed according to a stable distribution, the paper shows that the infinite-channel limit of a deep feed-forward convolutional neural network is a multivariate stable stochastic process.

result The infinite-channel limit of a deep feed-forward convolutional neural network, under suitable scaling, is a multivariate stable stochastic process.

deep learning convolutional neural networks stable distributions infinite-width networks Gaussian processes

2021-02-07abs ↗pdf ↗

Exact marginal prior distributions of finite Bayesian neural networkscs.LG

Researchers derive exact priors for finite Bayesian neural networks.

problem Understanding non-Gaussian priors in finite Bayesian neural networks.

method Analytical derivation of function space priors for finite fully-connected feedforward networks.

result Exact solutions for priors of finite networks, including Meijer G-function for linear networks and mixtures for ReLU networks.

Bayesian neural networks finite networks function space priors Meijer G-function ReLU networks

2021-04-23abs ↗pdf ↗

Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processesmath.ST

Wide deep neural networks with Gaussian weights approximate Gaussian processes closely.

problem Understanding the approximation of deep neural networks with Gaussian weights to Gaussian processes.

method Established novel rates for the Gaussian approximation of random deep neural networks with Gaussian parameters and Lipschitz activation functions in the wide limit.

result The distance between the network output and the Gaussian approximation scales inversely with the width of the network.

deep neural networks Gaussian approximation Gaussian processes wide limit Lipschitz activation

2023-12-18abs ↗pdf ↗

Universal approximations of permutation invariant/equivariant functions by deep neural networkscs.LG

In this paper, we develop a theory about the relationship between $G$ -invariant/equivariant functions and deep neural networks for finite group $G$ . Especially, for a given $G$ -invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$ -actions and each affine t…

2019-03-05abs ↗pdf ↗

A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNscs.LG

Theory captures feature learning effects in finite CNNs.

problem Feature learning in finite deep neural networks.

method Derive a self-consistent Gaussian Process theory.

result Good agreement with experiments and sharp transition between regimes.

Gaussian Processes Feature Learning Finite DNNs Convolutional Neural Networks Noisy Gradient Descent

2021-06-08abs ↗pdf ↗

Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamicscs.LG

New framework analyzes deep learning optimization with finite width networks, revealing generalization gaps and excess risks.

problem Analyzing generalization error of deep learning with finite width networks.

method Formulating neural network training as transportation map estimation and analyzing via infinite dimensional Langevin dynamics.

result Achieves fast learning rate and minimax optimal rates for classification and regression problems.

deep learning generalization error transportation map Langevin dynamics neural networks

2020-07-11abs ↗pdf ↗

Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNsstat.ML

Deep neural networks learn by averaging fast variables, revealing a Gaussian process.

problem Analyzing the complex behavior of deep neural networks (DNNs) with billions of parameters.

method Identifying slow variables that average the erratic behavior of fast microscopic variables in fully trained DNNs.

result DNN layers couple only through the second moment (kernels) of their activations and pre-activations, which fluctuate in a nearly Gaussian manner.

deep learning neural networks feature learning Gaussian process thermodynamic theory

2021-12-31abs ↗pdf ↗

Deep Neural Network Approximation Theorycs.LG

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relat…

2019-01-08abs ↗pdf ↗

Deep Neural Networks as Gaussian Processesstat.ML

It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width. This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of eva…

2017-11-01abs ↗pdf ↗

Mechanisms of dimensionality reduction and decorrelation in deep neural networkscs.LG

Deep neural networks are widely used in various domains. However, the nature of computations at each layer of the deep networks is far from being well understood. Increasing the interpretability of deep neural networks is thus important. Here, we construct a mean-field framework to understand how compact representation…

2017-10-04abs ↗pdf ↗

What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorystat.ML

New framework explains deep neural networks using variational spline theory.

problem Understanding functions learned by deep neural networks.

method Developed a variational framework and function space.

result Deep ReLU networks are solutions to regularized data fitting problems over the proposed function space.

deep neural networks rectified linear units variational spline theory rectified functions Banach spaces

2021-05-07abs ↗pdf ↗

Depth induces scale-averaging in overparameterized linear Bayesian neural networkscs.LG

Study how depth affects inference in deep Bayesian neural networks.

problem Understanding how depth impacts inference in overparameterized linear Bayesian neural networks.

method Interpreting finite deep linear Bayesian neural networks as scale mixtures of Gaussian process predictors.

result Advances analytical understanding of how depth affects inference in a simple class of Bayesian neural networks.

Bayesian neural networks scale mixtures Gaussian processes inference depth effects

2021-11-23abs ↗pdf ↗

Quantitative CLTs in Deep Neural Networkscs.LG

Quantitative CLTs show neural network distributions converge to Gaussian as width increases.

problem Understanding the distribution of fully connected neural networks with random weights and biases.

method Analyzing the distribution of a fully connected neural network with random Gaussian weights and biases, proving quantitative bounds on normal approximations.

result The distance between a random fully connected network and the corresponding infinite width Gaussian process scales like

n^{-γ}

for

γ>0

quantitative CLTs deep neural networks random Gaussian weights finite width Gaussian process

2023-07-12abs ↗pdf ↗

Neural Contextual Bandits with Deep Representation and Shallow Explorationcs.LG

A new algorithm learns from raw feature vectors using deep neural networks and UCB for exploration.

problem Learning from raw feature vectors with unknown reward functions.

method Deep representation learning followed by UCB exploration in the last layer.

result Achieves

ilde{O}(\sqrt{T})

finite-time regret.

contextual bandits deep learning UCB neural networks exploration

2020-12-03abs ↗pdf ↗

Implicit regularization of deep residual networks towards neural ODEsstat.ML

Deep residual networks implicitly converge to neural ODEs.

problem Link between discrete and continuous deep learning models.

method Establishing implicit regularization for residual networks towards neural ODEs.

result Deep residual networks initialized as discretizations of neural ODEs converge to such ODEs during training.

deep learning neural networks neural ODEs gradient flow implicit regularization

2023-09-03abs ↗pdf ↗

Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysismath.DS

The paper introduces Causal Neural Operators to approximate operators in stochastic analysis.

problem Leveraging temporal structure in non-linear operators for deep learning models.

method Designing a deep learning model framework for infinite-dimensional linear metric spaces.

result Causal Neural Operators can uniformly approximate Hölder or smooth trace class operators.

Causal Neural Operators Stochastic Analysis Deep Learning Banach Spaces Temporal Structure

2022-10-24abs ↗pdf ↗

Estimation of the Mean Function of Functional Data via Deep Neural Networksstat.ML

Deep neural networks improve mean function estimation for functional data.

problem Estimating mean functions of functional data.

method Deep neural networks with ReLU activation, sparsely connected.

result Achieves optimal nonparametric convergence rate in empirical norm.

functional data deep learning mean function nonparametric regression ReLU activation

2020-12-08abs ↗pdf ↗

Neural reproducing kernel Banach spaces and representer theorems for deep networksstat.ML

Deep neural networks define suitable reproducing kernel Banach spaces.

problem Characterizing the function spaces of deep neural networks.

method Reproducing kernel Banach spaces and variational results.

result Deep neural networks define suitable reproducing kernel Banach spaces.

deep networks reproducing kernel Banach spaces representer theorems sparsity function spaces

2024-03-13abs ↗pdf ↗

On the Empirical Neural Tangent Kernel of Standard Finite-Width Convolutional Neural Network Architecturescs.LG

Empirical study shows standard CNNs deviate from NTK predictions.

problem Understanding how standard finite-width CNNs behave compared to their infinite-width NTK counterparts.

method Empirical analysis of AlexNet and LeNet architectures.

result Standard CNNs deviate significantly from their NTK counterparts, but deviation decreases with wider networks.

Neural Tangent Kernel Convolutional Neural Networks AlexNet LeNet Finite-width

2020-06-24abs ↗pdf ↗

Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)cs.LG

Paper studies ResNet dynamics using NTH, reducing width requirement.

problem Understanding ResNet dynamics and improving training efficiency.

method Uses Neural Tangent Hierarchy (NTH) to analyze ResNet dynamics.

result Reduces width requirement from quartic to cubic for ResNet.

ResNet Neural Tangent Kernel Gradient Descent Finite Width Skip-Connection

2020-07-07abs ↗pdf ↗

Rectangular knot diagrams classification with deep learningmath.GT

Deep learning classifies knots using rectangular diagrams.

problem Recognizing and distinguishing knots, especially the unknot.

method Represent knots as rectangular Dynnikov diagrams and use neural networks to classify them.

result Neural networks can effectively distinguish knots from each other.

knot classification neural networks rectangular diagrams Dynnikov moves unknotting problem

2020-11-06abs ↗pdf ↗

Neural Mixture Distributional Regressionstat.CO

NMDR estimates complex mixtures of distributions efficiently.

problem Estimating complex finite mixtures of distributions in high-dimensional settings.

method Flexible additive predictors, neural networks, and deep learning optimizers.

result Competitive performance in complex scenarios compared to existing approaches.

distributional regression neural networks finite mixtures high-dimensional data deep learning optimizers

2020-10-14abs ↗pdf ↗

Maximal Information Divergence from Statistical Models defined by Neural Networksmath.ST

We review recent results about the maximal values of the Kullback-Leibler information divergence from statistical models defined by neural networks, including naive Bayes models, restricted Boltzmann machines, deep belief networks, and various classes of exponential families. We illustrate approaches to compute the max…

2013-03-01abs ↗pdf ↗

Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networksstat.ML

This paper tightens bounds on the smallest eigenvalue of NTK for deep ReLU networks.

problem Analyzing the smallest eigenvalue of Neural Tangent Kernel for deep ReLU networks.

method Analyzing various quantities of independent interest, including lower bounds on the smallest singular value of hidden feature matrices and upper bounds on the Lipschitz constant of input-output feature maps.

result Tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU nets, both in the limiting case of infinite widths and for finite widths.

Neural Tangent Kernel ReLU Networks Eigenvalue Bounds Deep Learning Gradient Descent

2020-12-21abs ↗pdf ↗

Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networkscs.LG

SGD converges with positive probability for non-convex deep neural networks under specific conditions.

problem Convergence of SGD for non-convex deep neural networks.

method Established local convergence with positive probability under local Łojasiewicz condition and additional structural assumption.

result SGD converges with positive probability for non-convex deep neural networks under specific conditions.

SGD non-convex optimization deep neural networks Łojasiewicz condition local convergence

2023-04-18abs ↗pdf ↗

Optimal rates of approximation by shallow ReLU

^k

neural networks and applications to nonparametric regressionstat.ML

Optimal rates for shallow ReLU networks in nonparametric regression.

problem Approximating smooth and non-smooth functions with shallow ReLU networks.

method Analysis of shallow ReLU

^k

neural networks, using variation norms and deep learning theory.

result Optimal approximation rates for shallow ReLU networks in nonparametric regression.

ReLU networks nonparametric regression shallow neural networks convolutional neural networks Hölder functions

2023-04-04abs ↗pdf ↗

How do noise tails impact on deep ReLU networks?math.ST

This paper examines how noise affects deep neural networks and improves their performance.

problem The impact of noise on the stability of deep ReLU neural networks for nonparametric regression.

method Investigates the optimal rate of convergence for deep ReLU neural networks under Huber loss, considering the p-th moment of noise and the smoothness of the function.

result The optimal rate of convergence cannot be achieved by ordinary least squares but can be by Huber loss with a properly chosen parameter.

deep neural networks ReLU networks nonparametric regression Huber loss noise impact

2022-03-20abs ↗pdf ↗

Large and Deep Factor Modelsq-fin.ST

Deep neural networks decompose SDF into linear and nonlinear components.

problem Constructing accurate stochastic discount factors (SDFs) for pricing.

method Additive decomposition of a deep neural network trained to construct SDFs.

result The PTK representation delivers significant performance gains in equity data.

deep learning stochastic discount factor factor models neural networks pricing

2024-01-20abs ↗pdf ↗

Kernel Renormalization in Bayesian Deep Neural Networks: the Equivalent Wishart Ansatz in the Proportional Regimecs.LG

New approach predicts generalization of deep neural networks in proportional-width regime.

problem Predicting generalization of deep neural networks in proportional-width regime.

method Equivalent Wishart Ansatz for hierarchical empirical kernels, renormalized NNGP kernel.

result Renormalized NNGP kernel captures dominant stochastic fluctuations in deep neural networks.

Bayesian deep learning deep neural networks generalization Wishart Ansatz renormalization

2026-05-28abs ↗pdf ↗