On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “infinitely wide models”

Information in Infinite Ensembles of Infinitely-Wide Neural Networkscs.LG

Study on infinite neural networks for generalization.

problem Understanding generalization in infinitely-wide neural networks.

method Analytical and empirical investigations of infinite ensembles.

result Signals correlating with generalization identified.

neural networks infinite ensembles generalization information theory

2019-11-20abs ↗pdf ↗

Fast Approximation and Estimation Bounds of Kernel Quadrature for Infinitely Wide Modelsstat.ML

Study improves kernel quadrature for infinitely wide models with faster approximation and estimation rates.

problem Efficiently approximating and estimating expectations in infinitely wide models.

method Developed general kernel quadrature (GKQ) for parameter distributions, achieving faster rates.

result Achieved a fast approximation rate of

O(e^{-p})

and a fast estimation rate of

\widetilde{O}(1/n)

kernel quadrature infinitely wide models fast approximation estimation rates generalization performance

2019-02-02abs ↗pdf ↗

Asymptotics of Wide Convolutional Neural Networkscs.LG

Wide CNNs outperform infinite width networks, revealing scaling laws.

problem Understanding the performance difference between finite and infinite width convolutional networks.

method Diagrammatic approach to derive asymptotic width dependence for various quantities.

result The difference in performance between finite and infinite width models vanishes at a definite rate with respect to model width.

wide neural networks convolutional neural networks asymptotics scaling laws skip connections

2020-08-19abs ↗pdf ↗

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentstat.ML

Wide neural networks simplify to linear models under gradient descent.

problem Understanding the training dynamics of deep neural networks.

method Analyzing wide neural networks in the infinite width limit and showing they evolve as linear models.

result Gradient-based training of wide neural networks results in predictions from a Gaussian process with a specific kernel.

wide neural networks gradient descent linear models Gaussian process infinite width limit

2019-02-18abs ↗pdf ↗

How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterizationcs.LG

Wide neural networks can benefit from multi-task learning in their infinite-width limit.

problem The generalization behavior of wide neural networks in multi-task learning settings.

method Optimizing wide ReLU neural networks with L2-regularization promotes multi-task learning in the infinite-width limit.

result An exact quantitative characterization of multi-task learning in the infinite-width limit of wide ReLU neural networks.

multi-task learning wide neural networks infinite-width limit ReLU networks L2 regularization

2021-12-31abs ↗pdf ↗

Stable behaviour of infinitely wide deep neural networksstat.ML

Stable processes emerge as limits of deep neural networks with symmetric stable distributions.

problem Understanding the behavior of deep neural networks as they become infinitely wide.

method Analyzing fully connected feed-forward deep neural networks with symmetric stable distributions and showing the limit as a stable process.

result The infinite wide limit of the network is a stable process with multivariate stable distributions.

deep neural networks stable distributions infinite width stochastic processes symmetric distributions

2020-03-01abs ↗pdf ↗

Infinitely Wide Tensor Networks as Gaussian Processstat.ML

This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.

problem Understanding the relationship between Tensor Networks and Gaussian Processes.

method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.

result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.

Tensor Networks Gaussian Processes Infinite-width limit Neural Networks Kernel methods

2021-01-07abs ↗pdf ↗

Infinite-width limit of deep linear neural networkscs.LG

The paper examines how deep linear neural networks behave as they become infinitely wide.

problem Understanding the behavior of deep linear neural networks as they approach infinite width.

method Analyzes the infinite-width limit of deep linear neural networks, proving convergence to deterministic models and providing precise laws for random weights.

result The training dynamics of deep linear neural networks converge to those of a deterministic model, and the weights' behavior is precisely described.

deep learning neural networks infinite width gradient descent random weights

2022-11-29abs ↗pdf ↗

Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Modelscs.LG

Wide neural networks converge to Gaussian processes, improving generalization.

problem Understanding the generalization of wide neural networks, especially deep equilibrium models.

method Investigation of deep equilibrium models (DEQs) with infinite-depth layers, focusing on their convergence to Gaussian processes as width and depth approach infinity.

result Wide DEQs converge to Gaussian processes, maintaining generalization performance.

deep equilibrium models neural networks Gaussian processes benign overfitting infinite-depth networks

2023-10-16abs ↗pdf ↗

Infinitely Wide Graph Convolutional Networks: Semi-supervised Learning via Gaussian Processescs.LG

Infinitely wide GCNs perform as GPs for graph semi-supervised learning.

problem Graph-based semi-supervised classification with limited labeled data.

method Proposes GPGC model combining GCNs and GPs for semi-supervised learning.

result GPGC outperforms state-of-the-art methods on various datasets.

Graph Convolutional Networks Gaussian Processes Semi-supervised Learning Graph-based Classification Infinite Width

2020-02-26abs ↗pdf ↗

Infinite CNNs lose spatial correlations, but can be restored by correlated weights.

problem Infinite CNNs lose spatial correlations, which are crucial for their performance.

method Introduced correlated weights to restore spatial correlations in infinite CNNs.

result Optimal performance is achieved with a moderate level of weight correlation.

deep learning convolutional neural networks infinite limits correlated weights spatial correlations

2021-01-11abs ↗pdf ↗

On Infinite-Width Hypernetworkscs.LG

Wide hypernetworks don't guarantee convergence under gradient descent.

problem Theoretical guarantees for wide hypernetworks in over-parameterized settings.

method Analyzing infinitely wide hypernetworks and their convergence properties.

result Infinitely wide hypernetworks do not guarantee convergence to a global minimum under gradient descent.

hypernetworks over-parameterization wide architectures gradient descent convergence

2020-03-27abs ↗pdf ↗

Infinitely wide limits for deep Stable neural networks: sub-linear, linear and super-linear activation functionscs.LG

Study of deep Stable neural networks with various activation functions.

problem Characterizing the infinitely wide limits of deep Stable neural networks.

method Investigation of large-width properties of deep Stable NNs with a generalized central limit theorem for heavy tails.

result Extension of characterization to a broader class of activation functions, including sub-linear, asymptotically linear, and super-linear functions.

Deep learning Stable distributions Neural networks Activation functions Infinitely wide limits

2023-04-08abs ↗pdf ↗

Convexifying Sparse Interpolation with Infinitely Wide Neural Networks: An Atomic Norm Approachcs.LG

This study uses neural networks to solve interpolation problems with sparse, infinitely wide layers.

problem Exact data interpolation using sparse, infinitely wide neural networks.

method Atomic norm framework to derive convex hulls and equivalent convex formulations.

result Simple characterizations of convex hulls for different constraints on network weights and biases.

sparse neural networks interpolation atomic norm leaky rectified linear units convex hulls

2020-07-15abs ↗pdf ↗

A comparison between initialization strategies for the infinite hidden Markov modelstat.ME

Study evaluates initialization strategies for infinite hidden Markov models.

problem Limited attention to initialization in infinite hidden Markov models.

method Systematically evaluated distance-based clustering, model-based, and uniform initializations.

result Distance-based clustering initializations consistently outperform other methods.

infinite hidden Markov models initialization strategies distance-based clustering model-based initialization uniform initialization

2025-12-03abs ↗pdf ↗

Harnessing the Power of Infinitely Wide Deep Nets on Small-data Taskscs.LG

Infinitely wide neural nets perform well on small datasets.

problem Performing well on small datasets with limited training samples.

method Using Neural Tangent Kernels (NTKs) for kernel regression.

result NTK SVM outperforms Random Forests and Convolutional NTK on CIFAR-10 with 10-640 training samples.

Neural Tangent Kernels Deep Learning Small-Data Tasks Kernel Regression CIFAR-10

2019-10-03abs ↗pdf ↗

On the relationship between multivariate splines and infinitely-wide neural networkscs.LG

Multivariate splines linked to infinitely-wide neural networks with improved numerical performance.

problem Understanding the relationship between multivariate splines and neural networks.

method Showed multivariate splines can be represented as random features in infinitely-wide neural networks with a homogeneous activation function.

result The function space of multivariate splines is a Sobolev space on a Euclidean ball with explicit norm bounds on derivatives.

multivariate splines neural networks random features Sobolev space rectified linear unit

2023-02-07abs ↗pdf ↗

Infinite attention: NNGP and NTK for deep attention networksstat.ML

Attention mechanisms in deep learning become Gaussian process-like as the number of heads increases.

problem Understanding the behavior of attention mechanisms in deep learning models.

method Extending the equivalence between wide neural networks and Gaussian processes to attention architectures.

result Multi-head attention architectures behave as Gaussian processes as the number of heads tends to infinity.

attention mechanisms deep learning Gaussian processes neural networks multi-head

2020-06-18abs ↗pdf ↗

What Can Be Learnt With Wide Convolutional Neural Networks?stat.ML

Study on infinitely-wide CNNs and their adaptability to function spatial scales.

problem Understanding how CNNs efficiently learn high-dimensional functions and their adaptability to function spatial scales.

method Study infinitely-wide deep CNNs in the kernel regime, characterizing their spectrum and using generalisation bounds to prove adaptability.

result Deep CNNs adapt to the spatial scale of the target function, with error decay controlled by the effective dimensionality of function subsets.

Convolutional Neural Networks Kernel Regime Generalization Error Spatial Scale Hierarchical Structure

2022-08-01abs ↗pdf ↗

Infinitely imbalanced binomial regression and deformed exponential familiesmath.ST

The logistic regression model is known to converge to a Poisson point process model if the binary response tends to infinitely imbalanced. In this paper, it is shown that this phenomenon is universal in a wide class of link functions on binomial regression. The proof relies on the extreme value theory. For the logit, p…

2013-03-29abs ↗pdf ↗

Band width and the Rosenberg indexmath.KT

The Rosenberg index vanishes if a manifold admits a wide Riemannian band or cube-like domain.

problem Proving the Rosenberg index does not vanish for certain manifolds.

method Analyzing isometric immersions of wide Riemannian bands and cube-like domains on spin manifolds.

result Closed spin manifolds with infinite

\mathcal{KO}

-width have non-vanishing Rosenberg index.

Riemannian manifold spin manifold Rosenberg index higher index isometric immersion

2021-08-19abs ↗pdf ↗

Property

R_{\infty}

for groups with infinitely many endsmath.GR

Accessible groups with infinitely many ends have infinitely many twisted conjugacy classes.

problem Characterizing groups with infinitely many ends and their conjugacy classes.

method Analyzing accessible groups and relatively hyperbolic groups to deduce properties.

result Groups with infinitely many ends have infinitely many twisted conjugacy classes.

accessible groups infinitely many ends twisted conjugacy classes relatively hyperbolic groups undecidability

2025-04-16abs ↗pdf ↗

Finite Versus Infinite Neural Networks: an Empirical Studycs.LG

Empirical study compares wide neural networks to kernel methods, resolving open questions.

problem Understanding the relationship between wide neural networks and kernel methods.

method Large-scale empirical study using various neural network architectures and kernel methods.

result Wide neural networks outperform fully-connected finite-width networks in some cases, but underperform convolutional finite-width networks.

neural networks kernel methods finite width infinite width CIFAR-10

2020-07-31abs ↗pdf ↗

Neural Tangent Kernel Analysis of Deep Narrow Neural Networkscs.LG

Study of infinitely deep but narrow neural networks using NTK theory.

problem Analyzing the role of depth in deep learning with overparameterized networks.

method Infinite-depth limit analysis of MLP and CNN using Neural Tangent Kernel (NTK) theory.

result Established trainability guarantee for infinitely deep but narrow neural networks.

Neural Tangent Kernel Deep Learning Overparameterization Infinite Depth Narrow Neural Networks

2022-02-07abs ↗pdf ↗

Infinite-Dimensional Sparse Learning in Linear System Identificationeess.SY

Proposes an algorithm for infinite-dimensional sparse learning in system identification.

problem System identification without known model structures.

method Atomic norm regularization and greedy algorithm for solving an infinite-dimensional group lasso problem.

result The proposed algorithm outperforms benchmark methods in impulse response fitting and pole location estimation.

system identification sparse learning atomic norm group lasso greedy algorithm

2022-03-28abs ↗pdf ↗

Revisiting the Neural Tangent Kernel: the role of large width and depthcs.LG

This work challenges the Neural Tangent Kernel's role in overparameterized neural networks, especially with large width and depth.

problem The Neural Tangent Kernel's behavior in overparameterized neural networks with large width and depth is unclear.

method Experimental and theoretical analysis of ReLU networks with large width and depth.

result The aggregate norm of hidden neuron deviations does not vanish in infinitely-wide ReLU networks, indicating non-trivial behavior.

Neural Tangent Kernel ReLU networks large width large depth overparameterization

2025-11-10abs ↗pdf ↗

How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspectivecs.LG

New learning rules for wide neural networks without backpropagation.

problem Training wide neural networks efficiently and without backpropagation.

method Input-weight alignment driven by gradient descent in the NTK regime.

result Biologically-motivated learning rules equivalent to backpropagation in wide networks.

wide neural networks input-weight alignment gradient descent Neural Tangent Kernel backpropagation-free

2021-06-15abs ↗pdf ↗

Satellites of Infinite Rank in the Smooth Concordance Groupmath.GT

We conjecture that satellite operations are either constant or have infinite rank in the concordance group. We reduce this to the difficult case of winding number zero satellites, and use $SO(3)$ gauge theory to provide a general criterion sufficient for the image of a satellite operation to generate an infinite rank s…

2018-09-11abs ↗pdf ↗

Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approachcs.LG

This paper explains robust overfitting in wide DNNs using adversarial training and NTK theory.

problem Robust overfitting in adversarially trained wide DNNs.

method Theoretical analysis using neural tangent kernel (NTK) theory and adversarial training dynamics.

result Adversarial training can lead to robust overfitting in wide DNNs, which can be mitigated by the proposed Adv-NTK method.

robust overfitting adversarial training wide DNNs neural tangent kernel NTK theory

2023-10-09abs ↗pdf ↗

Neural Tangents: Fast and Easy Infinite Neural Networks in Pythonstat.ML

Neural Tangents simplifies infinite-width neural networks for research.

problem Training and studying infinite-width neural networks.

method High-level API for specifying complex architectures, analytical or gradient-based training, and automatic distribution.

result Analytical training of infinite-width networks and automatic parallelization.

neural networks infinite width infinite-width networks Bayesian inference gradient descent

2019-12-05abs ↗pdf ↗

α

-Stable convergence of heavy-tailed infinitely-wide neural networksstat.ML

Deep neural networks with heavy-tailed weights converge to stable distributions.

problem Understanding the convergence of heavy-tailed weights in infinitely-wide neural networks.

method Analyzing infinitely-wide multi-layer perceptrons with i.i.d. symmetric

α

-stable weight distributions.

result The vector of pre-activation values converges to i.i.d. symmetric

α

-stable distributions.

deep learning neural networks heavy-tailed distributions stability infinite width

2021-06-18abs ↗pdf ↗

On the Equivalence between Neural Network and Support Vector Machinestat.ML

This work establishes the equivalence between neural networks and support vector machines.

problem Establishing the equivalence between neural networks and support vector machines.

method Proposed a method to establish the equivalence between infinitely wide neural networks trained by soft margin loss and standard soft margin SVMs with NTK trained by subgradient descent.

result The equivalence between NN and SVM is established, enabling practical applications such as non-vacuous generalization bounds and robustness certificates.

neural networks support vector machines equivalence generalization robustness

2021-11-11abs ↗pdf ↗

Deterministic Value-Policy Gradientscs.LG

Paper improves reinforcement learning efficiency with deterministic value gradients.

problem High sample complexity in model-free DDPG algorithms for continuous control tasks.

method Proposes DVG and DVPG algorithms with infinite horizon value gradients to improve sample efficiency.

result DVPG algorithm substantially outperforms state-of-the-art methods on continuous control benchmarks.

reinforcement learning DDPG value gradients sample efficiency continuous control

2019-09-09abs ↗pdf ↗

Tight Regret Bounds for Infinite-armed Linear Contextual Banditsstat.ML

Paper tackles infinite action linear bandits with tight regret bounds.

problem Linear contextual bandit with infinite action sets.

method Proves a regret upper bound of

O(\sqrt{d^2T\log T}) imes ext{poly}(\log\log T)

result Upper bound matches previous lower bound of

Ω(\sqrt{d^2 T\log T})

up to iterated logarithmic terms.

Linear bandits Infinite action sets Regret bounds Contextual decision making

2019-05-04abs ↗pdf ↗

Large-width asymptotics for ReLU neural networks with

α

-Stable initializationscs.LG

This paper studies large-width asymptotics for ReLU neural networks with α-Stable initializations.

problem Characterizing the large-width behavior of ReLU neural networks with α-Stable initializations.

method Analysis of the large-width distributions and training dynamics of ReLU neural networks initialized with α-Stable distributions.

result For ReLU neural networks with α-Stable initializations, the large-width training dynamics achieve zero training error at a linear rate, characterized by a random kernel.

ReLU neural networks α-Stable distributions large-width asymptotics training dynamics random kernel

2022-06-16abs ↗pdf ↗

Hedging Cryptocurrency Optionsq-fin.PR

Study the hedging of cryptocurrency options in a volatile market.

problem Hedging options in a volatile, non-stationary cryptocurrency market.

method Calibrated to SVI-implied volatility surfaces, Monte Carlo price paths generated using SVCJ, GARCH, and historical data. Delta, Delta-Gamma, Delta-Vega, and Minimum Variance strategies applied. Wide range of market models tested.

result Calibration results indicate stochastic volatility, low jump frequency, and infinite activity. Short-dated options less sensitive to volatility or Gamma hedges; longer-dated options benefit from multiple-instrument hedges.

cryptocurrency hedging options volatility jump diffusion

2021-11-23abs ↗pdf ↗

Preconditioned Langevin Dynamics with Score-Based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problemsstat.ML

Paper analyzes Langevin dynamics for solving infinite-dimensional Bayesian inverse problems.

problem Solving high-dimensional Bayesian inverse problems in infinite-dimensional function spaces.

method Preconditioned Langevin dynamics with score-based generative models (SGMs).

result Derives error estimates and sufficient conditions for global convergence in Kullback-Leibler divergence.

Bayesian inverse problems infinite-dimensional spaces Langevin dynamics score-based models preconditioning

2025-05-23abs ↗pdf ↗

Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacesstat.ML

Deep neural nets estimate operators between infinite-dimensional spaces with fast rates.

problem Estimating operators between infinite-dimensional spaces.

method Deep neural networks for nonparametric estimation of Lipschitz operators.

result Error bounds decay with fast rates depending on intrinsic dimension.

deep learning operators infinite dimensions Lipschitz generalization error

2022-01-01abs ↗pdf ↗

Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regimecs.LG

Softmax policy gradient achieves global optimality in wide neural networks with entropy regularization.

problem Optimizing softmax policies with neural networks in the mean-field regime.

method Modeling neural networks as Wasserstein gradient flows and proving global optimality of fixed points.

result Global optimality of softmax policy gradient in wide single hidden layer neural networks with entropy regularization.

softmax policy neural networks policy gradient mean-field global optimality

2020-10-22abs ↗pdf ↗

Catoni-style Confidence Sequences under Infinite Variancemath.ST

The paper extends confidence sequences for infinite variance data.

problem Addressing confidence sequences for distributions with infinite variance.

method Establishing lower bounds and deriving tight confidence sequences for relaxed bounded

p^{th}

-moment distributions.

result Derived confidence sequences are tighter than those using Dubins-Savage inequality.

confidence sequences infinite variance relaxed bounded moment Dubins-Savage inequality

2022-08-05abs ↗pdf ↗

DinTucker: Scaling up Gaussian process models on multidimensional arrays with billions of elementscs.LG

Infinite Tucker Decomposition (InfTucker) and random function prior models, as nonparametric Bayesian models on infinite exchangeable arrays, are more powerful models than widely-used multilinear factorization methods including Tucker and PARAFAC decomposition, (partly) due to their capability of modeling nonlinear rel…

2013-11-12abs ↗pdf ↗

Avoiding pathologies in very deep networksstat.ML

Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance. To shed light on this problem, we analyze the analogous problem of constructing useful priors on compositions of functions. Specifically, we study the deep Gaussian process, a type of infinitely…

2014-02-24abs ↗pdf ↗

A self-supervised learning approach for denoising autoregressive models with additive noise: finite and infinite variance casesstat.ME

Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.

problem Denoising autoregressive signals corrupted by heavy-tailed noise.

method Self-supervised learning approach without requiring full noise distribution knowledge.

result Strong denoising performance compared to baseline methods, especially for impulsive noise.

autoregressive models denoising self-supervised learning heavy-tailed noise finite variance

2025-08-18abs ↗pdf ↗

Distributed, partially collapsed MCMC for Bayesian Nonparametricsstat.ML

New algorithm speeds Bayesian nonparametric model inference.

problem Slow inference in Bayesian nonparametric models.

method Decompose random measures into finite and infinite sub-measures; use different algorithms for each.

result Hybrid algorithm improves scalability and mixing.

Bayesian nonparametrics collapsed MCMC distributed computing random measures asymptotic convergence

2020-01-15abs ↗pdf ↗

Conformal Predictions for Longitudinal Datastat.ML

LPCI provides valid prediction intervals for longitudinal data.

problem Current conformal prediction methods for time series data lack cross-sectional coverage when applied to longitudinal datasets.

method Modeling residual data as a quantile fixed-effects regression problem, constructing prediction intervals with a trained quantile regressor.

result LPCI achieves valid cross-sectional coverage and outperforms existing benchmarks in terms of longitudinal coverage rates.

conformal prediction longitudinal data time series quantile regression cross-sectional coverage

2023-10-04abs ↗pdf ↗

Twisted Alexander polynomials of 2-bridge knots for parabolic representationsmath.GT

In this paper we show that the twisted Alexander polynomial associated to a parabolic representation determines fiberedness and genus of a wide class of 2-bridge knots. As a corollary we give an affirmative answer to a conjecture of Dunfield, Friedl and Jackson for infinitely many hyperbolic knots.

2013-01-07abs ↗pdf ↗

Characteristics of Monte Carlo Dropout in Wide Neural Networkscs.LG

Study on MC dropout in wide neural networks and its convergence to Gaussian processes.

problem Understanding the behavior of Monte Carlo dropout in wide neural networks.

method Rigorously studied the limiting distribution of wide untrained NNs under dropout, proving convergence to Gaussian processes. Investigated correlations and non-Gaussian behavior in finite width NNs.

result Wide untrained neural networks under dropout converge to Gaussian processes for fixed sets of weights and biases.

Monte Carlo dropout wide neural networks Bayesian inference Gaussian processes uncertainty estimation

2020-07-10abs ↗pdf ↗

Universal Approximation Theorem for Equivariant Maps by Group CNNsstat.ML

Unified method for CNNs to approximate equivariant maps across various groups.

problem Limited universal approximation theorems for CNNs with specific groups and settings.

method Unified approach to derive universal approximation theorems for equivariant maps by CNNs in diverse settings.

result Ability to handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.

CNNs equivariant maps universal approximation group symmetry non-compact groups

2020-12-27abs ↗pdf ↗