On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

count % of corpus

6 results for “over-parameterised”

Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learningstat.ML

Wide neural networks with asymmetrical node scaling converge globally and learn features.

problem Global convergence and feature learning in over-parameterised shallow networks.

method Gradient-based optimisation of wide, shallow neural networks with asymmetrical node scaling.

result Gradient flow and gradient descent converge to a global minimum and learn features, unlike in the NTK parameterisation.

neural networks global convergence feature learning asymmetrical scaling gradient descent

2023-02-02abs ↗pdf ↗

Wide stochastic networks: Gaussian limit and PAC-Bayesian trainingstat.ML

Wide stochastic networks show Gaussian behavior and improve training with PAC-Bayesian methods.

problem Analyzing and training over-parameterised neural networks with large width.

method Establishing Gaussian behavior for a stochastic architecture, applying PAC-Bayesian training.

result PAC-Bayesian training on large but finite-width networks outperforms standard methods.

wide networks stochastic architectures PAC-Bayesian Gaussian limit over-parameterisation

2021-06-17abs ↗pdf ↗

Generalisation dynamics of online learning in over-parameterised neural networksstat.ML

Linear generalisation error with network size in neural networks.

problem Understanding generalisation in over-parameterised neural networks.

method Teacher-student setup with SGD dynamics captured by differential equations.

result Generalisation error increases linearly with network size.

neural networks generalisation over-parameterised SGD differential equations

2019-01-25abs ↗pdf ↗

Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setupstat.ML

Study on SGD dynamics in neural networks, revealing generalisation patterns.

problem Understanding generalisation in over-parameterised neural networks.

method Analysis of SGD dynamics in a teacher-student setup using differential equations.

result Network size affects generalisation error differently depending on training layers and activation functions.

neural networks stochastic gradient descent teacher-student setup generalisation error over-parameterisation

2019-06-18abs ↗pdf ↗

VINNAS: Variational Inference-based Neural Network Architecture Searchcs.LG

VINNAS uses variational inference to avoid mode collapse in neural architecture search.

problem Mode collapse in gradient-based NAS methods, leading to suboptimal architectures.

method Differentiable variational inference with variational dropout and automatic relevance determination.

result State-of-the-art accuracy with up to twice fewer non-zero parameters.

neural architecture search gradient-based NAS mode collapse variational inference sparse convolutional networks

2020-07-12abs ↗pdf ↗

Layer Dynamics of Linearised Neural Netscs.LG

This research explores the dynamics of linearised neural nets, revealing distinct learning phases and layer growth rates.

problem Understanding the fundamental mechanics of neural nets and their learning dynamics.

method Derivation of properties of learning dynamics in general multi-layer linear neural nets, including orthogonal networks.

result Linear multi-layer neural nets exhibit distinct phases of learning with different layer growth rates, and nonlinearity affects these dynamics.

neural nets learning dynamics layer design linear networks nonlinearity

2019-04-24abs ↗pdf ↗