On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “shallow tasks”

When MAML Can Adapt Fast and How to Assist When It Cannotcs.LG

MAML adapts faster with deeper architectures, especially in shallow tasks.

problem Understanding and improving MAML's fast adaptation.

method Empirical and theoretical studies on MAML's properties and optimization.

result MAML adapts better with deep architectures even for shallow tasks.

MAML meta-learning fast adaptation deep architectures shallow tasks

2019-10-30abs ↗pdf ↗

Hierarchically Compositional Tasks and Deep Convolutional Networkscs.LG

Convolutional networks outperform shallow classifiers on certain tasks due to hierarchical structure.

problem Understanding why convolutional networks outperform shallow classifiers on specific tasks.

method Approximation theory, visual tasks with deterministic scrambling, and network performance evaluation.

result Hierarchical structure is crucial for convolutional networks' performance on certain tasks, but not all.

convolutional networks deep learning approximation theory visual tasks scrambling

2020-06-24abs ↗pdf ↗

Chain-of-Thought Enhanced Shallow Transformers for Wireless Symbol Detectioncs.LG

CHOOSE enhances shallow Transformers for wireless symbol detection.

problem Improving wireless symbol detection with shallow Transformers.

method Introducing autoregressive latent reasoning steps within hidden space.

result Lightweight Transformers achieve comparable performance to deep models.

Transformers wireless communication symbol detection shallow models resource-constrained

2025-06-26abs ↗pdf ↗

A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexitycs.LG

This paper analyzes shallow ViTs, providing sample complexity and SGD behavior insights.

problem Theoretical understanding of shallow ViTs, especially their sample complexity and SGD behavior.

method Data model with label-relevant and label-irrelevant tokens, theoretical analysis of shallow ViT training.

result Characterization of sample complexity for zero generalization error in shallow ViTs.

Vision Transformers Shallow ViTs Sample Complexity SGD Generalization

2023-02-12abs ↗pdf ↗

Deep Learning the Ising Model Near Criticalitycond-mat.dis-nn

It is well established that neural networks with deep architectures perform better than shallow networks for many tasks in machine learning. In statistical physics, while there has been recent interest in representing physical data with generative modelling, the focus has been on shallow neural networks. A natural ques…

2017-08-15abs ↗pdf ↗

Investigating the Impact of CNN Depth on Neonatal Seizure Detection Performancestat.ML

This study presents a novel, deep, fully convolutional architecture which is optimized for the task of EEG-based neonatal seizure detection. Architectures of different depths were designed and tested; varying network depth impacts convolutional receptive fields and the corresponding learned feature complexity. Two deep…

2018-06-08abs ↗pdf ↗

Linearly Separable Features in Shallow Nonlinear Networks: Width Scales Polynomially with Intrinsic Data Dimensioncs.LG

Shallow nonlinear networks can separate classes linearly with polynomially scaling width.

problem Understanding the linear separability of deep networks' features.

method Modeling inputs as a union of low-dimensional subspaces and using random weights and quadratic activations.

result Shallow nonlinear networks can achieve linear separation with polynomially scaling width.

deep learning nonlinear networks linear separability shallow networks intrinsic dimensionality

2025-01-04abs ↗pdf ↗

Gradient Boosting Neural Networks: GrowNetcs.LG

GrowNet uses shallow neural networks for gradient boosting, outperforming existing methods.

problem Improving gradient boosting performance through shallow neural networks.

method Unified gradient boosting framework with shallow neural networks as weak learners, incorporating corrective steps.

result GrowNet outperformed state-of-the-art boosting methods in classification, regression, and learning to rank tasks.

gradient boosting neural networks weak learners corrective step classification

2020-02-19abs ↗pdf ↗

Shallow neural network representation of polynomialsstat.ML

Shallow neural networks can represent polynomials efficiently.

problem Representing polynomials using shallow neural networks.

method Using shallow neural networks of width

2(R+d)^d

to represent

d

-variate polynomials of degree

R

result Derives minimax optimal convergence rate for shallow networks to unknown univariate regression functions.

shallow neural networks polynomials Taylor polynomials minimax optimal convergence rate

2022-08-17abs ↗pdf ↗

Neural Variance-aware Dueling Bandits with Deep Representation and Shallow Explorationcs.LG

New algorithms reduce dueling bandits' regret with neural networks and efficient exploration.

problem Optimizing dueling bandits with neural networks for better performance.

method Combines shallow exploration strategies with neural networks for utility approximation, using iterative self-improvement and spectral analysis to reduce network width.

result Achieves sublinear regret of

\widetilde{\mathcal{O}}(d\sqrt{\sum_{t=1}^{T} σ_t^2} + \sqrt{dT})

dueling bandits neural networks shallow exploration regret minimization spectral analysis

2025-06-02abs ↗pdf ↗

A generalized quadratic loss for SVM and Deep Neural Networkscs.LG

Extends quadratic loss for SVM and deep learning to improve pattern correlation.

problem Improving generalization in supervised binary classification and regression tasks.

method Extends quadratic loss, restarts from problem (8) in [3], proposes new algorithms, uses multiple kernel learning.

result Comparable results with standard losses and parameterized quadratic loss.

quadratic loss SVM Deep Learning pattern correlation generalization

2021-02-15abs ↗pdf ↗

Asymptotic convexity of wide and shallow neural networksstat.ML

Wide and shallow networks approximate convex functions well.

problem Understanding why wide and shallow neural networks perform well.

method Analyzing the epigraph of the input-output map of shallow and wide neural networks.

result The epigraph of the input-output map approximates a convex function.

neural networks convexity shallow networks wide networks

2025-06-23abs ↗pdf ↗

Optimal rates of approximation by shallow ReLU

^k

neural networks and applications to nonparametric regressionstat.ML

Optimal rates for shallow ReLU networks in nonparametric regression.

problem Approximating smooth and non-smooth functions with shallow ReLU networks.

method Analysis of shallow ReLU

^k

neural networks, using variation norms and deep learning theory.

result Optimal approximation rates for shallow ReLU networks in nonparametric regression.

ReLU networks nonparametric regression shallow neural networks convolutional neural networks Hölder functions

2023-04-04abs ↗pdf ↗

How deep should be the depth of convolutional neural networks: a backyard dog case studycs.NE

The work concerns the problem of reducing a pre-trained deep neuronal network to a smaller network, with just few layers, whilst retaining the network's functionality on a given task The proposed approach is motivated by the observation that the aim to deliver the highest accuracy possible in the broadest range of oper…

2018-05-03abs ↗pdf ↗

Comparing Deep Learning Models for the Task of Volatility Prediction Using Multivariate Dataq-fin.ST

Study compares deep learning models for volatility prediction using multivariate data.

problem Predicting volatility using multivariate data.

method Evaluated multiple deep learning models including MLP, RNN, TCN, and Temporal Fusion Transformer.

result Temporal Fusion Transformer and TCN variants outperform classical models and shallow networks.

volatility prediction deep learning multivariate data temporal fusion transformer temporal convolutional networks

2023-06-20abs ↗pdf ↗

Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-levelcs.CL

This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015), on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016). Our findings are as follows. The shallow word-level CNN…

2016-08-31abs ↗pdf ↗

Any Deep ReLU Network is Shallowcs.LG

Deep ReLU networks can be simplified to a three-layer model.

problem Understanding the behavior of deep neural networks.

method Constructive proof and algorithm to transform deep networks into shallow ones.

result Deep ReLU networks can be represented by a simpler three-layer structure.

deep learning ReLU networks shallow networks neural networks algorithm

2023-06-20abs ↗pdf ↗

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioneess.AS

We present SpecAugment, a simple data augmentation method for speech recognition. SpecAugment is applied directly to the feature inputs of a neural network (i.e., filter bank coefficients). The augmentation policy consists of warping the features, masking blocks of frequency channels, and masking blocks of time steps. …

2019-04-18abs ↗pdf ↗

Eigen-GNN: A Graph Structure Preserving Plug-in for GNNscs.LG

Eigen-GNN enhances GNNs by preserving graph structures.

problem Existing shallow GNNs fail to effectively preserve graph structures.

method Integrates eigenspace of graph structures into GNNs as a dimensionality reduction module.

result Eigen-GNN boosts GNNs' ability to preserve graph structures without increasing depth.

Graph Neural Networks Eigen-GNN Graph Structure Preservation Dimensionality Reduction Node Classification

2020-06-08abs ↗pdf ↗

Nonparametric regression using over-parameterized shallow ReLU neural networksstat.ML

Deep neural networks can learn smooth functions without parameters.

problem Learning smooth functions from shallow ReLU neural networks.

method Using over-parameterized shallow ReLU neural networks with norm constraints.

result Least squares estimators based on shallow neural networks are minimax optimal.

deep learning ReLU networks nonparametric regression minimax optimality shallow neural networks

2023-06-14abs ↗pdf ↗

How do Minimum-Norm Shallow Denoisers Look in Function Space?stat.ML

The paper characterizes functions of shallow ReLU NN denoisers under minimal norm constraints.

problem Understanding the theoretical success of neural network denoisers.

method Characterization of functions realized by shallow ReLU NN denoisers under minimal norm constraints.

result The functions realized by shallow ReLU NN denoisers are contractive toward clean data points and generalize better than the empirical MMSE estimator at low noise levels.

Neural Networks Denoising Minimum-Norm ReLU Function Space

2023-11-12abs ↗pdf ↗

Exploiting Unsupervised Pre-training and Automated Feature Engineering for Low-resource Hate Speech Detection in Polishcs.CL

This paper presents our contribution to PolEval 2019 Task 6: Hate speech and bullying detection. We describe three parallel approaches that we followed: fine-tuning a pre-trained ULMFiT model to our classification task, fine-tuning a pre-trained BERT model to our classification task, and using the TPOT library to find …

2019-06-17abs ↗pdf ↗

Compositional Explanations of Neuronscs.LG

Describes explaining neurons in deep representations using compositional logical concepts.

problem Interpreting neuron behavior in deep neural networks.

method Identifying compositional logical concepts that closely approximate neuron behavior.

result Compositional explanations provide insights into model performance and allow for adversarial example creation.

neural networks interpretability deep learning neuron explanations adversarial examples

2020-06-24abs ↗pdf ↗

Greedy Layerwise Learning Can Scale to ImageNetcs.LG

Shallow supervised 1-hidden layer neural networks have a number of favorable properties that make them easier to interpret, analyze, and optimize than their deep counterparts, but lack their representational power. Here we use 1-hidden layer learning problems to sequentially build deep networks layer by layer, which ca…

2018-12-29abs ↗pdf ↗

Non-Vacuous Generalisation Bounds for Shallow Neural Networkscs.LG

New bounds for shallow neural networks with deterministic parameters.

problem Developing generalisation bounds for shallow neural networks.

method PAC-Bayesian theory applied to shallow neural networks with deterministic parameters.

result Empirical non-vacuous bounds for shallow neural networks trained with vanilla SGD.

shallow neural networks PAC-Bayesian theory generalisation bounds vanilla SGD MNIST

2022-02-03abs ↗pdf ↗

Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Networkcs.LG

NKN deep neural network learns governing equations and classifies images.

problem Learning governing equations and classifying images with deep neural networks.

method Nonlocal kernel network (NKN) that is resolution independent, deep, and handles various tasks.

result NKN outperforms baseline methods in learning governing equations and image classification tasks.

deep learning neural operators nonlocal diffusion resolution independence image classification

2022-01-06abs ↗pdf ↗

Multi-resolution Multi-task Gaussian Processesstat.ML

We consider evidence integration from potentially dependent observation processes under varying spatio-temporal sampling resolutions and noise levels. We develop a multi-resolution multi-task (MRGP) framework while allowing for both inter-task and intra-task multi-resolution and multi-fidelity. We develop shallow Gauss…

2019-06-19abs ↗pdf ↗

Efficient Toxicity Prediction via Simple Features Using Shallow Neural Networks and Decision Treescs.LG

Toxicity prediction of chemical compounds is a grand challenge. Lately, it achieved significant progress in accuracy but using a huge set of features, implementing a complex blackbox technique such as a deep neural network, and exploiting enormous computational resources. In this paper, we strongly argue for the models…

2019-01-26abs ↗pdf ↗

Sharp Lower Bounds on the Approximation Rate of Shallow Neural Networksstat.ML

Sharp lower bounds on shallow neural networks' approximation rates are derived.

problem The efficiency of shallow neural networks in approximating functions.

method Lower bounding the

L^2

-metric entropy and Kolmogorov

n

-widths of the convex hull of neural network basis functions.

result Sharp lower bounds on the approximation rates for shallow neural networks are provided.

shallow neural networks approximation rates metric entropy Kolmogorov n-widths convex hull

2021-06-28abs ↗pdf ↗

Expressivity of Shallow and Deep Neural Networks for Polynomial Approximationcs.LG

Study shows shallow ReLU networks struggle with high-dimensional Lipschitz functions.

problem Expressing high-dimensional Lipschitz functions with shallow ReLU networks.

method Established lower bounds on shallow network complexity for polynomial approximation.

result Shallow ReLU networks suffer from the curse of dimensionality for Lipschitz functions.

ReLU networks polynomial approximation curse of dimensionality Lipschitz functions

2023-03-06abs ↗pdf ↗

Inter-domain Deep Gaussian Processesstat.ML

Inter-domain Deep Gaussian Processes improve inference for non-stationary data.

problem Inference limitations in Gaussian processes for non-stationary data.

method Combines inter-domain and deep Gaussian processes for scalable approximate inference.

result Outperforms inter-domain shallow GPs and conventional DGPs on non-stationary data.

Gaussian processes deep learning non-stationary data approximate inference inter-domain

2020-11-01abs ↗pdf ↗

On the existence of optimal shallow feedforward networks with ReLU activationcs.LG

Proves existence of optimal shallow neural networks with ReLU activation.

problem Proving the existence of optimal shallow feedforward networks with ReLU activation.

method Proves existence of global minima in the loss landscape for continuous target functions using shallow feedforward neural networks with ReLU activation.

result Existence of global minima in the loss landscape for shallow feedforward networks with ReLU activation.

ReLU activation shallow neural networks global minima loss landscape continuous target functions

2023-03-06abs ↗pdf ↗

DiNo and RanBu: Lightweight Predictions from Shallow Random Forestscs.LG

Two new methods reduce random forest latency and improve accuracy.

problem High latency and memory demands in deep random forest models.

method DiNo and RanBu convert shallow random forests into efficient predictors.

result RanBu matches or exceeds full-depth random forest accuracy with up to 95% reduction in time.

random forests shallow trees tabular prediction inference latency kernel smoothing

2025-10-23abs ↗pdf ↗

Can Implicit Bias Imply Adversarial Robustness?cs.LG

Gradient-trained shallow networks can generalize well but are vulnerable to small-radius adversarial attacks.

problem Adversarial robustness of gradient-trained shallow networks.

method Analysis of neuron alignment and polynomial ReLU activation.

result Gradient-trained shallow networks with polynomial ReLU activation are robust to small-radius adversarial attacks.

Adversarial robustness Gradient flow Shallow networks Polynomial ReLU Neuron alignment

2024-05-24abs ↗pdf ↗

From Deep to Shallow: Transformations of Deep Rectifier Networkscs.LG

In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …

2017-03-30abs ↗pdf ↗

Proximal Mean Field Learning in Shallow Neural Networkscs.LG

A new algorithm for learning shallow neural networks with infinite width.

problem Learning shallow over-parameterized neural networks.

method Sinkhorn proximal algorithm approximating mean field learning dynamics.

result The algorithm performs gradient descent of the free energy associated with the risk functional.

Shallow neural networks Over-parameterization Mean field Sinkhorn algorithm Gradient descent

2022-10-25abs ↗pdf ↗

Limitations on approximation by deep and shallow neural networksstat.ML

Study shows limits on deep and shallow neural networks for approximating compact sets.

problem Understanding the limitations of deep and shallow neural networks in approximating compact sets.

method Proved Carl's type inequalities for approximation error, using Lipschitz widths.

result Lower bounds on approximation error for neural network outputs.

neural networks approximation error Carl's inequalities Lipschitz widths compact sets

2022-11-30abs ↗pdf ↗

Rates of Approximation by ReLU Shallow Neural Networkscs.LG

Paper studies shallow ReLU networks' approximation rates for Hölder functions.

problem Understanding shallow ReLU networks' efficiency in approximating Hölder functions.

method Analyzes rates of uniform approximation by ReLU shallow neural networks with

m

hidden neurons.

result Shows ReLU shallow neural networks can uniformly approximate Hölder functions with rates close to optimal.

ReLU networks Hölder spaces shallow neural networks approximation rates uniform approximation

2023-07-24abs ↗pdf ↗

Classifying textual data: shallow, deep and ensemble methodscs.CL

This paper focuses on a comparative evaluation of the most common and modern methods for text classification, including the recent deep learning strategies and ensemble methods. The study is motivated by a challenging real data problem, characterized by high-dimensional and extremely sparse data, deriving from incoming…

2019-02-18abs ↗pdf ↗

Deep Equals Shallow for ReLU Networks in Kernel Regimesstat.ML

Deep ReLU networks approximate as well as shallow ones in kernel regimes.

problem Understanding the limitations of kernel methods for deep ReLU networks.

method Characterizing eigenvalue decays of kernels derived from deep ReLU networks.

result Deep ReLU networks and shallow two-layer networks have equivalent approximation properties in kernel regimes.

deep learning ReLU networks kernel methods approximation theory eigenvalue decay

2020-09-30abs ↗pdf ↗

Deep orthogonal linear networks are shallowstat.ML

Deep networks without non-linearities are equivalent to shallow ones.

problem Training deep orthogonal linear networks with no non-linearity.

method Riemannian gradient descent and gradient descent on factorization.

result Training deep overparametrized networks is equivalent to shallow ones.

deep learning orthogonal networks gradient descent overparametrization implicit bias

2020-11-27abs ↗pdf ↗

Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machinescs.LG

Unified theorem for deep and shallow joint-equivariant machines.

problem Universal approximation of joint-equivariant machines.

method Constructive universal approximation theorem based on ridgelet transform.

result Unified approximation of deep and shallow networks.

universal approximation joint-equivariant ridgelet transform group representation deep networks

2024-05-22abs ↗pdf ↗

Deep Convolutional Transform Learning -- Extended versioncs.LG

New unsupervised learning technique learns independent kernels for better machine learning tasks.

problem Improving unsupervised representation learning for machine learning tasks.

method Stacking convolutional transforms using alternating proximal minimization scheme.

result DCTL outperforms shallow version CTL on benchmark datasets.

unsupervised learning representation learning convolutional transforms machine learning kernel learning

2020-10-02abs ↗pdf ↗

MSE-Optimal Neural Network Initialization via Layer Fusioncs.LG

FuseInit optimizes neural network initialization by fusing layers from deeper networks.

problem Initialization sensitivity of neural networks.

method Fusing neighboring layers from deeper networks trained with random initialization.

result Shallower networks initialized with FuseInit can perform as well as deeper networks.

neural networks initialization layer fusion stochastic gradient descent mean-square error

2020-01-28abs ↗pdf ↗

Efficient Proximal Mapping of the 1-path-norm of Shallow Networkscs.LG

New method for efficient proximal mapping of 1-path-norm in shallow networks.

problem Efficiently handling the 1-path-norm of shallow neural networks.

method Closed-form proximal operator for efficient computation and upper bound on Lipschitz constant.

result Proximal mapping allows robust training against adversarial perturbations.

1-path-norm shallow networks proximal operator Lipschitz constant adversarial robustness

2020-07-02abs ↗pdf ↗

Tangent Space Separability in Feedforward Neural Networkscs.LG

A new method approximates tangent spaces to simplify neural networks.

problem Efficiency of hierarchical neural networks is hindered by their complexity and training requirements.

method Approximates tangent subspace to enable sparse representation and switch to shallow networks.

result The method improves and sometimes surpasses the performance of original networks after a few epochs.

neural networks tangent spaces sparse representation shallow networks metric approximation

2019-12-18abs ↗pdf ↗

Shallow vs deep learning architectures for white matter lesion segmentation in the early stages of multiple sclerosiscs.LG

In this work, we present a comparison of a shallow and a deep learning architecture for the automated segmentation of white matter lesions in MR images of multiple sclerosis patients. In particular, we train and test both methods on early stage disease patients, to verify their performance in challenging conditions, mo…

2018-09-10abs ↗pdf ↗

On the Impact of Overparameterization on the Training of a Shallow Neural Network in High Dimensionsmath.OC

Study shows overparameterization helps shallow neural networks recover signals in high dimensions.

problem Signal recovery in shallow neural networks with overparameterization.

method Gradient flow on population risk, Gaussian distribution assumption, high-dimensional limit analysis.

result Minimal overparameterization is sufficient for strong recovery of signals.

signal recovery overparameterization shallow neural networks high dimensions gradient flow

2023-11-07abs ↗pdf ↗