On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

count % of corpus

7 results for “k-layer”

Best k-layer neural network approximationscs.LG

No best k-layer neural network approximations exist in general for common activations.

problem The non-existence of best k-layer neural network approximations for common activation functions.

method Empirical Risk Minimization (ERM) for neural networks with k layers and various activation functions.

result For k = 2 and common activations like ReLU, hyperbolic tangent, and sigmoid, the ERM infimum is not attainable in general.

neural networks k-layer ReLU tangent sigmoid

2019-07-02abs ↗pdf ↗

Benefits of depth in neural networkscs.LG

For any positive integer $k$ , there exist neural networks with $Θ(k^3)$ layers, $Θ(1)$ nodes per layer, and $Θ(1)$ distinct parameters which can not be approximated by networks with $\mathcal{O}(k)$ layers unless they are exponentially large --- they must possess $Ω(2^k)$ nodes. This result is proved here for a class o…

2016-02-14abs ↗pdf ↗

Biclustering Methods via Sparse Penaltystat.ML

New sparse penalty improves biclustering for gene expression data.

problem Identifying significant clusters in gene expression data.

method Prenet penalty applied to SSVD for biclustering.

result Mixed Prenet penalty effectively clusters non-overlapped data.

biclustering gene expression sparse penalty SSVD factor analysis

2023-08-28abs ↗pdf ↗

Differentiable Linearized ADMMcs.LG

Proposes D-LADMM for solving constrained optimization problems.

problem Constrained optimization problems with ill-posed inverse problems.

method Introduces Differentiable Linearized ADMM (D-LADMM) with learnable weights and activation functions.

result Rigorously proves globally converged solutions for D-LADMM.

Differentiable Linearized ADMM Constrained Optimization Learning-based Optimization Convergence Analysis Inverse Problems

2019-05-15abs ↗pdf ↗

Transformers on Markov Data: Constant Depth Sufficescs.LG

Transformers can learn Markov processes with constant depth, surprising results.

problem Understanding how transformers learn context in Markov processes.

method Empirical study and theoretical analysis of attention-based transformers on Markov data.

result Transformers with constant depth can achieve low test loss on Markov sequences, matching empirical and theoretical findings.

transformers Markov processes context learning attention empirical evidence

2024-07-25abs ↗pdf ↗

Probabilistic bounds on neuron death in deep rectifier networkscs.LG

Probabilistic bounds on neuron death in deep networks, showing depth can be increased indefinitely.

problem Understanding neuron death in deep rectifier networks and its impact on model trainability.

method Deriving upper and lower bounds on neuron death probability as a function of model hyperparameters.

result The probability of neuron death decreases as network depth increases, provided width increases proportionally.

neuron death deep networks ReLU networks trainability probability bounds

2020-07-13abs ↗pdf ↗

An Optimal Multi-layer Reinsurance Policy under Conditional Tail Expectationstat.ME

A usual reinsurance policy for insurance companies admits one or two layers of the payment deductions. Under optimal criterion of minimizing the conditional tail expectation (CTE) risk measure of the insurer's total risk, this article generalized an optimal stop-loss reinsurance policy to an optimal multi-layer reinsur…

2017-01-19abs ↗pdf ↗