On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,786 papers · 148 categories

Trend · papers per month

count % of corpus

2 results for “shifted-ReLU”

Single-bit-per-weight deep convolutional neural networks without batch-normalization layers for embedded systemscs.LG

New approach replaces BN layers with shifted-ReLU for embedded systems.

problem Complexity and overheads of BN layers in low-power embedded systems.

method Used shifted-ReLU layers instead of BN layers in wide residual networks.

result Shifted-ReLU layers offer advantages in speed, memory, and complexity without significant accuracy loss.

batch-normalization deep learning embedded systems shifted-ReLU CNN

2019-07-16abs ↗pdf ↗

Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networkscs.LG

A new optimizer DDC improves deep learning models by respecting symmetries.

problem Deep networks' loss is invariant to continuous symmetries, leading to optimization issues.

method DDC builds a Dead-Direction Conditioner that lifts a base optimizer into a G-equivariant one, preserving the quotient geometry.

result DDCAdam and DDCMuon outperform standard optimizers in various tasks, improving validation-train loss gaps and learning dynamics.

Deep Learning Optimization Symmetry Gauge Equivariance Adam

2026-06-28abs ↗pdf ↗