Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,738 papers · 148 categories

Trend · papers per month

0111 · Jul 201919922001200920172026
9 results for memory-optimality

Study identifies three quantization regimes for ReLU networks.

problem Approximation of Lipschitz functions by ReLU networks with finite-precision weights.
method Established through nonasymptotic tight lower and upper bounds on minimax approximation error.
result Memory-optimality achieved in proper quantization regime for deep networks.

PyKEEN 1.0 simplifies KGE model creation and optimization.

problem Training and evaluating knowledge graph embeddings (KGEs).
method Composes KGEMs with various interaction models, training approaches, and loss functions. Implements automatic memory optimization and extensive HPO functionalities.
result PyKEEN 1.0 streamlines KGE model creation and optimization.

We propose two optimization techniques to minimize memory usage and computation while meeting system timing constraints for real-time classification in wearable systems. Our method derives a hierarchical classifier structure for Support Vector Machine (SVM) in order to reduce the amount of computations, based on the pr…

2019-07-07abs ↗pdf ↗

POET enables large neural network training on tiny devices with reduced energy.

problem Training large neural networks on memory-limited edge devices.
method Jointly optimizes rematerialization and paging for memory reduction, formulating an MILP for energy-efficient training.
result POET trains ResNet-18 and BERT within Cortex-M memory constraints, outperforming current methods in energy efficiency.

A new memory-efficient Adam variant reduces second moments when feasible.

problem Memory constraints in training machine learning models.
method Signal-to-Noise Ratio (SNR) analysis to identify dimensions where second moments can be replaced by means.
result Memory-efficient Adam variant (SlimAdam) matches performance and stability of Adam while saving up to 98% of second moments.

HybridSGD improves SGD performance by balancing computation and communication.

problem Limited scalability and performance of SGD due to communication costs.
method 2D parallel SGD method (HybridSGD) that trades off between 1D ss-step SGD and 1D Federated SGD (FedAvg).
result HybridSGD achieves better convergence than FedAvg at similar processor scales and up to 121x speedup over FedAvg.

Large deep learning models offer significant accuracy gains, but training billions to trillions of parameters is challenging. Existing solutions such as data and model parallelisms exhibit fundamental limitations to fit these models into limited device memory, while obtaining computation, communication and development …

2019-10-04abs ↗pdf ↗