On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,042 papers · 148 categories

Trend · papers per month

count % of corpus

3 results for “Supermasks”

Supermasks in Superpositioncs.LG

SupSup model learns thousands of tasks without forgetting, using randomly initialized subnetworks.

problem Sequentially learning many tasks without forgetting.

method Randomly initialized base network with task-specific subnetworks (supermasks).

result Gradient-based optimization can identify the correct subnetwork for new tasks.

Sequential learning Catastrophic forgetting Random initialization Subnetworks Gradient optimization

2020-06-26abs ↗pdf ↗

Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcs.LG

New insights into why sparse networks perform well, including Supermasks.

problem Understanding why sparse networks trained from scratch perform better than non-sparse models.

method Analyzed three critical components of the Lottery Ticket algorithm: zeroing weights, signs, and masking.

result Discovered Supermasks that can improve performance of untrained networks.

Lottery Ticket Hypothesis Sparse Networks Supermasks MNIST CIFAR-10

2019-05-03abs ↗pdf ↗

Multicoated and Folded Graph Neural Networks with Strong Lottery Ticketscs.LG

This research uncovers high-performing subnetworks in deep GNNs without training.

problem Challenges in applying SLTH to deeper GNNs with high memory requirements.

method Introduces Multicoated Supermasks (M-Sup) and Multi-Stage Folding for GNNs.

result Uncovered untrained recurrent networks with performance similar to trained models.

graph neural networks pruning memory efficiency memory reduction performance

2023-12-06abs ↗pdf ↗