On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

count % of corpus

13 results for “supernet”

AlphaNet: Improved Training of Supernets with Alpha-Divergencecs.CV

AlphaNet improves supernets training with alpha-divergence.

problem Improving the uncertainty distillation in weight-sharing NAS.

method Proposes alpha-divergence for better uncertainty distillation in supernets.

result Significant improvements in model performance across various FLOPs regimes.

supernets alpha-divergence uncertainty distillation weight-sharing NAS neural architecture search

2021-02-16abs ↗pdf ↗

Neural Architecture Search as Sparse Supernetcs.CV

Paper tackles NAS problem by modeling it as a sparse supernet.

problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.

method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.

result Proposed method finds compact, general, and powerful neural architectures.

Neural Architecture Search Sparse Supernet Mixed-Path Search Hierarchical Accelerated Proximal Gradient Bi-level Optimization

2020-07-31abs ↗pdf ↗

SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Searchcs.LG

SCARLET-NAS improves neural architecture search by stabilizing feature inconsistencies.

problem Difficulty in ranking subnetworks due to feature inconsistency in skip connections.

method Introduced an equivariant learnable stabilizer to homogenize feature disparities.

result SCARLET-A achieves 76.9% top-1 accuracy on ImageNet.

Neural Architecture Search Skip Connections Feature Inconsistency Stabilizer Supernet

2019-08-16abs ↗pdf ↗

SuperNet -- An efficient method of neural networks ensemblingcs.LG

SuperNet speeds up neural network ensembling by training a single DNN for various phases.

problem High computational demand of ensembling large neural networks.

method Train a single DNN for multiple phases of training to represent various sub-models.

result SuperNet execution time comparable to single DNN training time plus coupling factors.

neural networks ensembles compression training phases overfitting

2020-03-29abs ↗pdf ↗

Efficient Novelty-Driven Neural Architecture Searchcs.LG

This paper improves neural architecture search by focusing on novelty-driven sampling.

problem Lack of positive correlation between validation accuracy and test accuracy in One-Shot NAS.

method Single-path supernet with optimized weights and novelty search for architecture sampling.

result Novelty search method achieves state-of-the-art test error rate on CIFAR-10.

Neural Architecture Search Novelty Search One-Shot NAS Supernet Efficiency

2019-07-22abs ↗pdf ↗

FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcs.LG

FairNAS improves fairness in neural architecture search evaluation.

problem Improving accuracy of model ranking in neural architecture search.

method Proposes expectation and strict fairness constraints to ensure equal optimization opportunities for all blocks.

result Demonstrates improved model ranking and accuracy with FairNAS-A.

neural architecture search fairness supernet evaluation multi-objective evolutionary search

2019-07-03abs ↗pdf ↗

Provable Pathways: Learning Multiple Tasks over Multiple Pathscs.LG

This research improves multitask learning by creating task-specific pathways.

problem Creating task-specific representations for efficient learning.

method Developed generalization bounds for learning multiple tasks over multiple pathways.

result Proves the superiority of multipath representation over traditional shallow supernets.

multitask learning pathway statistical learning generalization bounds supernets

2023-03-08abs ↗pdf ↗

MixPath: A Unified Approach for One-shot Neural Architecture Searchcs.LG

MixPath unifies multi-path neural architecture search with one-shot training.

problem Efficiently searching multi-path neural architectures.

method One-shot multi-path supernet with Shadow Batch Normalization (SBN).

result SBN stabilizes optimization and improves ranking performance.

Neural Architecture Search Multi-path One-shot Shadow Batch Normalization Supernet

2020-01-16abs ↗pdf ↗

S3NAS: Fast NPU-aware Neural Architecture Search Methodologycs.LG

S3NAS finds high-accuracy CNN architectures for NPUs in 3 hours.

problem Finding optimal CNN architectures for NPUs with high accuracy and low latency.

method Supernet design, Single-Path NAS, scaling, analytical latency estimation.

result 82.72% top-1 accuracy on ImageNet with 11.66 ms latency.

Neural Architecture Search NPU Convolutional Neural Networks Latency Estimation Image Classification

2020-09-04abs ↗pdf ↗

BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channelscs.LG

BS-NAS broadens and shrinks search space for optimal neural architectures.

problem Suboptimal channel numbers and model averaging effects in One-Shot NAS methods.

method Broadening with spring block for channel search, shrinking with underperforming operations removal, evolutionary algorithm for optimal architecture search.

result BS-NAS achieves state-of-the-art performance on ImageNet.

Neural Architecture Search One-Shot NAS Channel Search Spring Block Shrinking Strategy

2020-03-22abs ↗pdf ↗

Balanced One-shot Neural Architecture Optimizationcs.LG

Improved neural architecture search through balanced training.

problem Inconsistent ranking of architectures under one-shot training.

method Balanced NAO: balanced training of supernet during search.

result Significant improvements in architecture discovery and performance.

Neural Architecture Search One-Shot NAS Balanced Training Supernet CIFAR-10

2019-09-24abs ↗pdf ↗

Neural Architecture Search on Acoustic Scene Classificationcs.SD

A lightweight network and NAS method improve ASC tasks.

problem Heavy computational burden in acoustic scene classification.

method Inspired by MobileNetV2, unidirectional convolutions; dynamic NAS with evolutionary algorithm.

result 90.3% F1-score on DCASE2018 task 5, 25% fewer FLOPs.

Acoustic Scene Classification Neural Architecture Search MobileNetV2 Dynamic Architecture Convolutional Neural Networks

2019-12-30abs ↗pdf ↗

Growing Efficient Deep Networks by Structured Continuous Sparsificationcs.LG

New method grows deep networks efficiently by dynamically pruning and growing layers.

problem Training deep networks is computationally expensive and inefficient.

method Structured continuous sparsification starting from a small seed architecture.

result 49.7% inference FLOPs and 47.4% training FLOPs savings with 75.2% top-1 accuracy.

deep learning network pruning continuous relaxation dynamic growth FLOPs reduction

2020-07-30abs ↗pdf ↗