NAS helps find best neural network designs.
problem Designing optimal neural network architectures.
method Optimization algorithms and search spaces.
result Introduction to major advances in NAS for CNNs.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Network Architecture Search”
Simplified NAS for GNN architectures improves efficiency and expressiveness.
problem Efficient and effective discovery of optimal GNN architectures.
method SNAG framework with a novel search space and reinforcement learning.
result SNAG framework outperforms human-designed and existing NAS methods.
One-shot neural architecture search limits depth search space and prunes networks for better performance and uncertainty.
problem Finding optimal depth in residual networks for efficient training and inference.
method Formulated a variational objective to approximate the depth distribution and pruned networks based on this distribution.
result Pruned networks achieve competitive accuracy with unpruned networks and better uncertainty calibration.
Neural architecture search (NAS) automatically finds the best task-specific neural network topology, outperforming many manual architecture designs. However, it can be prohibitively expensive as the search requires training thousands of different networks, while each can last for hours. In this work, we propose the Gra…
Contrastive embeddings improve neural architecture search performance.
problem Improving performance of neural architecture search algorithms.
method Contrastive learning to identify networks based on data Jacobians and produce embeddings.
result Traditional black-box optimization algorithms can reach state-of-the-art performance with contrastive embeddings.
RARTS improves neural architecture search efficiency and accuracy.
problem Efficiency and accuracy issues in first-order DARTS.
method Formulates a relaxed architecture search (RARTS) method that uses whole dataset splitting.
result RARTS outperforms DARTS and its variants in accuracy and efficiency.
For network architecture search (NAS), it is crucial but challenging to simultaneously guarantee both effectiveness and efficiency. Towards achieving this goal, we develop a differentiable NAS solution, where the search space includes arbitrary feed-forward network consisting of the predefined number of connections. Be…
SGAS improves neural architecture search by choosing and pruning operations greedily.
problem NAS often fails to generalize in final evaluation.
method Divides search into sub-problems and chooses/prunes candidate operations greedily.
result SGAS finds state-of-the-art architectures with minimal computational cost.
PDNAS optimizes GNN architectures for diverse datasets.
problem Inadequate adaptability and combinatorial search space in GNNs.
method Dual architecture search (micro- and macro-architectures) with gradient-based optimization.
result PDNAS finds deeper GNNs with better performance on diverse datasets.
A ranking network improves neural architecture search efficiency.
problem Efficiently search for neural network architectures without extensive training.
method Pairwise ranking loss for a performance predictor trained on task meta-features.
result The ranking network outperforms the performance predictor in architecture search.
DC-NAS improves neural architecture search by clustering and evaluating sub-networks.
problem Inaccurate evaluation of neural architectures in large search spaces.
method Divide-and-Conquer approach: feature representation, clustering, and evaluation of clusters.
result Achieved 75.1% top-1 accuracy on ImageNet, surpassing state-of-the-art methods.
FiGS searches over a larger space of architectures for efficient mobile models.
problem Designing small, efficient deep networks for mobile devices.
method Differentiable search method using sparse regularization and Logistic-Sigmoid distribution.
result FiGS produces state-of-the-art parameter-efficient models on ImageNet and improves object detection performance.
Study reduces NAS search cost by generating multiple complex architectures in one shot.
problem Finding multiple neural architectures with varying complexities efficiently.
method Uses importance sampling to generate and update multiple distributions of architectures.
result Reduces search cost by finding multiple architectures with different complexities in a single search.
TextNAS finds optimal text representation networks using neural architecture search.
problem Finding the optimal text representation networks is challenging.
method Proposes a novel search space for text representation and uses automatic neural architecture search.
result Automatic search discovers network architectures that outperform state-of-the-art models on text classification and natural language inference tasks.
einspace expands NAS search space to include diverse neural architectures.
problem NAS results are often limited to existing structures; new designs are rare.
method einspace uses a probabilistic context-free grammar to create a versatile search space.
result einspace discovers novel and improved architectures, including convolutions and attention.
The use of automatic methods, often referred to as Neural Architecture Search (NAS), in designing neural network architectures has recently drawn considerable attention. In this work, we present an efficient NAS approach, named HM- NAS, that generalizes existing weight sharing based NAS approaches. Existing weight shar…
HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.
problem Finding fitting neural networks that adhere to hard resource constraints.
method Accurate formulation of resource requirement and scalable search method.
result HardCoRe-NAS generates state-of-the-art architectures strictly satisfying hard resource constraints.
DONNA rapidly finds optimal neural networks across diverse spaces.
problem Efficient scaling and handling of diverse architectural search-spaces in NAS.
method Three-phase pipeline: accuracy predictor, rapid evolutionary search, and optimal model finetuning.
result 100x faster than MNasNet in finding state-of-the-art architectures on-device.
GroSS enables efficient search for grouped convolutional architectures.
problem Training grouped convolutional architectures efficiently and effectively.
method GroSS: Group-Size Series Decomposition for Grouped Architecture Search.
result Simultaneous training of differing numbers of groups within a single layer and all possible combinations between layers.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
BINAS improves neural architecture search with interpretable models.
problem Finding efficient neural networks under resource constraints.
method BINAS uses a bilinear formulation for accuracy and resource estimation, with a scalable search method.
result BINAS generates comparable or better architectures than state-of-the-art methods, while strictly satisfying resource constraints.
VINNAS uses variational inference to avoid mode collapse in neural architecture search.
problem Mode collapse in gradient-based NAS methods, leading to suboptimal architectures.
method Differentiable variational inference with variational dropout and automatic relevance determination.
result State-of-the-art accuracy with up to twice fewer non-zero parameters.
SSNAS finds neural architectures without labeled data.
problem Limited labeled data for NAS.
method Self-supervised learning for NAS.
result Comparable results to supervised NAS with labeled data.
Neural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a neural network while changing its neural architecture, could be helpfu…
Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials are usually required to identify a suitable GNN architecture. It is because the performance of a GNN architecture is significantly affected b…
NAS-Navigator automates neural network architecture search with visual steering.
problem Difficulty in configuring large neural networks efficiently.
method Formulates architecture optimization as graph space exploration, trains all candidate architectures in one-shot.
result Allows analysts to effectively select and guide the search for optimal neural network architectures.
Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernelscs.LG
Bayesian optimisation with graph kernels improves neural architecture search and provides interpretability.
problem Lack of insight into why architectures perform well and how to improve them.
method Combines Bayesian optimisation with Weisfeiler-Lehman graph kernels for highly data-efficient and interpretable architecture search.
result Demonstrates state-of-the-art performance on closed- and open-domain search spaces.
Optimizes neural architecture search to generate novel lightweight models.
problem Over-reliance on expert knowledge limits NAS to local optima, preventing architectural breakthroughs.
method Casts NAS as an optimization problem, introduces a hierarchical graph-based search space, and uses Bayesian optimization.
result Generates extremely lightweight yet competitive models on six benchmark datasets.
Extends NAS to learn both intra-cell and inter-cell architectures for language modeling.
problem Limited NAS systems restrict search to recurrent or convolutional cells.
method Designs a joint learning method to perform intra-cell and inter-cell NAS simultaneously.
result Significantly outperforms a strong baseline on PTB and WikiText data.
Novel framework learns efficient student models from teacher networks.
problem Model capacity gap between teacher and student networks.
method Neural architecture search and oracle knowledge distillation.
result Searched student models often outperform teacher models.
A method for faster neural architecture search using low-fidelity training.
problem Time-consuming evaluations in neural architecture search.
method Bayesian multi-fidelity method with knowledge distillation.
result Training for a few epochs with knowledge distillation leads to better architecture selection.
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being d…
Smooth embedding space improves NAS performance.
problem Efficiently predicting good neural architectures.
method Two-sided variational graph autoencoder.
result Smooth embedding space facilitates extrapolation to unseen architectures.
Differentially-private FNAS protects privacy while collaboratively searching for neural architectures.
problem Collaborative neural architecture search with privacy concerns.
method Federated Neural Architecture Search (FNAS) with differential privacy (DP-FNAS).
result DP-FNAS can search for highly-performant neural architectures while protecting individual parties' privacy.
DNAS disentangles neural architecture search for better interpretability and performance.
problem Lack of interpretability in existing neural architecture search methods.
method DNAS disentangles the hidden representation of the controller into semantically meaningful concepts.
result DNAS achieves state-of-the-art performance and competitive architectures.
This paper compares Grid Search, Random Search, and Genetic Algorithm for NAS.
problem Hyperparameter optimization for neural architecture search.
method Comparison of Grid Search, Random Search, and Genetic Algorithm.
result Genetic Algorithm outperforms Grid Search and Random Search in terms of accuracy and execution time.
A lightweight network and NAS method improve ASC tasks.
problem Heavy computational burden in acoustic scene classification.
method Inspired by MobileNetV2, unidirectional convolutions; dynamic NAS with evolutionary algorithm.
result 90.3% F1-score on DCASE2018 task 5, 25% fewer FLOPs.
Proposes a new method to optimize graph neural network architectures on heterogeneous information networks.
problem Weaknesses in instability and inflexibility of existing graph neural architecture search methods.
method Partial Message Meta Multigraph search (PMMM) using a differentiable framework to search for a meaningful meta multigraph.
result Significantly more stable and effective than state-of-the-art heterogeneous GNNs.
Graph Neural Networks (GNNs) have been popularly used for analyzing non-Euclidean data such as social network data and biological data. Despite their success, the design of graph neural networks requires a lot of manual work and domain knowledge. In this paper, we propose a Graph Neural Architecture Search method (Grap…
Automates neural network design without training, speeding up search by seconds.
problem Time and effort in hand-designing deep neural networks.
method Predict trained accuracy from untrained network state using activation overlap.
result Search for powerful networks in seconds, verified on various benchmarks.
AutoShrink optimizes neural architectures by shrinking cell structures.
problem Resource constraints in deploying DNNs on mobile devices.
method Topology-aware node-based Neural Architecture Search (NAS).
result AutoShrink achieves up to 48% parameter reduction and 34% MACs savings.
Optimal transport kernels improve neural architecture search efficiency.
problem Comparing complex neural architectures similarity using Euclidean metric fails.
method Developed a novel discrepancy using tree-Wasserstein (TW) for neural architectures.
result TW-based approaches outperform other methods in sequential and parallel NAS.
This paper uses graph convolutional networks to improve the accuracy of neural architecture search.
problem Improving the precision of sampled sub-networks in weight-sharing NAS.
method Training a graph convolutional network to fit the performance of sampled sub-networks.
result Achieved higher rank correlation coefficient and better final architecture performance.
The design of neural network architectures for a new data set is a laborious task which requires human deep learning expertise. In order to make deep learning available for a broader audience, automated methods for finding a neural network architecture are vital. Recently proposed methods can already achieve human expe…
Bayesian optimization reduces RNN architecture search time.
problem Optimizing RNN architectures for time series prediction.
method Bayesian Optimization with a training-free performance metric.
result BO designs well-performing RNN architectures with reduced optimization time.
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
In neural architecture search, the structure of the neural network to best model a given dataset is determined by an automated search process. Efficient Neural Architecture Search (ENAS), proposed by Pham et al. (2018), has recently received considerable attention due to its ability to find excellent architectures with…