Paper bounds NAS function approximation, showing computational limits.
problem NAS function approximation problem.
method Reformulated FA problem, showed computational infeasibility, described a-ASP.
result NAS cannot solve FA for all functions to zero error.
MiLeNAS improves neural architecture search by reducing approximation errors and achieving better accuracy.
problem Improving efficiency and accuracy in neural architecture search (NAS).
method Mixed-level reformulation (MiLeNAS) to optimize efficiently and reliably.
result MiLeNAS achieves lower validation error and higher accuracy than bilevel optimization methods.
This work proposes searching for optimal operation distribution in neural architecture search.
problem Finding optimal neural architecture with specific operations and connections.
method Search for the optimal operation distribution, providing a stochastic and approximate solution.
result Operation distribution holds enough discriminating power to reliably identify a solution and is easier to optimise than traditional encodings.
DARTS fails to generalize well; adding regularization improves robustness.
problem DARTS fails to find architectures that generalize well across different tasks.
method Identified failure modes, added regularization, proposed variations.
result Regularization robustifies DARTS to find better generalizing architectures.
This study uncovers how neural architectures and weights interact in classification tasks.
problem Understanding the role of neural architecture and weights in classification performance.
method Developed a novel method to find optimal task-specific architectures as binary networks with {0, 1}-valued weights, using approximate gradient descent.
result Well-trained architectures may not require fine-tuning of weights, highlighting the importance of structure over weights.
Neural Architecture Search aims at automatically finding neural architectures that are competitive with architectures designed by human experts. While recent approaches have achieved state-of-the-art predictive performance for image recognition, they are problematic under resource constraints for two reasons: (1)the ne…
UNAS combines DNAS and RL for efficient architecture search.
problem Discovering high accuracy or low latency neural architectures.
method Unified framework combining differentiable and reinforcement learning approaches.
result UNAS achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets.
GroSS enables efficient search for grouped convolutional architectures.
problem Training grouped convolutional architectures efficiently and effectively.
method GroSS: Group-Size Series Decomposition for Grouped Architecture Search.
result Simultaneous training of differing numbers of groups within a single layer and all possible combinations between layers.
One-shot neural architecture search limits depth search space and prunes networks for better performance and uncertainty.
problem Finding optimal depth in residual networks for efficient training and inference.
method Formulated a variational objective to approximate the depth distribution and pruned networks based on this distribution.
result Pruned networks achieve competitive accuracy with unpruned networks and better uncertainty calibration.
A Bayesian approach to neural architecture search improves efficiency and accuracy.
problem Improving the efficiency and accuracy of neural architecture search.
method Formulating NAS from a Bayesian perspective, explicitly estimating the joint posterior distribution over architectures and weights, using Variational Dropout, and posterior-guided sampling.
result Posterior-guided NAS (PGNAS) achieves a good trade-off between precision and speed of search.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
Paper tackles unfair advantages in DARTS, presenting Fair DARTS to improve neural architecture search.
problem Performance collapse in DARTS due to unfair advantages in skip connections.
method Relax exclusive competition to collaborative, let architectural weights be independent, and use zero-one loss for discretization.
result New state-of-the-art results on CIFAR-10 and ImageNet, demonstrating the effectiveness of Fair DARTS.
GeNAS uses genetic algorithms to automatically assess human sperm morphology.
problem Manual sperm morphology analysis is inexact, subjective, and hard to teach.
method Genetic Neural Architecture Search (GeNAS) for automatic SMA.
result GeNAS achieved state-of-the-art results in sperm morphology detection.
SGAS improves neural architecture search by choosing and pruning operations greedily.
problem NAS often fails to generalize in final evaluation.
method Divides search into sub-problems and chooses/prunes candidate operations greedily.
result SGAS finds state-of-the-art architectures with minimal computational cost.
Paper uses CMAB to improve NAS efficiency and accuracy.
problem Improving efficiency and accuracy of NAS for DNNs.
method Formulated NAS as CMAB, used Nested Monte-Carlo Search.
result Discovered cell structure achieves comparable accuracy to state-of-the-art, 20x faster.
Novel RL-based NPG improves multi-objective NAS efficiency and performance.
problem Discovering optimal neural architectures with multiple conflicting objectives.
method Non-stationary policy gradient with adaptive reward functions and shared model.
result Framework efficiently approximates full Pareto front and achieves superior performance.
Optimizes neural architecture search to generate novel lightweight models.
problem Over-reliance on expert knowledge limits NAS to local optima, preventing architectural breakthroughs.
method Casts NAS as an optimization problem, introduces a hierarchical graph-based search space, and uses Bayesian optimization.
result Generates extremely lightweight yet competitive models on six benchmark datasets.
DNAS disentangles neural architecture search for better interpretability and performance.
problem Lack of interpretability in existing neural architecture search methods.
method DNAS disentangles the hidden representation of the controller into semantically meaningful concepts.
result DNAS achieves state-of-the-art performance and competitive architectures.
Differentially-private FNAS protects privacy while collaboratively searching for neural architectures.
problem Collaborative neural architecture search with privacy concerns.
method Federated Neural Architecture Search (FNAS) with differential privacy (DP-FNAS).
result DP-FNAS can search for highly-performant neural architectures while protecting individual parties' privacy.
Improves neural architecture search methods to be more stable and efficient.
problem Neural architecture search methods are unstable and sensitive to hyperparameters.
method Discusses practical considerations to improve stability and efficiency.
result Improves overall performance of neural architecture search methods.
TextNAS finds optimal text representation networks using neural architecture search.
problem Finding the optimal text representation networks is challenging.
method Proposes a novel search space for text representation and uses automatic neural architecture search.
result Automatic search discovers network architectures that outperform state-of-the-art models on text classification and natural language inference tasks.
Efficient search methods can outperform random search on challenging tasks.
problem Comparing the performance of efficient and random search methods in neural architecture search.
method Comparison of weight sharing and random search methods on progressively larger search spaces for image classification and detection.
result Efficient search methods can provide substantial gains over random search on large, realistic tasks.
SAEP prunes sub-architectures to reduce search cost while maintaining performance.
problem Redundancy in ensemble sub-architectures leads to high computational cost.
method SAEP leverages diversity to prune sub-architectures, reducing ensemble size.
result SAEP reduces the number of sub-architectures without degrading performance.
NASP uses proximal gradient descent to speed up neural architecture search.
problem Efficiently search for high-performance neural architectures.
method Differentiable Neural Architecture Search using Proximal gradient descent.
result NASP achieves 10 times speedup over DARTS while maintaining high performance.
New method improves neural architecture search by optimizing for both performance and diversity.
problem Traditional multi-objective NAS fails to address practical constraints and niches.
method Formulated as quality diversity optimization, introduces multifidelity optimizers.
result Quality diversity NAS outperforms multi-objective NAS in quality and efficiency.
AGNN automates GNN architecture search, achieving best performance.
problem Finding optimal GNN architectures is laborious and requires human expertise.
method AGNN uses reinforcement learning to search for optimal GNN architectures within a predefined space, with a novel parameter sharing strategy.
result AGNN identifies optimal GNN architectures achieving best performance.
New hierarchical search algorithm improves neural architecture design across different operator sets.
problem DARTS's performance drops when search space changes due to operator correlation and optimization complexity.
method Operator clustering and optimization complexity matching in a hierarchical search algorithm.
result The algorithm consistently finds high-performance architectures across various search spaces, outperforming other methods.
Interstellar searches for recurrent architecture to enhance KG embedding.
problem Learning long-term information in KGs.
method Recurrent neural architecture search for relational paths.
result Effectiveness and efficiency of searched models.
NASES uses embedding space for efficient NAS in image classification tasks.
problem Difficulty in optimizing high-dimensional discrete architecture spaces.
method NASES employs architecture encoders and decoders to search in an embedding space using reinforcement learning.
result NASES discovers comparable final architectures to other NAS approaches in less time.
TabNAS improves neural architecture search for tabular datasets by rejecting suboptimal architectures.
problem Finding optimal neural architectures for tabular datasets with resource constraints.
method Develops a reinforcement learning controller motivated by rejection sampling to handle resource constraints.
result TabNAS finds better models that obey resource constraints compared to previous methods.
In neural architecture search (NAS), the space of neural network architectures is automatically explored to maximize predictive accuracy for a given task. Despite the success of recent approaches, most existing methods cannot be directly applied to large scale problems because of their prohibitive computational complex…
RARTS improves neural architecture search efficiency and accuracy.
problem Efficiency and accuracy issues in first-order DARTS.
method Formulates a relaxed architecture search (RARTS) method that uses whole dataset splitting.
result RARTS outperforms DARTS and its variants in accuracy and efficiency.
A new method approximates expected empirical loss for stochastic deep learning tasks.
problem Determining optimal step sizes for stochastic gradient descent in deep learning.
method Applying one-dimensional function fitting to noisy losses of vertical cross sections to approximate expected empirical loss.
result The method leads to a robust and straightforward optimization method that performs well across datasets and architectures.
A method for faster neural architecture search using low-fidelity training.
problem Time-consuming evaluations in neural architecture search.
method Bayesian multi-fidelity method with knowledge distillation.
result Training for a few epochs with knowledge distillation leads to better architecture selection.
Neural Architecture Search (NAS) has been quite successful in constructing state-of-the-art models on a variety of tasks. Unfortunately, the computational cost can make it difficult to scale. In this paper, we make the first attempt to study Meta Architecture Search which aims at learning a task-agnostic representation…
New algorithms improve neural architecture search with faster convergence.
problem Improving efficiency and accuracy of neural architecture search.
method Geometry-aware gradient algorithms to optimize continuous relaxation of discrete search spaces.
result Exceeds state-of-the-art results on CIFAR and ImageNet benchmarks.
DrNAS improves neural architecture search with Dirichlet distribution and progressive learning.
problem Efficiently search for neural architectures with improved generalization and exploration.
method Formulates architecture search as a distribution learning problem using Dirichlet distribution and gradient-based optimization. Introduces a progressive learning scheme to handle large-scale tasks.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet, demonstrating improved generalization and exploration.
Paper evaluates robustness of NAS against poisoning attacks.
problem Robustness of Neural Architecture Search (NAS) against poisoning attacks.
method Evaluation of Efficient NAS (ENAS) against carefully designed ineffective operations in poisoning attacks.
result Demonstrates how poisoning attacks exploit design flaws in ENAS controller.
DC-NAS improves neural architecture search by clustering and evaluating sub-networks.
problem Inaccurate evaluation of neural architectures in large search spaces.
method Divide-and-Conquer approach: feature representation, clustering, and evaluation of clusters.
result Achieved 75.1% top-1 accuracy on ImageNet, surpassing state-of-the-art methods.
RandomNet uses random search to design neural architectures without much human intervention.
problem Designing neural architectures without excessive human intervention.
method Random search strategy for multimodal neural architecture design.
result RandomNet performs close to state-of-the-art on AV-MNIST with minimal human supervision.
PDNAS optimizes GNN architectures for diverse datasets.
problem Inadequate adaptability and combinatorial search space in GNNs.
method Dual architecture search (micro- and macro-architectures) with gradient-based optimization.
result PDNAS finds deeper GNNs with better performance on diverse datasets.
MixPath unifies multi-path neural architecture search with one-shot training.
problem Efficiently searching multi-path neural architectures.
method One-shot multi-path supernet with Shadow Batch Normalization (SBN).
result SBN stabilizes optimization and improves ranking performance.
NAT optimizes neural architectures to improve performance without extra cost.
problem Redundant operations in neural architectures consume memory and degrade performance.
method Transformed Markov Decision Process (MDP) and reinforcement learning to replace redundant operations with more efficient ones.
result Transformed architectures outperform original and existing methods on CIFAR-10 and ImageNet datasets.
Paper proposes HCDC to improve hyperparameter search efficiency.
problem Poor generalizability of dataset condensation across different hyperparameters.
method HCDC algorithm that matches hyperparameter gradients for synthetic validation dataset.
result HCDC effectively maintains validation-performance rankings of models.
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline.…
HMCNAS generates competitive neural architectures without human-defined parameters.
problem Lack of human-defined parameters in Neural Architecture Search.
method Combines Hidden Markov Chains and Bayesian Optimization for autonomous search space generation and competitive model generation.
result HMCNAS generates competitive models in a short time without human-defined parameters.
Synthetic Petri Dish predicts neural architecture performance faster.
problem Expensive NAS evaluation process with ground-truth data.
method Instantiates motifs in small networks, evaluates with few synthetic samples.
result Significantly higher accuracy in predicting motif performance.
Improved software flaw detection using NAS on multimodal DL models.
problem Software flaw detection in multimodal deep learning models.
method Adapted NAS framework for multimodal learning, combined with multimodal deep learning models.
result Improved performance on the Juliet Test Suite.