Survey of automated neural architecture search methods.
problem Manual development of neural architectures is time-consuming and error-prone.
method Categorizes existing automated neural architecture search methods.
result Growing interest in automated methods due to manual development's limitations.
Survey of neural architecture search methods.
problem Automating the selection of neural network architectures.
method Comprehensive analysis of existing methods using reinforcement learning, evolutionary algorithms, and surrogate models.
result Unified formalism for categorizing and comparing architecture search methods.
Automated meta-learning improves model performance on small datasets.
problem Improving machine learning model performance on limited data.
method Gradient-based meta-learning combined with automated neural architecture search.
result Automatically found meta-learner achieved 74.65% accuracy on 5-shot 5-way Mini-ImageNet, 11.54% better than MAML.
Survey of 1000 NAS papers, automating neural architecture design.
problem Designing high-performing neural architectures for various tasks.
method Taxonomy of search spaces, algorithms, and speedup techniques.
result NAS has surpassed human-designed architectures on many tasks.
AGAN automates GAN design, outperforming human-designed models.
problem Designing effective GAN architectures requires human expertise and trial-and-error.
method Automated neural architecture search (AGAN) for deep generative models.
result AGAN finds architectures that outperform state-of-the-art models in unsupervised and supervised image generation tasks.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
Flexible DSL generates novel RNN architectures for various tasks.
problem Limited flexibility and components in existing RNN architectures.
method Domain-specific language (DSL) for automated architecture search.
result Novel RNN architectures perform well on language modeling and machine translation tasks.
RandomNet uses random search to design neural architectures without much human intervention.
problem Designing neural architectures without excessive human intervention.
method Random search strategy for multimodal neural architecture design.
result RandomNet performs close to state-of-the-art on AV-MNIST with minimal human supervision.
AutoOD automates outlier detection using curiosity-guided search and self-imitation learning.
problem Automated outlier detection for complex tasks with big data.
method Curiosity-guided search strategy and self-imitation learning.
result AutoOD identifies optimal neural network models with superior performance.
A graph-based evolutionary algorithm automates machine learning workflows.
problem Automated machine learning to reduce manual operations.
method Graph-based architecture for flexible model combinations, evolutionary algorithm with mutation and heredity operators, Bayesian hyper-parameter optimization.
result State-of-the-art performance compared to other AutoML systems.
AGNN automates GNN architecture search, achieving best performance.
problem Finding optimal GNN architectures is laborious and requires human expertise.
method AGNN uses reinforcement learning to search for optimal GNN architectures within a predefined space, with a novel parameter sharing strategy.
result AGNN identifies optimal GNN architectures achieving best performance.
Automates neural network design without training, speeding up search by seconds.
problem Time and effort in hand-designing deep neural networks.
method Predict trained accuracy from untrained network state using activation overlap.
result Search for powerful networks in seconds, verified on various benchmarks.
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Survey of techniques for automating hyperparameter optimization in machine learning.
problem Finding optimal hyperparameters for machine learning algorithms.
method Review of various hyperparameter optimization techniques.
result Unified treatment of hyperparameter optimization techniques.
MNMS learns from previous tasks to speed up model search for new tasks.
problem NAS requires extensive manual design and optimization for each new task.
method MNMS uses reinforcement learning to condition model construction on previously successful searches.
result MNMS can conduct simultaneous searches for multiple tasks and transfer knowledge to new tasks.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
Efficiently searches for optimal neural architecture and hyperparameters.
problem Separate tuning of architecture and hyperparameters leads to suboptimal results.
method Combines Bayesian optimization and Hyperband for joint search.
result Joint search yields better results with fewer epochs.
Evo-NAS combines neural and evolutionary methods for efficient neural architecture search.
problem Efficiently searching for optimal neural architectures in deep learning.
method Evolutionary-Neural hybrid agents that combine the strengths of neural and evolutionary algorithms.
result Evo-NAS outperforms both neural and evolutionary agents in architecture search for various classification tasks.
FiGS searches over a larger space of architectures for efficient mobile models.
problem Designing small, efficient deep networks for mobile devices.
method Differentiable search method using sparse regularization and Logistic-Sigmoid distribution.
result FiGS produces state-of-the-art parameter-efficient models on ImageNet and improves object detection performance.
A method to reduce memory usage in NAS by pruning the search space.
problem High GPU memory consumption in One-Shot NAS techniques.
method Utilising Zero-Shot NAS to prune the search space before applying One-Shot NAS.
result Reduces memory consumption by 81% while maintaining accuracy.
Weight-sharing is rarely helpful in NAS, especially on small datasets.
problem The efficiency of weight-sharing in Neural Architecture Search (NAS) is questionable.
method Comparison of a state-of-the-art weight-sharing approach to random search on the nasbench dataset.
result Weight-sharing is only rarely significantly helpful in NAS, highlighting the importance of the search space.
Automates deep learning model development for cancer data.
problem Manual design of high-performing deep learning models for cancer data is time-consuming and requires expertise.
method Reinforcement-learning-based neural architecture search with custom building blocks.
result Automated discovery of deep neural network architectures with similar or higher accuracy.
A ranking network improves neural architecture search efficiency.
problem Efficiently search for neural network architectures without extensive training.
method Pairwise ranking loss for a performance predictor trained on task meta-features.
result The ranking network outperforms the performance predictor in architecture search.
NAS-Navigator automates neural network architecture search with visual steering.
problem Difficulty in configuring large neural networks efficiently.
method Formulates architecture optimization as graph space exploration, trains all candidate architectures in one-shot.
result Allows analysts to effectively select and guide the search for optimal neural network architectures.
Flat learning curves reveal no progress in ENAS controller.
problem Improving learning speed in neural architecture search.
method Evaluated learning progress of ENAS controller through architecture re-training.
result No observable progress in controller's generated architectures.
Paper evaluates robustness of NAS against poisoning attacks.
problem Robustness of Neural Architecture Search (NAS) against poisoning attacks.
method Evaluation of Efficient NAS (ENAS) against carefully designed ineffective operations in poisoning attacks.
result Demonstrates how poisoning attacks exploit design flaws in ENAS controller.
Automates GNN design for molecular property prediction.
problem Designing and tuning GNN architectures for molecular property prediction is labor-intensive.
method Developed a NAS approach to automatically discover high-performing GNN architectures for MPNNs.
result Automatically discovered MPNNs outperform manually designed GNNs in molecular property prediction.
Study reduces NAS search cost by generating multiple complex architectures in one shot.
problem Finding multiple neural architectures with varying complexities efficiently.
method Uses importance sampling to generate and update multiple distributions of architectures.
result Reduces search cost by finding multiple architectures with different complexities in a single search.
Develops a robust, fast, and widely-applicable neural architecture search method.
problem Inability of current NAS methods to be easily applied to new problems.
method Adaptive stochastic natural gradient method for simultaneous optimization of weights and architecture.
result Near state-of-the-art performances with low computational budgets.
BioNAS optimizes deep learning models for biomedical research, revealing new knowledge.
problem Building interpretable deep learning models for biomedical research.
method Neural architecture search with knowledge dissimilarity functions for joint optimization of predictive power and biological knowledge.
result BioNAS optimal models reveal novel knowledge in both simulated and real functional genomics data.
Transfer Neural AutoML speeds up deep learning architecture design.
problem High computational cost in Neural AutoML.
method Transfer learning to speed up architecture design.
result Reduces convergence time by over an order of magnitude.
Optimal transport kernels improve neural architecture search efficiency.
problem Comparing complex neural architectures similarity using Euclidean metric fails.
method Developed a novel discrepancy using tree-Wasserstein (TW) for neural architectures.
result TW-based approaches outperform other methods in sequential and parallel NAS.
Automated ML simplifies machine learning configurations.
problem Difficulty in configuring and selecting ML methods.
method Bi-level learning objective, learning strategy, and evaluation strategy.
result Satisfactory ML configurations generated automatically.
Two methods accelerate neural architecture optimization by transferring knowledge.
problem Expensive neural architecture search.
method Transfer knowledge from previous tasks to new tasks.
result Acceleration of neural architecture optimization without significant loss in accuracy.
New method finds competitive neural networks in a day.
problem Designing neural network architectures is laborious and requires human expertise.
method Combining UCT with two UCT derivations for efficient network architecture search.
result Competitive networks found for MNIST, SVHN, and CIFAR-10 in a single GPU day.
Efficiently infers neural architectures for unseen tasks.
problem High cost of neural architecture search.
method Gradient-based framework sharing information across tasks.
result Quick identification of good candidate architectures for new tasks.
AgEBO-Tabular combines NAS and hyperparameter tuning for fast, high-performing tabular models.
problem Developing high-performing predictive models for large tabular data sets is challenging.
method Combines aging evolution NAS and asynchronous Bayesian optimization for hyperparameter tuning in data-parallel training.
result Automatically discovered neural network models outperform state-of-the-art AutoML ensembles in inference speed by two orders of magnitude.
NAS best practices guide reduces evaluation issues.
problem Lack of scientific evaluation quality in NAS.
method Described NAS best practices and checklist.
result Reduces evaluation issues in NAS.
Parallelizes MCTS for continuous domains using leaf and root parallelization.
problem Solving challenging tasks in continuous domains using MCTS.
method Extends existing parallelization strategies to continuous domains, focusing on leaf and root parallelization.
result Proposes two final selection strategies for continuous states in root parallelization.
Deep-n-Cheap automates deep learning model search for low complexity.
problem Finding efficient deep learning models for various datasets.
method Automated search framework for architecture and hyperparameters, including search transfer.
result Models offer comparable performance to state-of-the-art but are faster to train.
NASirt automates CNN architecture design for spectral data.
problem Designing optimal neural architectures for complex data.
method Neural Architecture Search (NAS) with Item Response Theory (IRT) for instance-level complexity.
result NASirt achieves high accuracy (97.40%) on spectral datasets.
AlphaX uses MCTS and Meta-DNN to improve NAS efficiency and accuracy.
problem Improving sample efficiency and network evaluation cost in NAS.
method Adaptive MCTS with Meta-DNN for prediction and distributed rollouts for cost reduction.
result AlphaX finds architectures with high accuracy (97.84% on CIFAR-10, 75.5% on ImageNet) in fewer samples.
S3NAS finds high-accuracy CNN architectures for NPUs in 3 hours.
problem Finding optimal CNN architectures for NPUs with high accuracy and low latency.
method Supernet design, Single-Path NAS, scaling, analytical latency estimation.
result 82.72% top-1 accuracy on ImageNet with 11.66 ms latency.
AMEIR automates recommender system design using NAS.
problem Manual feature and architecture engineering in recommender systems.
method Three-stage search space and tailored three-step pipeline.
result AMEIR outperforms manual and complex NAS methods.
Automates fairness and accuracy optimization in deep learning models for tabular data.
problem Improving fairness and accuracy in neural models for tabular data.
method Employed multi-objective Neural Architecture Search (NAS) and Hyperparameter Optimization (HPO) to find new models.
result Jointly optimized architectures that consistently outperform single-objective fairness mitigation methods.
Automated framework forecasts correlated time series in minutes.
problem Forecasting correlated time series with high accuracy and efficiency.
method Data-driven iterative pruning, zero-shot search, fast parameter adaptation.
result Framework achieves state-of-the-art accuracy and is faster than existing methods.
A new method for finding efficient neural interaction functions in collaborative filtering.
problem Finding consistent good performance for complex interactions in collaborative filtering.
method Proposes a search algorithm for simple neural interaction functions (SIF) in CF, using a structured multi-layer perceptron.
result Demonstrates much better prediction performance and distinct IFCs for different data sets and tasks.
BO-Aug automates data augmentation for image classification tasks.
problem Designing optimal data augmentation policies is time-consuming and costly.
method Bayesian optimization for finding optimal data augmentation policies.
result BO-Aug achieves state-of-the-art classification accuracy.