GraphNAS uses reinforcement learning to automatically design graph neural network architectures.
problem Designing effective graph neural network architectures requires manual work and domain knowledge.
method GraphNAS generates variable-length strings to describe architectures and trains a recurrent network with reinforcement learning to maximize validation accuracy.
result GraphNAS achieves consistently better performance on various citation and protein networks.
Survey of 1000 NAS papers, automating neural architecture design.
problem Designing high-performing neural architectures for various tasks.
method Taxonomy of search spaces, algorithms, and speedup techniques.
result NAS has surpassed human-designed architectures on many tasks.
Paper proposes NASAIC framework for co-designing neural architectures and heterogeneous ASICs.
problem Designing efficient neural architectures and ASICs for multiple tasks.
method Build ASIC templates and propose NASAIC framework for simultaneous design of architectures and ASICs.
result NASAIC ensures design specifications and maximizes accuracy with minimal performance loss.
HM-NAS improves neural architecture search by learning optimal architectures.
problem Limited flexibility in architecture candidates due to hand-designed heuristics.
method Incorporates multi-level encoding and hierarchical masking to automatically learn optimal architectures.
result Achieves better architecture search performance and competitive model accuracy.
New neural network architecture for auction design exploiting permutation symmetry.
problem Designing incentive-compatible auctions that maximize expected revenue.
method Constructed a permutation-equivariant neural network architecture.
result Permutation-equivariant architectures can perfectly recover optimal mechanisms.
A new NAS framework optimizes 3D medical image segmentation architectures.
problem Optimizing neural architectures for high-resolution 3D medical images.
method Stochastic sampling algorithm for scalable gradient-based optimization of neural connectivities and operation types in both encoder and decoder.
result Automatically designed architecture outperforms human-designed U-Net.
Survey tackles challenges in neural architecture design.
problem Challenges in designing optimal neural architectures.
method New classification perspective based on early NAS algorithms' characteristics, problems, and solutions.
result Comprehensive analysis and comparison of NAS works.
New ShuffleNASNets improve efficiency and speed of CNN models.
problem Complexity and inefficiency of search-based neural architecture designs.
method Adopted and enhanced Efficient Neural Architecture Search (ENAS) with ShuffleNet V2 principles.
result Achieved significantly less complex, faster, and more efficient CNN models.
AGAN automates GAN design, outperforming human-designed models.
problem Designing effective GAN architectures requires human expertise and trial-and-error.
method Automated neural architecture search (AGAN) for deep generative models.
result AGAN finds architectures that outperform state-of-the-art models in unsupervised and supervised image generation tasks.
EENA efficiently searches neural architectures with minimal resources.
problem Lack of direction and high computational cost in neural architecture search.
method EENA uses guided evolution with mutation and crossover operations.
result EENA designs highly effective neural architectures with minimal resources.
NAS helps find best neural network designs.
problem Designing optimal neural network architectures.
method Optimization algorithms and search spaces.
result Introduction to major advances in NAS for CNNs.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
Optimizes crypto-oriented neural architectures for faster secure inference.
problem Privacy conflicts between model users and providers in neural network applications.
method Proposes a novel Partial Activation layer to optimize the initial design of crypto-oriented neural architectures.
result Significant improvement in the efficiency of secure inference on common evaluation metrics.
Simplified NAS for GNN architectures improves efficiency and expressiveness.
problem Efficient and effective discovery of optimal GNN architectures.
method SNAG framework with a novel search space and reinforcement learning.
result SNAG framework outperforms human-designed and existing NAS methods.
This thesis aims to automate deep neural network design for efficiency and complexity reduction.
problem Manual design of deep neural networks is inefficient and complex.
method Examines and proposes automated approaches to neural network design.
result Creation of less complex models with good performance through automation.
GP-CNAS uses genetic programming to automatically design CNN architectures.
problem Designing optimal CNN architectures is laborious and error-prone.
method GP-CNAS uses a tree-based representation of CNNs and dynamic crossover operators to search for optimal architectures.
result GP-CNAS finds optimal CNN architectures with balanced depth and width in limited trials.
Automates GNN design for molecular property prediction.
problem Designing and tuning GNN architectures for molecular property prediction is labor-intensive.
method Developed a NAS approach to automatically discover high-performing GNN architectures for MPNNs.
result Automatically discovered MPNNs outperform manually designed GNNs in molecular property prediction.
RandomNet uses random search to design neural architectures without much human intervention.
problem Designing neural architectures without excessive human intervention.
method Random search strategy for multimodal neural architecture design.
result RandomNet performs close to state-of-the-art on AV-MNIST with minimal human supervision.
SAEP prunes sub-architectures to reduce search cost while maintaining performance.
problem Redundancy in ensemble sub-architectures leads to high computational cost.
method SAEP leverages diversity to prune sub-architectures, reducing ensemble size.
result SAEP reduces the number of sub-architectures without degrading performance.
A ranking network improves neural architecture search efficiency.
problem Efficiently search for neural network architectures without extensive training.
method Pairwise ranking loss for a performance predictor trained on task meta-features.
result The ranking network outperforms the performance predictor in architecture search.
We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. In ENAS, a controller learns to discover neural network architectures by searching for an optimal subgraph within a large computational graph. The controller is trained with policy gradient to select a su…
Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep n…
Graph HyperNetworks (GHN) speed up neural architecture search.
problem Expensive neural architecture search (NAS) requiring training thousands of networks.
method GHN models architecture topology and generates weights via graph neural network.
result GHNs can search nearly 10 times faster than other methods on CIFAR-10 and ImageNet.
NAT optimizes neural architectures to improve performance without extra cost.
problem Redundant operations in neural architectures consume memory and degrade performance.
method Transformed Markov Decision Process (MDP) and reinforcement learning to replace redundant operations with more efficient ones.
result Transformed architectures outperform original and existing methods on CIFAR-10 and ImageNet datasets.
Unified framework for U-Net design and analysis.
problem Understudied design and architecture of U-Nets.
method Theoretical results, Multi-ResNets, function constraints encoding.
result Competitive and superior performance in various tasks.
Graph-based NAS improves sample efficiency in architecture design.
problem Current NAS search spaces are static sequences, limiting expressiveness.
method Proposed graph-based search space with vertices and edges for iterative and branching decisions.
result Graph representation improves sample efficiency in architecture design.
Paper evaluates robustness of NAS against poisoning attacks.
problem Robustness of Neural Architecture Search (NAS) against poisoning attacks.
method Evaluation of Efficient NAS (ENAS) against carefully designed ineffective operations in poisoning attacks.
result Demonstrates how poisoning attacks exploit design flaws in ENAS controller.
We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural A…
A novel neural network approach for optimization problems.
problem Constrained optimization problems.
method Neural Optimization Machine (NOM) using a specially designed NN architecture and training procedure.
result Solves optimization problems efficiently, especially in high-dimensional spaces.
This study optimizes quantized neural networks by considering model architecture and quantization types.
problem Optimizing quantized neural networks for low-power, high-throughput applications.
method Holistic approach including training methods and quantization-friendly architecture design.
result Deeper models are more sensitive to activation quantization, while wider models improve resilience to both weight and activation quantization.
The paper introduces capacity allocation analysis for neural networks, focusing on spatial capacity.
problem Designing neural network architectures is challenging due to the interplay of intuition, experimentation, and luck.
method Introduces capacity allocation analysis, focusing on spatial capacity allocation in linear settings.
result Quantitative comparison of classical architectures on various synthetic tasks reveals insights into model capacity allocation.
einspace expands NAS search space to include diverse neural architectures.
problem NAS results are often limited to existing structures; new designs are rare.
method einspace uses a probabilistic context-free grammar to create a versatile search space.
result einspace discovers novel and improved architectures, including convolutions and attention.
MemNet optimizes neural architectures for memory efficiency.
problem Memory constraints in mobile devices limit the use of large neural networks.
method Augment-trim learning with memory consumption ranking score.
result MemNet finds architectures with 24.17% less memory usage compared to state-of-the-art methods.
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
Neural Architecture Search aims at automatically finding neural architectures that are competitive with architectures designed by human experts. While recent approaches have achieved state-of-the-art predictive performance for image recognition, they are problematic under resource constraints for two reasons: (1)the ne…
Deep learning models require extensive architecture design exploration and hyperparameter optimization to perform well on a given task. The exploration of the model design space is often made by a human expert, and optimized using a combination of grid search and search heuristics over a large space of possible choices…
AGNN automates GNN architecture search, achieving best performance.
problem Finding optimal GNN architectures is laborious and requires human expertise.
method AGNN uses reinforcement learning to search for optimal GNN architectures within a predefined space, with a novel parameter sharing strategy.
result AGNN identifies optimal GNN architectures achieving best performance.
Optimized neural networks for Edge TPU achieve high accuracy in real-time image classification.
problem Designing neural networks for hardware accelerators to achieve optimal performance.
method Hardware-aware neural architecture search and model customization for Edge TPU.
result Improved accuracy-latency tradeoff on Pixel 4's Edge TPU compared to existing models.
Improved neural architecture optimization for energy efficiency.
problem Designing energy-efficient deep learning networks for mobile and edge devices.
method Incorporates energy cost in splitting process and uses a scalable stochastic gradient algorithm to speed up the splitting.
result Trains highly accurate and energy-efficient networks on challenging datasets like ImageNet.
Flat learning curves reveal no progress in ENAS controller.
problem Improving learning speed in neural architecture search.
method Evaluated learning progress of ENAS controller through architecture re-training.
result No observable progress in controller's generated architectures.
EGO optimizes neural network architectures without manual tuning.
problem Designing optimal neural network architectures is difficult and time-consuming.
method Adapted EGO algorithm for efficient optimization of neural network architectures.
result Automatically optimized neural networks achieve competitive performance compared to hand-crafted ones.
Automates neural network design without training, speeding up search by seconds.
problem Time and effort in hand-designing deep neural networks.
method Predict trained accuracy from untrained network state using activation overlap.
result Search for powerful networks in seconds, verified on various benchmarks.
Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being d…
Comparative study of neural networks for short-term FOREX forecasting.
problem Simulating expert judgment in foreign exchange market forecasting.
method Implemented and compared LSTM and ANN architectures for short-term FOREX forecasting.
result ANN custom architecture outperforms LSTM in prediction quality and resource efficiency.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
Serenity optimizes neural network execution for edge devices by scheduling with optimal memory footprint.
problem Order of nodes in irregular neural networks affects memory footprint, complicating execution under resource constraints.
method Memory-aware compiler using dynamic programming and graph rewriting to find optimal schedules.
result Achieves optimal peak memory and further improves it with graph rewriting, reducing memory usage by 1.68x-1.86x compared to TensorFlow Lite.
DANCE optimizes neural network and accelerator design for faster, more efficient DNN execution.
problem Challenges in optimizing neural network and accelerator design for efficient DNN execution.
method Differentiable approach to co-exploration of accelerator and network architecture design.
result Significantly shorter time to achieve superior accuracy and hardware cost metrics.
A new design methodology for neural networks that is guided by traditional algorithm design is presented. To prove our point, we present two heuristics and demonstrate an algorithmic technique for incorporating additional weights in their signal-flow graphs. We show that with training the performance of these networks …