A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
NAS-Bench-Suite simplifies NAS evaluation across diverse tasks.
Enhanced tabular benchmarks for energy-efficient neural architecture search.
New benchmarks provide full training data for NAS research.
This study analyzes NAS benchmarks and finds that only a subset of operations is crucial for generating high-performing architectures.
One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice. Nevertheless, there is still a lack of understanding on how these weight-sharing algorithms exactly work due to the many factors controlling the dynamics of the process. In order to allow a sc…
This work benchmarks and theorizes robust NAS under adversarial training.
Neural architecture search (NAS) is a promising research direction that has the potential to replace expert-designed networks with learned, task-specific architectures. In this work, in order to help ground the empirical results in this field, we propose new NAS baselines that build off the following observations: (i) …
NATS-Bench benchmarks NAS algorithms for architecture topology and size.
Improved software flaw detection using NAS on multimodal DL models.
A benchmark for NLP models trained on text datasets.
Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation. We aim to ameliorate these problems by introducing NAS-Bench-101, the first public arc…
Neural Architecture Search (NAS) is an exciting new field which promises to be as much as a game-changer as Convolutional Neural Networks were in 2012. Despite many great works leading to substantial improvements on a variety of tasks, comparison between different methods is still very much an open issue. While most al…
MAML with over-parameterized DNNs converges globally at a linear rate.
Hill-climbing is a powerful baseline for NAS, even with reduced noise.
New algorithms improve neural architecture search with faster convergence.
The recent progress in neural architecture search (NAS) has allowed scaling the automated design of neural architectures to real-world domains, such as object detection and semantic segmentation. However, one prerequisite for the application of NAS are large amounts of labeled data and compute resources. This renders i…
Due to the recent advances on Neural Architecture Search (NAS), it gains popularity in designing best networks for specific tasks. Although it shows promising results on many benchmarks and competitions, NAS still suffers from its demanding computation cost for searching high dimensional architectural design space, and…
Proposes a method to improve neural architectures reproducibly.
In this paper, a neural architecture search (NAS) framework is proposed for 3D medical image segmentation, to automatically optimize a neural architecture from a large design space. Our NAS framework searches the structure of each layer including neural connectivities and operation types in both of the encoder and deco…
Neural Architecture Search (NAS) is an emerging topic in machine learning and computer vision. The fundamental ideology of NAS is using an automatic mechanism to replace manual designs for exploring powerful network architectures. One of the key factors of NAS is to scale-up the search space, e.g., increasing the numbe…
Survey of 1000 NAS papers, automating neural architecture design.
Neural Architecture Search has shown potential to automate the design of neural networks. Deep Reinforcement Learning based agents can learn complex architectural patterns, as well as explore a vast and compositional search space. On the other hand, evolutionary algorithms offer higher sample efficiency, which is criti…
Smooth embedding space improves NAS performance.
New approach to handle ranking function variation in zero-shot NAS.
New method speeds up image denoising models without sacrificing performance.
Paper proposes GP-NAS-ensemble for fast neural architecture performance prediction.
Study explores calibration properties in neural architectures.
BRP-NAS uses GCNs to predict neural network performance for more efficient NAS.
Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the runtime constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challen…
Weight-sharing (WS) has recently emerged as a paradigm to accelerate the automated search for efficient neural architectures, a process dubbed Neural Architecture Search (NAS). Although very appealing, this framework is not without drawbacks and several works have started to question its capabilities on small hand-craf…
Automates GNN design for molecular property prediction.
The use of automatic methods, often referred to as Neural Architecture Search (NAS), in designing neural network architectures has recently drawn considerable attention. In this work, we present an efficient NAS approach, named HM- NAS, that generalizes existing weight sharing based NAS approaches. Existing weight shar…
AgEBO-Tabular combines NAS and hyperparameter tuning for fast, high-performing tabular models.
Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the latency constraint of a mobile device? Neural Architecture Search (NAS) for ConvNet design is a challenging problem due to the combinatorially large design space and search time (at least 200 GPU-hours…
Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials are usually required to identify a suitable GNN architecture. It is because the performance of a GNN architecture is significantly affected b…
Improved hyperparameter optimization using simplified Transformer blocks.
Can we reduce the search cost of Neural Architecture Search (NAS) from days down to only few hours? NAS methods automate the design of Convolutional Networks (ConvNets) under hardware constraints and they have emerged as key components of AutoML frameworks. However, the NAS problem remains challenging due to the combin…
New method improves neural architecture search by optimizing for both performance and diversity.
HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.
Neural Architecture Search (NAS) aims to facilitate the design of deep networks for new tasks. Existing techniques rely on two stages: searching over the architecture space and validating the best architecture. NAS algorithms are currently compared solely based on their results on the downstream task. While intuitive, …
Finding a well-performing architecture is often tedious for both DL practitioners and researchers, leading to tremendous interest in the automation of this task by means of neural architecture search (NAS). Although the community has made major strides in developing better NAS methods, the quality of scientific empiric…
BS-NAS broadens and shrinks search space for optimal neural architectures.
Proposes baselines for joint NAS and HPO optimization.
Neural attention (NA) has become a key component of sequence-to-sequence models that yield state-of-the-art performance in as hard tasks as abstractive document summarization (ADS) and video captioning (VC). NA mechanisms perform inference of context vectors; these constitute weighted sums of deterministic input sequen…
A method to reduce memory usage in NAS by pruning the search space.
Study on encoding neural architectures for NAS, showing impact on performance.
DSNAS optimizes neural architecture and parameters in one step.
SemiNAS reduces NAS cost by predicting accuracy of unlabeled architectures.