Efficient search methods can outperform random search on challenging tasks.
problem Comparing the performance of efficient and random search methods in neural architecture search.
method Comparison of weight sharing and random search methods on progressively larger search spaces for image classification and detection.
result Efficient search methods can provide substantial gains over random search on large, realistic tasks.
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
Recently, randomly mapping vectorial data to strings of discrete symbols (i.e., sketches) for fast and space-efficient similarity searches has become popular. Such random mapping is called similarity-preserving hashing and approximates a similarity metric by using the Hamming distance. Although many efficient similarit…
MICO uses mutual information co-training to improve selective search efficiency.
problem Efficiently search and route unseen queries in large-scale search systems.
method Mutual Information Co-training framework for selective search with minimal supervision.
result Significantly improves performance on multiple metrics of selective search.
A method to reduce memory usage in NAS by pruning the search space.
problem High GPU memory consumption in One-Shot NAS techniques.
method Utilising Zero-Shot NAS to prune the search space before applying One-Shot NAS.
result Reduces memory consumption by 81% while maintaining accuracy.
Improves neural architecture search methods to be more stable and efficient.
problem Neural architecture search methods are unstable and sensitive to hyperparameters.
method Discusses practical considerations to improve stability and efficiency.
result Improves overall performance of neural architecture search methods.
Automates optimizer design for diverse tasks efficiently.
problem Scalability, generability, and sample efficiency in optimizer search.
method Re-arranged optimizer space into a super-tree, applying tree traversal methods.
result Discover optimizers with only 128 evaluations, surpassing human-designed and prior methods.
Paper uses CMAB to improve NAS efficiency and accuracy.
problem Improving efficiency and accuracy of NAS for DNNs.
method Formulated NAS as CMAB, used Nested Monte-Carlo Search.
result Discovered cell structure achieves comparable accuracy to state-of-the-art, 20x faster.
FiGS searches over a larger space of architectures for efficient mobile models.
problem Designing small, efficient deep networks for mobile devices.
method Differentiable search method using sparse regularization and Logistic-Sigmoid distribution.
result FiGS produces state-of-the-art parameter-efficient models on ImageNet and improves object detection performance.
Neural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a neural network while changing its neural architecture, could be helpfu…
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
Bonsai-Net efficiently discovers state-of-the-art models with fewer parameters.
problem Efficiently discovering state-of-the-art neural architectures with minimal computational expense.
method Bonsai-Net uses a modified differential pruner to explore a relaxed search space.
result Bonsai-Net consistently discovers better architectures than random search with fewer parameters.
Retro* uses neural networks to efficiently find high-quality synthetic routes in organic chemistry.
problem Finding efficient synthetic routes in organic chemistry is challenging due to the vast search space.
method Retro* is a neural-based A*-like algorithm that learns a neural search bias to guide efficient best-first search.
result Retro* outperforms existing methods in both success rate and solution quality while being more efficient.
Recent advances in Neural Architecture Search (NAS) have produced state-of-the-art architectures on several tasks. NAS shifts the efforts of human experts from developing novel architectures directly to designing architecture search spaces and methods to explore them efficiently. The search space definition captures pr…
Simplified NAS for GNN architectures improves efficiency and expressiveness.
problem Efficient and effective discovery of optimal GNN architectures.
method SNAG framework with a novel search space and reinforcement learning.
result SNAG framework outperforms human-designed and existing NAS methods.
We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human ex…
Neural network architectures found by sophistic search algorithms achieve strikingly good test performance, surpassing most human-crafted network models by significant margins. Although computationally efficient, their design is often very complex, impairing execution speed. Additionally, finding models outside of the …
Neural A* uses machine learning to improve path planning efficiency.
problem Challenges in applying machine learning to search-based path planning.
method Reformulated A* search as a differentiable network coupled with a convolutional encoder.
result Neural A* outperforms state-of-the-art planners in optimality and efficiency.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
MiLeNAS improves neural architecture search by reducing approximation errors and achieving better accuracy.
problem Improving efficiency and accuracy in neural architecture search (NAS).
method Mixed-level reformulation (MiLeNAS) to optimize efficiently and reliably.
result MiLeNAS achieves lower validation error and higher accuracy than bilevel optimization methods.
A new search-control strategy improves Dyna's efficiency.
problem Improving sample efficiency in model-based reinforcement learning.
method Proposes a novel search-control strategy by sampling high frequency regions of the value function.
result Empirically shows that high frequency regions require more samples to approximate, suggesting a better search-control strategy.
RARTS improves neural architecture search efficiency and accuracy.
problem Efficiency and accuracy issues in first-order DARTS.
method Formulates a relaxed architecture search (RARTS) method that uses whole dataset splitting.
result RARTS outperforms DARTS and its variants in accuracy and efficiency.
Efficiently selects nearest neighbors for labeling to speed up active learning.
problem Intractable active learning and search for large-scale unlabeled data.
method Restricts candidate pool to nearest neighbors of labeled set.
result Achieved similar performance to global approach but reduced computational cost by up to 3 orders of magnitude.
In the 'Big Data' era, many real-world applications like search involve the ranking problem for a large number of items. It is important to obtain effective ranking results and at the same time obtain the results efficiently in a timely manner for providing good user experience and saving computational costs. Valuable …
Neural Architecture Search (NAS) has emerged as a promising technique for automatic neural network design. However, existing MCTS based NAS approaches often utilize manually designed action space, which is not directly related to the performance metric to be optimized (e.g., accuracy), leading to sample-inefficient exp…
BO limits search to lower dimensions with LGPR, improving efficiency and accuracy.
problem High computational costs in BO for high-dimensional optimization.
method Limits search region to lower dimensions and uses LGPR for local Gaussian process regression.
result Search efficiencies improved by about 69% and 40% for Ackley and Rosenbrock functions.
Weight-sharing is rarely helpful in NAS, especially on small datasets.
problem The efficiency of weight-sharing in Neural Architecture Search (NAS) is questionable.
method Comparison of a state-of-the-art weight-sharing approach to random search on the nasbench dataset.
result Weight-sharing is only rarely significantly helpful in NAS, highlighting the importance of the search space.
We propose a novel neural architecture search algorithm via reinforcement learning by decoupling structure and operation search processes. Our approach samples candidate models from the multinomial distribution on the policy vectors defined on the two search spaces independently. The proposed technique improves the eff…
Efficient algorithm finds fast Transformer models.
problem Slow inference time of Transformer models.
method Decompose Transformer architecture into components, use sampling-based one-shot search.
result Achieved 10% to 30% speedup on pre-trained BERT and 70% on top of a previous state-of-the-art model.
Embedding representation learning via neural networks is at the core foundation of modern similarity based search. While much effort has been put in developing algorithms for learning binary hamming code representations for search efficiency, this still requires a linear scan of the entire dataset per each query and tr…
For network architecture search (NAS), it is crucial but challenging to simultaneously guarantee both effectiveness and efficiency. Towards achieving this goal, we develop a differentiable NAS solution, where the search space includes arbitrary feed-forward network consisting of the predefined number of connections. Be…
Efficiently searches through Gaussian process kernels using symbolic representation and Bayesian optimization.
problem Manual selection of kernels in Gaussian processes is complex and computationally expensive.
method Proposes a novel method using symbolic representation and Bayesian optimization to search through a structured kernel space.
result Empirically shows a computationally more efficient way of searching through a discrete kernel space.
Neural Architecture Search (NAS) has shown great success in automating the design of neural networks, but the prohibitive amount of computations behind current NAS methods requires further investigations in improving the sample efficiency and the network evaluation cost to get better results in a shorter time. In this …
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline.…
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architectu…
Latest algorithms for automatic neural architecture search perform remarkable but are basically directionless in search space and computational expensive in training of every intermediate architecture. In this paper, we propose a method for efficient architecture search called EENA (Efficient Evolution of Neural Archit…
New method optimizes hyperparameters for non-smooth problems efficiently.
problem Efficiently tuning hyperparameters for non-smooth cost functions.
method Combines hyperparameter search with proximal gradient updates.
result Method converges to local optimum of LOO validation error.
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
Recently it was shown that the problem of Maximum Inner Product Search (MIPS) is efficient and it admits provably sub-linear hashing algorithms. Asymmetric transformations before hashing were the key in solving MIPS which was otherwise hard. In the prior work, the authors use asymmetric transformations which convert th…
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
In this paper, a new sequential surrogate-based optimization (SSBO) algorithm is developed, which aims to improve the global search ability and local search efficiency for the global optimization of expensive black-box models. The proposed method involves three basic sub-criteria to infill new samples asynchronously to…
One-Shot Neural architecture search (NAS) attracts broad attention recently due to its capacity to reduce the computational hours through weight sharing. However, extensive experiments on several recent works show that there is no positive correlation between the validation accuracy with inherited weights from the supe…
Quantum-assisted VAE improves similarity search in high-dimensional datasets.
problem Finding fast and memory-efficient similarity search in high-dimensional data.
method Construct a space-efficient search index based on the latent space of a Quantum-assisted Variational Autoencoder (QVAE).
result Real-world speedups and memory-efficient scaling to half a billion data points.
UNAS combines DNAS and RL for efficient architecture search.
problem Discovering high accuracy or low latency neural architectures.
method Unified framework combining differentiable and reinforcement learning approaches.
result UNAS achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets.
DR-MCTS improves decision quality and sample efficiency in complex environments.
problem Improving decision quality and sample efficiency in complex environments.
method Integrates Doubly Robust off-policy estimation into Monte Carlo Tree Search (MCTS).
result DR-MCTS achieves superior performance in Tic-Tac-Toe and VirtualHome tasks.
GroSS enables efficient search for grouped convolutional architectures.
problem Training grouped convolutional architectures efficiently and effectively.
method GroSS: Group-Size Series Decomposition for Grouped Architecture Search.
result Simultaneous training of differing numbers of groups within a single layer and all possible combinations between layers.
Efficiently find near-optimal medical treatments with less trial and error.
problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.
Paper evaluates robustness of NAS against poisoning attacks.
problem Robustness of Neural Architecture Search (NAS) against poisoning attacks.
method Evaluation of Efficient NAS (ENAS) against carefully designed ineffective operations in poisoning attacks.
result Demonstrates how poisoning attacks exploit design flaws in ENAS controller.