This paper compares Grid Search, Random Search, and Genetic Algorithm for NAS.
problem Hyperparameter optimization for neural architecture search.
method Comparison of Grid Search, Random Search, and Genetic Algorithm.
result Genetic Algorithm outperforms Grid Search and Random Search in terms of accuracy and execution time.
A new language for neural architecture search decouples search spaces and algorithms.
problem Current neural architecture search methods are limited to specific use-cases and lack general-purpose constructs.
method Proposes a formal language for encoding search spaces over general computational graphs, allowing modular, composable, and reusable encodings.
result The language enables easy experimentation with different search spaces and algorithms without reinventing the wheel.
Tune simplifies hyperparameter search for distributed computing.
problem Adapting hyperparameter search to distributed environments.
method Unified framework for model selection and training.
result Simplifies implementation of state-of-the-art search algorithms.
New findings limit the effectiveness of machine learning algorithms.
problem Limiting the effectiveness of machine learning algorithms.
method Analyzing the proportion of problems favorable for a fixed algorithm.
result No single algorithm can perform well over a large fraction of problems.
Beam search policies learned via imitation learning.
problem Beam search policies are not explicitly learned by models during training.
method Developed a meta-algorithm for learning beam search policies using imitation learning.
result Showed no-regret guarantees for learning beam search policies.
NAS helps find best neural network designs.
problem Designing optimal neural network architectures.
method Optimization algorithms and search spaces.
result Introduction to major advances in NAS for CNNs.
Petridish efficiently searches neural architectures by iteratively adding shortcut connections.
problem Finding efficient neural architectures for various tasks.
method Iteratively adds shortcut connections to existing network layers, motivated by feature selection.
result Petridish efficiently finds competitive models with few GPU days.
New hierarchical search algorithm improves neural architecture design across different operator sets.
problem DARTS's performance drops when search space changes due to operator correlation and optimization complexity.
method Operator clustering and optimization complexity matching in a hierarchical search algorithm.
result The algorithm consistently finds high-performance architectures across various search spaces, outperforming other methods.
Describes MLC search spaces in MEKA and WEKA software.
problem Understanding MLC algorithms and their transformations into SLC problems.
method Overviewed 26 MLC algorithms and 28 SLC algorithms, proposed a context-free grammar.
result Formal description of MLC search spaces and their transformations.
A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.
problem Non-convex objective functions in simulation optimization.
method Integrates adaptive sampling with recursive partitioning of the search space.
result Proves global convergence and reliably identifies the global optimum.
Approaches to learning Bayesian networks from data typically combine a scoring function with a heuristic search procedure. Given a Bayesian network structure, many of the scoring functions derived in the literature return a score for the entire equivalence class to which the structure belongs. When using such a scoring…
Efficient neural architecture search discovers top-performing models.
problem Finding optimal neural architectures efficiently.
method Hierarchical genetic representation and expressive search space.
result Discovered architectures outperform manually designed models.
Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.
problem Efficient optimisation of expensive black-box functions in unknown search spaces.
method Expands search space over iterations based on a hyperharmonic series, scales to high dimensions.
result Sub-linear regret growth for both algorithms.
New algorithms improve contextual search in the presence of adversarial corruptions.
problem Improving search accuracy in dynamic pricing settings with corrupted responses.
method Two algorithms based on binary search and gradient descent methods.
result Achieve near-optimal regret in the absence of adversarial corruptions and gracefully degrade with corrupted agents.
NATS-Bench benchmarks NAS algorithms for architecture topology and size.
problem Incomparable performance of NAS algorithms due to different search spaces and training setups.
method Unified benchmarking platform for architecture topology and size searching.
result Validated benchmark for 15,625 topology and 32,768 size candidates.
FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
A neural network approach to Monte-Carlo tree search.
problem Improving tree search algorithms for planning problems.
method Learning neural network architecture to control search parameters.
result The learned search algorithm outperformed traditional MCTS.
Quality-Diversity algorithms explore multiple high-performing solutions in a search space.
problem Finding multiple high-performing solutions in complex optimization problems.
method Evolutionary computation approach focusing on behavioral space and holistic solution distribution.
result Quality-Diversity algorithms provide a comprehensive view of high-performing solutions in a search space.
Empirical comparison of 18 hyperparameter tuning algorithms for SVM.
problem Tuning hyperparameters C and γ for SVM with RBF kernel. method Compared 18 search algorithms on 115 real-life data sets.
result Trees of Parzen estimators and particle swarm optimization perform similarly to grid search.
Parallel algorithm finds sparse solutions for nonconvex problems.
problem Nonconvex sparsity-regularized rank minimization.
method Parallel best-response algorithm with exact line search.
result Guaranteed convergence to a stationary point.
Neural A* uses machine learning to improve path planning efficiency.
problem Challenges in applying machine learning to search-based path planning.
method Reformulated A* search as a differentiable network coupled with a convolutional encoder.
result Neural A* outperforms state-of-the-art planners in optimality and efficiency.
Unified framework for combinatorial and rounding algorithms in experimental design.
problem Designing and analyzing combinatorial and rounding algorithms for experimental design problems.
method Local search framework for combinatorial algorithms and regret minimization framework for rounding algorithms.
result Unified approach to match and improve all known results in D/A/E-design and obtain new results in unknown settings.
New algorithm speeds up Bayesian structure learning in Gaussian graphical models.
problem Computational bottleneck in evaluating ratios of G-Wishart normalizing constants.
method Explicit closed-form approximation of the ratio of normalizing constants within the search algorithm.
result Significant improvement in scalability of structure learning without sacrificing accuracy.
Algorithm improves search efficiency for sparse signals using region sensing.
problem Efficiently search for sparse signals in large spaces.
method Greedy maximization of information gain using noisy average region measurements.
result Requires fewer measurements to recover signal locations compared to passive methods.
Contrastive embeddings improve neural architecture search performance.
problem Improving performance of neural architecture search algorithms.
method Contrastive learning to identify networks based on data Jacobians and produce embeddings.
result Traditional black-box optimization algorithms can reach state-of-the-art performance with contrastive embeddings.
New metrics solve machine learning limitations.
problem Previous success metrics restrict application to specific forms of machine learning.
method Define decomposable metrics as linear operations on probability distributions.
result Demonstrated theorems bounding success in various ways, generalizing existing results.
This study evaluates cluster search algorithms using Gaussian mixture models.
problem Determining the optimal number of clusters in data sets generated by Gaussian mixture models.
method Examined centroid- and model-based cluster search algorithms in various cases.
result Model-based algorithms are more robust to cluster overlap and covariance type than centroid-based methods.
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
AgABC improves ABC algorithm by balancing exploration and exploitation.
problem Balancing global and local search abilities in ABC algorithm.
method Divide population into groups and assign different search strategies to members.
result Proposed AgABC algorithm outperforms other algorithms in accuracy and stability.
Paper proposes a new evaluation method for NAS search phase.
problem NAS search phase effectiveness not well evaluated.
method Compare NAS solutions with random selection; evaluate weight sharing strategy.
result State-of-the-art NAS algorithms perform similarly to random selection.
DARTS efficiently searches for high-performance architectures using gradient descent.
problem Scalability challenge of architecture search in machine learning.
method Differentiable relaxation of architecture representation for continuous search space.
result Orders of magnitude faster than non-differentiable techniques.
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves …
New oracle Search improves active learning performance exponentially.
problem Enhancing active learning with limited oracle access.
method Combines Label and Search oracles for better decision-making.
result Exponential improvement in problem-solving performance.
A new search algorithm identifies causal effects from incomplete data.
problem Identifying causal effects from incomplete data sources.
method A search-based algorithm over do-calculus rules.
result The approach is complete for a wide range of identifiability problems.
Fast AutoAugment speeds up data augmentation search.
problem Efficiently searching for effective data augmentation policies.
method Density matching-based search strategy.
result Comparable performance with faster search time.
Evo-NAS combines neural and evolutionary methods for efficient neural architecture search.
problem Efficiently searching for optimal neural architectures in deep learning.
method Evolutionary-Neural hybrid agents that combine the strengths of neural and evolutionary algorithms.
result Evo-NAS outperforms both neural and evolutionary agents in architecture search for various classification tasks.
New algorithm improves similarity graph construction for nearest neighbor search.
problem Improving nearest neighbor search performance with more effective similarity graphs.
method Probabilistic model of a similarity graph learned through reinforcement learning.
result Higher recall rates achieved for the same number of distance computations.
Learning Bayesian networks is often cast as an optimization problem, where the computational task is to find a structure that maximizes a statistically motivated score. By and large, existing learning tools address this optimization problem using standard heuristic search techniques. Since the search space is extremely…
Auto-Keras efficiently searches neural architectures with less computation.
problem Expensive computational cost in existing NAS algorithms.
method Bayesian optimization guided by network morphism.
result Framework outperforms state-of-the-art methods on real-world datasets.
Interstellar searches for recurrent architecture to enhance KG embedding.
problem Learning long-term information in KGs.
method Recurrent neural architecture search for relational paths.
result Effectiveness and efficiency of searched models.
New stochastic gradient descent with random search directions improves efficiency and convergence.
problem Efficiency and convergence of stochastic gradient descent methods.
method Developed a new class of stochastic gradient descent algorithms with random search directions.
result Established almost sure convergence and provided Lp rates of convergence. Algorithm speeds up search for stationary targets with guaranteed accuracy.
problem Minimize search time while ensuring high detection accuracy of stationary targets.
method Multi-fidelity Gaussian process model and EMTS algorithm.
result Guaranteed performance in target detection accuracy and search time.
NAS favors wide and shallow cell structures, leading to fast convergence but not necessarily better generalization.
problem Understanding and improving the architectures generated by NAS algorithms.
method Empirical and theoretical study of existing NAS algorithms (DARTS, ENAS) and their architectures.
result Existing NAS algorithms favor wide and shallow cell structures, leading to fast convergence but not necessarily better generalization.
Neurally-Guided Structure Inference combines search and data-driven methods for efficient, robust structure inference.
problem Combining the advantages of exhaustive search and data-driven methods for structure inference.
method Neurally-Guided Structure Inference (NG-SI) uses a neural network to guide hierarchical search over structures.
result NG-SI outperforms search-based and data-driven methods on probabilistic matrix decomposition and symbolic program parsing.
Seeker allows real-time feedback to refine search results.
problem Users struggle to accurately describe desired items in words.
method Interactive refinement of search results through user feedback.
result Seeker improves search quality through user feedback.
New agent learns from previous search spaces to improve NAS efficiency.
problem NAS requires restarting learning from scratch between different search spaces.
method Transformer-based agent for joint training and knowledge transfer.
result Efficient knowledge transfer between search spaces improves NAS performance.