Automated framework forecasts correlated time series in minutes.
problem Forecasting correlated time series with high accuracy and efficiency.
method Data-driven iterative pruning, zero-shot search, fast parameter adaptation.
result Framework achieves state-of-the-art accuracy and is faster than existing methods.
Survey of 1000 NAS papers, automating neural architecture design.
problem Designing high-performing neural architectures for various tasks.
method Taxonomy of search spaces, algorithms, and speedup techniques.
result NAS has surpassed human-designed architectures on many tasks.
Survey of techniques for automating hyperparameter optimization in machine learning.
problem Finding optimal hyperparameters for machine learning algorithms.
method Review of various hyperparameter optimization techniques.
result Unified treatment of hyperparameter optimization techniques.
AutoOD automates outlier detection using curiosity-guided search and self-imitation learning.
problem Automated outlier detection for complex tasks with big data.
method Curiosity-guided search strategy and self-imitation learning.
result AutoOD identifies optimal neural network models with superior performance.
The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architecture search. The choice of the network architecture has proven to be critical, and many advances in deep learning spring from its immediate im…
Automates neural network design without training, speeding up search by seconds.
problem Time and effort in hand-designing deep neural networks.
method Predict trained accuracy from untrained network state using activation overlap.
result Search for powerful networks in seconds, verified on various benchmarks.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
Fast, fully-automated histograms for large data sets.
problem Efficiently constructing histograms for large-scale data.
method G-Enum histograms using Minimum Description Length (MDL) principle for model selection and a greedy search heuristic.
result Construct histograms in linearithmic time, significantly faster than previous methods.
Fully automating machine learning pipelines is one of the key challenges of current artificial intelligence research, since practical machine learning often requires costly and time-consuming human-powered processes such as model design, algorithm development, and hyperparameter tuning. In this paper, we verify that au…
PyODDS automates outlier detection for new data sources.
problem Manual outlier detection is inefficient and domain-specific.
method Automated end-to-end system with Apache Spark and database support.
result PyODDS optimizes outlier detection pipelines automatically.
This paper presents preliminary work on learning the search heuristic for the optimal motion planning for automated driving in urban traffic. Previous work considered search-based optimal motion planning framework (SBOMP) that utilized numerical or model-based heuristics that did not consider dynamic obstacles. Optimal…
RandomNet uses random search to design neural architectures without much human intervention.
problem Designing neural architectures without excessive human intervention.
method Random search strategy for multimodal neural architecture design.
result RandomNet performs close to state-of-the-art on AV-MNIST with minimal human supervision.
GAMA is a user-friendly AutoML system for machine learning pipeline optimization.
problem Empowering users to control and optimize machine learning pipelines.
method Modular AutoML system with three search algorithms and two post-processing steps.
result GAMA allows users to track and control the search process for optimal machine learning pipelines.
Deep-n-Cheap automates deep learning model search for low complexity.
problem Finding efficient deep learning models for various datasets.
method Automated search framework for architecture and hyperparameters, including search transfer.
result Models offer comparable performance to state-of-the-art but are faster to train.
To relieve the pain of manually selecting machine learning algorithms and tuning hyperparameters, automated machine learning (AutoML) methods have been developed to automatically search for good models. Due to the huge model search space, it is impossible to try all models. Users tend to distrust automatic results and …
Study compares MAPF and MARL algorithms for warehouse automation.
problem Optimizing multi-agent pickup and delivery in warehouse settings.
method Compared conflict-based search (MAPF) and shared experience actor-critic (MARL).
result Comprehensive benchmarking of MAPF and MARL in a simulated warehouse environment.
The process of designing neural architectures requires expert knowledge and extensive trial and error. While automated architecture search may simplify these requirements, the recurrent neural network (RNN) architectures generated by existing methods are limited in both flexibility and components. We propose a domain-s…
As an emerging field, Automated Machine Learning (AutoML) aims to reduce or eliminate manual operations that require expertise in machine learning. In this paper, a graph-based architecture is employed to represent flexible combinations of ML models, which provides a large searching space compared to tree-based and sta…
Automates optimizer design for diverse tasks efficiently.
problem Scalability, generability, and sample efficiency in optimizer search.
method Re-arranged optimizer space into a super-tree, applying tree traversal methods.
result Discover optimizers with only 128 evaluations, surpassing human-designed and prior methods.
AutoRec automates deep recommendation models using AutoML.
problem Adapting to changing data and tasks in recommender systems.
method AutoML platform for model search and hyperparameter tuning.
result Identifies models similar to the best model without prior knowledge.
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Parallelizes MCTS for continuous domains using leaf and root parallelization.
problem Solving challenging tasks in continuous domains using MCTS.
method Extends existing parallelization strategies to continuous domains, focusing on leaf and root parallelization.
result Proposes two final selection strategies for continuous states in root parallelization.
Deep learning models require extensive architecture design exploration and hyperparameter optimization to perform well on a given task. The exploration of the model design space is often made by a human expert, and optimized using a combination of grid search and search heuristics over a large space of possible choices…
Recent progress in Generative Adversarial Networks (GANs) has shown promising signs of improving GAN training via architectural change. Despite some early success, at present the design of GAN architectures requires human expertise, laborious trial-and-error testings, and often draws inspiration from its image classifi…
Automated discovery of adaptive attacks improves adversarial defense evaluation.
problem Challenges in reliably evaluating adversarial defenses.
method Formalizes adaptive attacks as reusable building blocks in a search space for automatic discovery.
result Our tool discovers significantly stronger attacks than AutoAttack, improving adversarial defense evaluation.
Automates supervised learning pipeline design with matrix and tensor factorization.
problem Designing effective supervised learning pipelines with many choices.
method Uses matrix and tensor factorization to model pipeline search space and develops greedy experiment design protocols.
result Demonstrates the effectiveness of the approach on real-world classification problems.
Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation. One crucial aspect for this progress are novel neural architectures. Currently employed architectures have mostly been developed manually by human experts, whi…
Auto-Surprise automates recommender system selection and optimization.
problem Finding the best algorithm and hyperparameters for recommender systems.
method Extends Surprise library with TPE optimization for algorithm selection and hyperparameter tuning.
result Significantly faster in finding optimal hyperparameters compared to grid search.
Mango automates hyperparameter tuning for large-scale ML training.
problem Manual hyperparameter tuning is tedious and inefficient for large-scale machine learning.
method Parallel hyperparameter tuning with intelligent search strategies and flexible abstractions.
result Mango achieves comparable performance to Hyperopt while supporting distributed computing.
A method to reduce memory usage in NAS by pruning the search space.
problem High GPU memory consumption in One-Shot NAS techniques.
method Utilising Zero-Shot NAS to prune the search space before applying One-Shot NAS.
result Reduces memory consumption by 81% while maintaining accuracy.
The increasing availability of large but noisy data sets with a large number of heterogeneous variables leads to the increasing interest in the automation of common tasks for data analysis. The most time-consuming part of this process is the Exploratory Data Analysis, crucial for better domain understanding, data clean…
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
Machine learning (ML) methods have been developing rapidly, but configuring and selecting proper methods to achieve a desired performance is increasingly difficult and tedious. To address this challenge, automated machine learning (AutoML) has emerged, which aims to generate satisfactory ML configurations for given tas…
Weight-sharing is rarely helpful in NAS, especially on small datasets.
problem The efficiency of weight-sharing in Neural Architecture Search (NAS) is questionable.
method Comparison of a state-of-the-art weight-sharing approach to random search on the nasbench dataset.
result Weight-sharing is only rarely significantly helpful in NAS, highlighting the importance of the search space.
Neural Architecture Search has shown potential to automate the design of neural networks. Deep Reinforcement Learning based agents can learn complex architectural patterns, as well as explore a vast and compositional search space. On the other hand, evolutionary algorithms offer higher sample efficiency, which is criti…
Review of automation's role in chemical discoveries.
problem Improving autonomous discovery in chemistry.
method Classification of discovery types, assessment of autonomy, case studies.
result Rapid advancements in automation and machine learning are transforming experimentation and modeling.
Automated method bounds causal effects in discrete data.
problem Partial identification of causal effects in discrete settings.
method Polynomial programming and dual relaxation for automated bounds.
result Algorithm provides guaranteed non-sharp and ε-sharp bounds. Automatically extracts hyperparameter schemas from AI library documentation.
problem Lack of machine-readable hyperparameter schemas for AI automation tools.
method Automatically mines Python docstrings in AI libraries to extract JSON Schemas.
result Effective at extracting machine-readable schemas from 119 AI models.
Automates GNN design for molecular property prediction.
problem Designing and tuning GNN architectures for molecular property prediction is labor-intensive.
method Developed a NAS approach to automatically discover high-performing GNN architectures for MPNNs.
result Automatically discovered MPNNs outperform manually designed GNNs in molecular property prediction.
Spectral clustering algorithms typically require a priori selection of input parameters such as the number of clusters, a scaling parameter for the affinity measure, or ranges of these values for parameter tuning. Despite efforts for automating the process of spectral clustering, the task of grouping data in multi-scal…
FiGS searches over a larger space of architectures for efficient mobile models.
problem Designing small, efficient deep networks for mobile devices.
method Differentiable search method using sparse regularization and Logistic-Sigmoid distribution.
result FiGS produces state-of-the-art parameter-efficient models on ImageNet and improves object detection performance.
NAS-Navigator automates neural network architecture search with visual steering.
problem Difficulty in configuring large neural networks efficiently.
method Formulates architecture optimization as graph space exploration, trains all candidate architectures in one-shot.
result Allows analysts to effectively select and guide the search for optimal neural network architectures.
Automates bias control in reinforcement learning algorithms.
problem Overestimation bias in reinforcement learning algorithms.
method Data-driven approach for automatic selection of bias control hyperparameters.
result Significant reduction in the number of interactions while maintaining performance.
While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few …
A method to improve LLMs by automating the construction of a mixture of expert prompts.
problem Limitation of single instruction prompts in covering complex problem spaces.
method Divide the problem space into sub-regions, each governed by a specialized expert with both an instruction and demos. A two-phase process constructs these experts.
result Achieves an average win rate of 81% across major benchmarks.
Automates design of lightweight neural networks for image classification.
problem Designing efficient neural networks for edge devices with limited computational resources.
method Uses the Mesh Adaptive Direct Search (MADS) algorithm to optimize network architecture.
result Achieves comparable performance to standard methods with fewer design trials.
We propose a novel method that makes use of deep neural networks and gradient decent to perform automated design on complex real world engineering tasks. Our approach works by training a neural network to mimic the fitness function of a design optimization task and then, using the differential nature of the neural netw…
MCD automates counterfactual design searches for multi-modal tasks.
problem Designing for multi-objective goals and complex constraints.
method Model-agnostic counterfactual search method for multi-modal design modifications.
result MCD streamlines and automates counterfactual search, recommending effective design modifications.