Deep-n-Cheap automates deep learning model search for low complexity.
problem Finding efficient deep learning models for various datasets.
method Automated search framework for architecture and hyperparameters, including search transfer.
result Models offer comparable performance to state-of-the-art but are faster to train.
MNMS learns from previous tasks to speed up model search for new tasks.
problem NAS requires extensive manual design and optimization for each new task.
method MNMS uses reinforcement learning to condition model construction on previously successful searches.
result MNMS can conduct simultaneous searches for multiple tasks and transfer knowledge to new tasks.
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
FiGS searches over a larger space of architectures for efficient mobile models.
problem Designing small, efficient deep networks for mobile devices.
method Differentiable search method using sparse regularization and Logistic-Sigmoid distribution.
result FiGS produces state-of-the-art parameter-efficient models on ImageNet and improves object detection performance.
This paper compares Grid Search, Random Search, and Genetic Algorithm for NAS.
problem Hyperparameter optimization for neural architecture search.
method Comparison of Grid Search, Random Search, and Genetic Algorithm.
result Genetic Algorithm outperforms Grid Search and Random Search in terms of accuracy and execution time.
Bonsai-Net efficiently discovers state-of-the-art models with fewer parameters.
problem Efficiently discovering state-of-the-art neural architectures with minimal computational expense.
method Bonsai-Net uses a modified differential pruner to explore a relaxed search space.
result Bonsai-Net consistently discovers better architectures than random search with fewer parameters.
Paper addresses bias in search intent affecting click behavior.
problem Bias in user search intent affects click behavior and relevance.
method Proposes a search intent bias hypothesis to improve click models.
result Click models can better interpret user clicks and improve retrieval performance.
New method reduces NAS search time and complexity.
problem High computational cost and complexity in NAS.
method Differentiable search space with annealing and pruning.
result Achieves 1.68% error on CIFAR-10 with 0.2 GPU days.
Modeling high-frequency speculative markets as auction search processes.
problem Understanding trading dynamics in high-frequency order-driven markets.
method Total order book model with diffusion-drift-reaction model, inspired by foraging and chemotaxis.
result Analytic and numerical analysis of trading performance in various search mechanisms.
TGLS generates text by optimizing search results and learning from them.
problem Unsupervised text generation without explicit training data.
method TGLS uses simulated annealing to estimate sentence quality, then a conditional generative model learns from search results.
result TGLS outperforms unsupervised baselines and matches state-of-the-art supervised methods in paraphrase generation.
Beam search improves feature selection for better model performance.
problem Improving feature selection for better model performance.
method Proposed beam search as a generalization of forward selection for feature selection.
result Beam search can outperform forward selection, especially with correlated features.
A new language for neural architecture search decouples search spaces and algorithms.
problem Current neural architecture search methods are limited to specific use-cases and lack general-purpose constructs.
method Proposes a formal language for encoding search spaces over general computational graphs, allowing modular, composable, and reusable encodings.
result The language enables easy experimentation with different search spaces and algorithms without reinventing the wheel.
Deep learning models improve talent search at LinkedIn.
problem Match candidates to hiring needs using complex feature interactions.
method Deep and representation learning models, including neural network models and learning to rank approaches.
result Improved offline and online evaluation results for talent search systems.
Novel framework learns efficient student models from teacher networks.
problem Model capacity gap between teacher and student networks.
method Neural architecture search and oracle knowledge distillation.
result Searched student models often outperform teacher models.
Adaptive contrastive search improves text generation quality and diversity.
problem Decoding high-quality text from language model outputs.
method Adaptive contrastive search incorporating uncertainty-guided degeneration penalty.
result Enhanced text generation quality and diversity across different models and datasets.
Efficient search methods can outperform random search on challenging tasks.
problem Comparing the performance of efficient and random search methods in neural architecture search.
method Comparison of weight sharing and random search methods on progressively larger search spaces for image classification and detection.
result Efficient search methods can provide substantial gains over random search on large, realistic tasks.
New oracle Search improves active learning performance exponentially.
problem Enhancing active learning with limited oracle access.
method Combines Label and Search oracles for better decision-making.
result Exponential improvement in problem-solving performance.
Adapts large transformer model for search query intent understanding.
problem Understanding and predicting user intents from search queries.
method Adapts BERT-like architecture for search queries, accounts for query noisiness and sparseness.
result Builds a shareable deep learning model for query intent identification.
Efficient neural architecture search by sampling structure and operations.
problem Efficiently searching for optimal neural architectures.
method Decouples structure and operation search, using reinforcement learning with policy vectors.
result Significantly improved efficiency compared to traditional methods.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
LA-MCTS learns search space partition for black-box optimization using Monte Carlo Tree Search.
problem High-dimensional black-box optimization challenges.
method LA-MCTS recursively splits search space into regions with high/low function values, learns nonlinear partition and local models online.
result LA-MCTS achieves strong performance in black-box optimization and reinforcement learning benchmarks, especially for high-dimensional problems.
Tune simplifies hyperparameter search for distributed computing.
problem Adapting hyperparameter search to distributed environments.
method Unified framework for model selection and training.
result Simplifies implementation of state-of-the-art search algorithms.
A method for policy search with high-dimensional context variables.
problem Learning from high-dimensional context variables like camera images is challenging.
method Model-based relative entropy stochastic search framework with integrated dimensionality reduction.
result The proposed method outperforms naive dimensionality reduction methods.
New algorithms improve contextual search in the presence of adversarial corruptions.
problem Improving search accuracy in dynamic pricing settings with corrupted responses.
method Two algorithms based on binary search and gradient descent methods.
result Achieve near-optimal regret in the absence of adversarial corruptions and gracefully degrade with corrupted agents.
New metrics for multilingual search improve user satisfaction.
problem No offline metrics to evaluate multilingual search quality.
method Developed a novel intent-aware user behavior model to improve multilingual search metrics.
result Improved metrics correlate better with user satisfaction.
This work recommends personalized search stories to users based on their interests.
problem Personalized search story recommendation within search engines.
method Deep reinforcement learning architecture trained by imitation learning and reinforcement learning.
result Empirically demonstrated effectiveness on real-world data sets.
Paper tackles real-world e-commerce search efficiency and user experience.
problem Efficiently rank large-scale e-commerce search results with multiple factors.
method Design and deploy a novel Cascade ranking model in a large-scale operational e-commerce search application.
result Demonstrates the advantage of the proposed model in addressing multiple factors of effectiveness, efficiency, and user experience.
SAVE combines Q-learning and MCTS with amortized value estimates for improved performance.
problem Combining model-free Q-learning and model-based MCTS for efficient learning and planning.
method SAVE uses a learned prior to guide MCTS, which estimates improved state-action values. These estimates are used to update the prior, creating a cooperative relationship between learning and search.
result SAVE achieves higher rewards with fewer training steps and strong performance with small search budgets.
Interstellar searches for recurrent architecture to enhance KG embedding.
problem Learning long-term information in KGs.
method Recurrent neural architecture search for relational paths.
result Effectiveness and efficiency of searched models.
FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
Efficient neural architecture search discovers top-performing models.
problem Finding optimal neural architectures efficiently.
method Hierarchical genetic representation and expressive search space.
result Discovered architectures outperform manually designed models.
A new search-control strategy improves Dyna's efficiency.
problem Improving sample efficiency in model-based reinforcement learning.
method Proposes a novel search-control strategy by sampling high frequency regions of the value function.
result Empirically shows that high frequency regions require more samples to approximate, suggesting a better search-control strategy.
Study compares variable selection methods for model evaluation and search.
problem Understanding underlying mechanisms in scientific questions through variable selection.
method Comprehensive comparison of BIC and AIC for model evaluation and various search methods (exhaustive, greedy, LASSO path, stochastic search) for model space exploration.
result Exhaustive search BIC and stochastic search BIC outperform other methods in small and large model spaces, respectively, improving correct identification rate and reducing false discovery rate.
AMEIR automates recommender system design using NAS.
problem Manual feature and architecture engineering in recommender systems.
method Three-stage search space and tailored three-step pipeline.
result AMEIR outperforms manual and complex NAS methods.
Framework for automatically designing and training deep models.
problem Manual design of deep architectures is slow and error-prone.
method Extensible language for representing complex search spaces over architectures and hyperparameters. Uses tree-structured search spaces and various algorithms like MCTS and SMBO.
result MCTS and SMBO outperform random search in model training.
Efficient algorithm finds fast Transformer models.
problem Slow inference time of Transformer models.
method Decompose Transformer architecture into components, use sampling-based one-shot search.
result Achieved 10% to 30% speedup on pre-trained BERT and 70% on top of a previous state-of-the-art model.
Beam search policies learned via imitation learning.
problem Beam search policies are not explicitly learned by models during training.
method Developed a meta-algorithm for learning beam search policies using imitation learning.
result Showed no-regret guarantees for learning beam search policies.
Optimizes search paths in river environments using Finslerian geometry.
problem Optimizing search paths in river environments.
method Using Finslerian geometry and time-optimal paths based on Randers metric.
result Time-optimal paths in river environments.
Predict accuracy of neural architectures using non-neural models.
problem Improving efficiency and effectiveness of neural architecture search.
method Used gradient boosting decision tree (GBDT) for accuracy prediction and pruned the search space based on features learned from GBDT.
result GBDT-based predictor achieves comparable or better accuracy than neural network-based predictors and is 22x more sample efficient on NASBench-101.
Method predicts disease outbreaks using search logs, overcoming instability.
problem Predicting disease outbreaks from search logs is challenging due to short-term and long-term instability.
method Seasonal-adjustment method decomposes logs into seasonal, trend, and irregular components; feature selection method selects relevant search terms.
result Proposed method outperforms comparative methods in prediction accuracy for seven of ten diseases.
Automated framework forecasts correlated time series in minutes.
problem Forecasting correlated time series with high accuracy and efficiency.
method Data-driven iterative pruning, zero-shot search, fast parameter adaptation.
result Framework achieves state-of-the-art accuracy and is faster than existing methods.
Petridish efficiently searches neural architectures by iteratively adding shortcut connections.
problem Finding efficient neural architectures for various tasks.
method Iteratively adds shortcut connections to existing network layers, motivated by feature selection.
result Petridish efficiently finds competitive models with few GPU days.
HotNAS reduces AI search time from hundreds of GPU hours to less than 3 GPU hours.
problem High time required for AI solution generation from scratch.
method HotNAS starts from a 'hot' state using pre-trained models and integrates compression during co-search.
result Reduces search time from 200 GPU hours to less than 3 GPU hours.
Graph-based NAS improves sample efficiency in architecture design.
problem Current NAS search spaces are static sequences, limiting expressiveness.
method Proposed graph-based search space with vertices and edges for iterative and branching decisions.
result Graph representation improves sample efficiency in architecture design.
AutoOD automates outlier detection using curiosity-guided search and self-imitation learning.
problem Automated outlier detection for complex tasks with big data.
method Curiosity-guided search strategy and self-imitation learning.
result AutoOD identifies optimal neural network models with superior performance.
This study investigates global normalization in neural models, showing its effectiveness in search-aware training.
problem Theoretical equivalence of global and local normalization in high-capacity models, practical advantage unclear.
method Continuous relaxation of beam search for training globally normalized recurrent sequence models.
result Globally normalized models are more effective than locally normalized ones in inexact search.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.