Deep-n-Cheap automates deep learning model search for low complexity.
problem Finding efficient deep learning models for various datasets.
method Automated search framework for architecture and hyperparameters, including search transfer.
result Models offer comparable performance to state-of-the-art but are faster to train.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
We consider a framework for structured prediction based on search in the space of complete structured outputs. Given a structured input, an output is produced by running a time-bounded search procedure guided by a learned cost function, and then returning the least cost output uncovered during the search. This framewor…
UNAS combines DNAS and RL for efficient architecture search.
problem Discovering high accuracy or low latency neural architectures.
method Unified framework combining differentiable and reinforcement learning approaches.
result UNAS achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets.
PDP framework learns CSP solvers without explicit search strategy.
problem Learning effective search strategies for CSP solvers.
method Proposes a generic neural framework based on propagation, decimation, and prediction.
result Demonstrates effectiveness in SAT solving compared to neural and state-of-the-art baselines.
Auto-Keras efficiently searches neural architectures with less computation.
problem Expensive computational cost in existing NAS algorithms.
method Bayesian optimization guided by network morphism.
result Framework outperforms state-of-the-art methods on real-world datasets.
Unified asymptotics for investment in markets with transaction costs and search frictions.
problem Investment in markets with transaction costs and search frictions.
method Power-utility maximization problem with proportional transaction costs and Poisson-triggered trades, analyzed using a novel asymptotic framework.
result Explicit asymptotics for the no-trade region and value function derived.
A new framework scales active search for large datasets.
problem Scaling active search for large, high-dimensional data sets.
method Hierarchical Batch Bandit Search (HBBS) framework.
result HBBS improves performance and scalability for batch search.
HotNAS reduces AI search time from hundreds of GPU hours to less than 3 GPU hours.
problem High time required for AI solution generation from scratch.
method HotNAS starts from a 'hot' state using pre-trained models and integrates compression during co-search.
result Reduces search time from 200 GPU hours to less than 3 GPU hours.
A new clustering framework optimizes customer search data for personalized travel recommendations.
problem Personalized travel recommendations based on customer search data.
method Multi-objective optimization-based clustering ensemble framework.
result Optimizes diversity in clustering ensemble search space and automatically determines the number of clusters.
Unified framework for combinatorial and rounding algorithms in experimental design.
problem Designing and analyzing combinatorial and rounding algorithms for experimental design problems.
method Local search framework for combinatorial algorithms and regret minimization framework for rounding algorithms.
result Unified approach to match and improve all known results in D/A/E-design and obtain new results in unknown settings.
Automated framework forecasts correlated time series in minutes.
problem Forecasting correlated time series with high accuracy and efficiency.
method Data-driven iterative pruning, zero-shot search, fast parameter adaptation.
result Framework achieves state-of-the-art accuracy and is faster than existing methods.
Simplified NAS for GNN architectures improves efficiency and expressiveness.
problem Efficient and effective discovery of optimal GNN architectures.
method SNAG framework with a novel search space and reinforcement learning.
result SNAG framework outperforms human-designed and existing NAS methods.
BASE framework learns task-agnostic architectures for faster NAS.
problem High computational cost in Neural Architecture Search.
method Bayesian Meta Architecture Search (BASE) framework.
result Found models achieving 25.7% top-1 error and 8.1% top-5 error in less than an hour.
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
In deep learning, performance is strongly affected by the choice of architecture and hyperparameters. While there has been extensive work on automatic hyperparameter optimization for simple spaces, complex spaces such as the space of deep architectures remain largely unexplored. As a result, the choice of architecture …
XferNAS reduces neural architecture search time by 33x.
problem Expensive neural architecture search for new tasks.
method Transfer knowledge from previous tasks to reduce search time.
result Reduction of search time from 200 to 6 GPU days.
A new NAS framework optimizes 3D medical image segmentation architectures.
problem Optimizing neural architectures for high-resolution 3D medical images.
method Stochastic sampling algorithm for scalable gradient-based optimization of neural connectivities and operation types in both encoder and decoder.
result Automatically designed architecture outperforms human-designed U-Net.
EffAcTS uses active learning to select model parameters for robust policy search.
problem Learning robust policies that perform well in unseen environments.
method EffAcTS is an active learning framework that selects model parameters using Linear Bandits to minimize data collection.
result Efficiency gains in sample collection and improved policy performance.
Neural model with parameterized algorithms improves graph CO problem solving.
problem Solving NP-hard graph combinatorial optimization problems efficiently and accurately.
method Combining neural models and parameterized algorithms to identify and handle hard and easy parts of CO instances.
result Framework produces superior solution quality and out-of-distribution generalization.
EENMF improves e-commerce sponsored search efficiency and effectiveness.
problem Improving efficiency and effectiveness of e-commerce sponsored search.
method End-to-end neural matching framework (EENMF) for vector-based ad retrieval and neural pre-ranking.
result Significantly outperforms baseline in real e-commerce traffic.
MICO uses mutual information co-training to improve selective search efficiency.
problem Efficiently search and route unseen queries in large-scale search systems.
method Mutual Information Co-training framework for selective search with minimal supervision.
result Significantly improves performance on multiple metrics of selective search.
Differentiable NAS frameworks grow networks wider and deeper, revealing biases in wiring evolution.
problem Understanding the evolution of neural architecture wiring in differentiable NAS methods.
method Unified view on searching algorithms, local cost minimization, empirical and theoretical analyses.
result Implicit inductive biases cause observed searching patterns in differentiable NAS methods.
This paper improves keyword recommendation for sponsored search using deep reinforcement learning.
problem Selecting keywords from given candidates considering internal and external competitions.
method Solves the combinatorial optimization problem of keyword recommendations with a modified pointer network structure trained in a deep reinforcement learning framework.
result Remarkable improvements in performance observed both offline and online.
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Novel framework learns efficient student models from teacher networks.
problem Model capacity gap between teacher and student networks.
method Neural architecture search and oracle knowledge distillation.
result Searched student models often outperform teacher models.
A new multilabel classification framework improves ANN search performance.
problem Efficiently finding approximate nearest neighbors in large datasets.
method Formulated ANN search as a multilabel classification problem, using partitioning classifiers.
result Natural classifier leads to strictly improved performance in ANN search.
A new framework for selecting base classes in multi-class classification boosts accuracy.
problem Selecting the base class in multi-class classification to improve accuracy.
method Introduces a unified framework with parameters (s,g,w) to search for the base class at each boosting iteration, improving computational efficiency. result Our framework can achieve better test accuracy than the exhaustive search strategy, providing a robust and reliable scheme.
Unified framework for gradient-free MDS improves efficiency and accuracy.
problem Efficiently solving Multidimensional Scaling problems without derivatives.
method Bootstrapped Coordinate Search (BS CSMDS) for MDS, using a probability matrix to guide search.
result BS CSMDS achieves significant speedup and maintains error rate compared to other CSMDS methods.
Automates optimizer design for diverse tasks efficiently.
problem Scalability, generability, and sample efficiency in optimizer search.
method Re-arranged optimizer space into a super-tree, applying tree traversal methods.
result Discover optimizers with only 128 evaluations, surpassing human-designed and prior methods.
A framework benchmarks and dissects one-shot neural architecture search methods.
problem Understanding the dynamics and components of one-shot NAS methods.
method General framework for one-shot NAS, benchmarking with NAS-Bench-101.
result Comparison of one-shot NAS methods and their sensitivity to hyperparameters.
NASES uses embedding space for efficient NAS in image classification tasks.
problem Difficulty in optimizing high-dimensional discrete architecture spaces.
method NASES employs architecture encoders and decoders to search in an embedding space using reinforcement learning.
result NASES discovers comparable final architectures to other NAS approaches in less time.
Unified framework connects EI and information-theoretic acquisition functions.
problem Distinguish between Expected Improvement and information-theoretic acquisition functions.
method Introduces Variational Entropy Search (VES) to unify EI and information-theoretic approaches.
result EI can be seen as a variational inference approximation of Max-value Entropy Search (MES).
A new model detects objects more efficiently with less computation.
problem Handling waste of regional information and compromise between efficiency and accuracy in CNN object detection.
method Proposes a probabilistic model with a search framework to map images into probabilistic object distributions.
result The proposed model gives more informative outputs with less computation.
On most sponsored search platforms, advertisers bid on some keywords for their advertisements (ads). Given a search request, ad retrieval module rewrites the query into bidding keywords, and uses these keywords as keys to select Top N ads through inverted indexes. In this way, an ad will not be retrieved even if querie…
We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
We present a novel view that unifies two frameworks that aim to solve sequential prediction problems: learning to search (L2S) and recurrent neural networks (RNN). We point out equivalences between elements of the two frameworks. By complementing what is missing from one framework comparing to the other, we introduce a…
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
TGLS generates text by optimizing search results and learning from them.
problem Unsupervised text generation without explicit training data.
method TGLS uses simulated annealing to estimate sentence quality, then a conditional generative model learns from search results.
result TGLS outperforms unsupervised baselines and matches state-of-the-art supervised methods in paraphrase generation.
This work recommends personalized search stories to users based on their interests.
problem Personalized search story recommendation within search engines.
method Deep reinforcement learning architecture trained by imitation learning and reinforcement learning.
result Empirically demonstrated effectiveness on real-world data sets.
Weight-sharing is rarely helpful in NAS, especially on small datasets.
problem The efficiency of weight-sharing in Neural Architecture Search (NAS) is questionable.
method Comparison of a state-of-the-art weight-sharing approach to random search on the nasbench dataset.
result Weight-sharing is only rarely significantly helpful in NAS, highlighting the importance of the search space.
Paper bounds NAS function approximation, showing computational limits.
problem NAS function approximation problem.
method Reformulated FA problem, showed computational infeasibility, described a-ASP.
result NAS cannot solve FA for all functions to zero error.
Paper proposes a new evaluation method for NAS search phase.
problem NAS search phase effectiveness not well evaluated.
method Compare NAS solutions with random selection; evaluate weight sharing strategy.
result State-of-the-art NAS algorithms perform similarly to random selection.
AI agents improve forecast combination but require transparency.
problem AI coding agents increase flexibility in empirical economics, leading to hidden degrees of freedom.
method Adapted open-source agent-loop architecture to empirical economics workflow, adding post-search holdout evaluation.
result Multiple agent runs outperform standard benchmarks in rolling evaluation but not all on post-search holdout.
Tune simplifies hyperparameter search for distributed computing.
problem Adapting hyperparameter search to distributed environments.
method Unified framework for model selection and training.
result Simplifies implementation of state-of-the-art search algorithms.
Paper proposes algorithms for sparse signal estimation with nonconvex regularization.
problem Sparse signal estimation with nonconvex regularization.
method Successive convex approximation framework combining majorization-minimization and line search.
result Flexibility, fast convergence, low complexity, guaranteed convergence to stationary point.
Proposes a new framework for predicting stock market movements using sparse neural architectures.
problem Challenging problem of predicting stock market movements using technical indicators.
method Multi-criteria optimization approach to evolve sparse neural architectures.
result Evolved parsimonious networks with better generalization capabilities.