A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Survey of techniques for automating hyperparameter optimization in machine learning.
Empirical comparison of 18 hyperparameter tuning algorithms for SVM.
Paper proposes HCDC to improve hyperparameter search efficiency.
Paper proposes an efficient bandit-based algorithm for hyperparameter optimization.
WRS improves CNN hyperparameter optimization.
Asynchronous method for hyperparameter and neural architecture search.
Improved Random Search for hyperparameter optimization.
Driven by the need for parallelizable hyperparameter optimization methods, this paper studies \emph{open loop} search methods: sequences that are predetermined and can be generated before a single configuration is evaluated. Examples include grid search, uniform random search, low discrepancy sequences, and other sampl…
We introduce the hyperparameter search problem in the field of machine learning and discuss its main challenges from an optimization perspective. Machine learning methods attempt to build models that capture some element of interest based on given data. Most common learning algorithms feature a set of hyperparameters t…
New method optimizes hyperparameters for non-smooth problems efficiently.
New tools reveal simple structure in complex hyperparameter loss surfaces near optima.
SMAC method optimizes tree-boosting hyperparameters best.
This paper reviews hyperparameter optimization methods and best practices.
We introduce an open source python framework named PHS - Parallel Hyperparameter Search to enable hyperparameter optimization on numerous compute instances of any arbitrary python function. This is achieved with minimal modifications inside the target function. Possible applications appear in expensive to evaluate nume…
A new Randomized-Hyperopt method improves XGBoost hyperparameter tuning.
While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few …
We propose a new algorithm for hyperparameter selection in machine learning algorithms. The algorithm is a novel modification of Harmonica, a spectral hyperparameter selection approach using sparse recovery methods. In particular, we show that a special encoding of hyperparameter space enables a natural group-sparse re…
Paper studies kernel hyperparameters for clustering, proposing an efficient search method.
Differentially private hyperparameter tuning improves privacy in machine learning.
In this paper, we propose the use of a black-box optimization method called deterministic Mesh Adaptive Direct Search (MADS) algorithm with orthogonal directions (Ortho-MADS) for the selection of hyperparameters of Support Vector Machines with a Gaussian kernel. Different from most of the methods in the literature that…
Improves neural architecture search methods to be more stable and efficient.
Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have been proposed for improving the efficiency of model selection, however their adapta…
Space-filling designs such as scrambled-Hammersley, Latin Hypercube Sampling and Jittered Sampling have been proposed for fully parallel hyperparameter search, and were shown to be more effective than random or grid search. In this paper, we show that these designs only improve over random search by a constant factor. …
Convolutional Neural Network is known as ConvNet have been extensively used in many complex machine learning tasks. However, hyperparameters optimization is one of a crucial step in developing ConvNet architectures, since the accuracy and performance are reliant on the hyperparameters. This multilayered architecture pa…
In deep learning, performance is strongly affected by the choice of architecture and hyperparameters. While there has been extensive work on automatic hyperparameter optimization for simple spaces, complex spaces such as the space of deep architectures remain largely unexplored. As a result, the choice of architecture …
Automatically searching for optimal hyperparameter configurations is of crucial importance for applying deep learning algorithms in practice. Recently, Bayesian optimization has been proposed for optimizing hyperparameters of various machine learning algorithms. Those methods adopt probabilistic surrogate models like G…
Hyperparameter optimization aims to find the optimal hyperparameter configuration of a machine learning model, which provides the best performance on a validation dataset. Manual search usually leads to get stuck in a local hyperparameter configuration, and heavily depends on human intuition and experience. A simple al…
Proposes a gradient-based bilevel optimization method for efficient hyperparameter tuning.
Proposes baselines for joint NAS and HPO optimization.
In order to find hyperparameters for a machine learning model, algorithms such as grid search or random search are used over the space of possible values of the models hyperparameters. These search algorithms opt the solution that minimizes a specific cost function. In language models, perplexity is one of the most pop…
HASSO improves SO algorithms by dynamically tuning hyperparameters.
Autodock is a widely used molecular modeling tool which predicts how small molecules bind to a receptor of known 3D structure. The current version of AutoDock uses meta-heuristic algorithms in combination with local search methods for doing the conformation search. Appropriate settings of hyperparameters in these algor…
Mango automates hyperparameter tuning for large-scale ML training.
Approximate nearest neighbor algorithms are used to speed up nearest neighbor search in a wide array of applications. However, current indexing methods feature several hyperparameters that need to be tuned to reach an acceptable accuracy--speed trade-off. A grid search in the parameter space is often impractically slow…
This paper explores BDL hyperparameters for robust polynomial mapping with noise.
Efficiently identifies promising hyperparameters for online learning models.
Bayesian optimization outperformed random search in machine learning hyperparameter tuning challenge.
LLMs learn to recommend models and hyperparameters from dataset metadata.
Systems based on artificial neural networks (ANNs) have achieved state-of-the-art results in many natural language processing tasks. Although ANNs do not require manually engineered features, ANNs have many hyperparameters to be optimized. The choice of hyperparameters significantly impacts models' performances. Howeve…
A new method reduces hyperparameter tuning evaluations by using sequential tests.
BLiE optimizes hyperparameters with theoretical guarantees and superior performance.
Bayesian optimization outperforms other methods in hyperparameter tuning for reinforcement learning.
Proposes FMS for more efficient neural network hyperparameter optimization.
We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified…
The paper proposes a method to learn hyperparameters without validation sets, improving efficiency and accuracy.
Study uses multi-task Bayesian optimization to speed up SVM hyperparameter tuning for nodules diagnosis.
VisEvol uses evolutionary optimization to find optimal hyperparameters for machine learning models.
We present TaskSet, a dataset of tasks for use in training and evaluating optimizers. TaskSet is unique in its size and diversity, containing over a thousand tasks ranging from image classification with fully connected or convolutional neural networks, to variational autoencoders, to non-volume preserving flows on a va…