A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This paper reviews hyperparameter optimization methods and best practices.
problem Finding optimal hyperparameters for machine learning models.
method Various hyperparameter optimization methods are reviewed, including grid search, random search, evolutionary algorithms, Bayesian optimization, Hyperband, and racing.
result Practical recommendations for conducting hyperparameter optimization are provided.
Bayesian Optimization (BO) is a common approach for hyperparameter optimization (HPO) in automated machine learning. Although it is well-accepted that HPO is crucial to obtain well-performing machine learning models, tuning BO's own hyperparameters is often neglected. In this paper, we empirically study the impact of o…
Automated machine learning aims to automate the whole process of machine learning, including model configuration. In this paper, we focus on automated hyperparameter optimization (HPO) based on sequential model-based optimization (SMBO). Though conventional SMBO algorithms work well when abundant HPO trials are availab…
Despite the recent progress in hyperparameter optimization (HPO), available benchmarks that resemble real-world scenarios consist of a few and very large problem instances that are expensive to solve. This blocks researchers and practitioners not only from systematically running large-scale comparisons that are needed …
HyperSTAR optimizes deep network hyperparameters for specific tasks.
problem Finding optimal hyperparameters for deep neural networks is computationally expensive and task-agnostic approaches are inefficient.
method HyperSTAR learns a dataset representation and a performance predictor from raw images, ranking and recommending hyperparameters conditioned on a joint dataset-hyperparameter space.
result HyperSTAR reduces HPO time by 50% on large-scale image classification datasets.
Tuning machine learning models at scale, especially finding the right hyperparameter values, can be difficult and time-consuming. In addition to the computational effort required, this process also requires some ancillary efforts including engineering tasks (e.g., job scheduling) as well as more mundane tasks (e.g., ke…
Paper develops an efficient approach to reduce HPO time.
problem Challenges in determining optimal hyperparameters due to large number and training time.
method Nested Latin hypercube design for initialization, truncated additive Gaussian process model for calibration, sequential model-based algorithm for optimization.
result Demonstrates competitive performance on various machine learning models.
In recent years, hyperparameter optimization (HPO) has become an increasingly important issue in the field of machine learning for the development of more accurate forecasting models. In this study, we explore the potential of HPO in modeling stock returns using a deep neural network (DNN). The potential of this approa…
Due to the high computational demands executing a rigorous comparison between hyperparameter optimization (HPO) methods is often cumbersome. The goal of this paper is to facilitate a better empirical evaluation of HPO methods by providing benchmarks that are cheap to evaluate, but still represent realistic use cases. W…
Decoupled PFNs improve sequential decision-making by separating epistemic and aleatoric uncertainties.
problem Sequential decision-making requires distinguishing between epistemic uncertainty about latent signals and irreducible aleatoric observation noise.
method Developed a decoupled PFN architecture that uses query-level labels to train separate heads for latent signal and aleatoric noise.
result Empirically, decoupled PFNs mitigate the failure mode of total-variance exploration in noisy and heteroscedastic settings.
PyKEEN 1.0 simplifies KGE model creation and optimization.
problem Training and evaluating knowledge graph embeddings (KGEs).
method Composes KGEMs with various interaction models, training approaches, and loss functions. Implements automatic memory optimization and extensive HPO functionalities.
result PyKEEN 1.0 streamlines KGE model creation and optimization.
BiSHop tackles tabular data challenges with sparse Hopfield layers.
problem Non-rotationally invariant data structure and feature sparsity in tabular data.
method Sequential column-wise and row-wise processing through interconnected directional learning modules with generalized sparse modern Hopfield layers.
result BiSHop surpasses current SOTA methods with significantly less hyperparameter tuning.