A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new Randomized-Hyperopt method improves XGBoost hyperparameter tuning.
In this paper, we compare the three most popular algorithms for hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm) and attempt to use them for neural architecture search (NAS). We use these algorithms for building a convolutional neural network (search architecture). Experimental results on…
Context: One of the black arts of data mining is learning the magic parameters which control the learners. In software analytics, at least for defect prediction, several methods, like grid search and differential evolution (DE), have been proposed to learn these parameters, which has been proved to be able to improve t…
Inexact acquisition solutions in BO lead to sublinear cumulative regret.
SMAC method optimizes tree-boosting hyperparameters best.
Empirical comparison of 18 hyperparameter tuning algorithms for SVM.
This paper reviews hyperparameter optimization methods and best practices.
Driven by the need for parallelizable hyperparameter optimization methods, this paper studies \emph{open loop} search methods: sequences that are predetermined and can be generated before a single configuration is evaluated. Examples include grid search, uniform random search, low discrepancy sequences, and other sampl…
In order to find hyperparameters for a machine learning model, algorithms such as grid search or random search are used over the space of possible values of the models hyperparameters. These search algorithms opt the solution that minimizes a specific cost function. In language models, perplexity is one of the most pop…
Approximate nearest neighbor algorithms are used to speed up nearest neighbor search in a wide array of applications. However, current indexing methods feature several hyperparameters that need to be tuned to reach an acceptable accuracy--speed trade-off. A grid search in the parameter space is often impractically slow…
Proposes learning regularization strength directly from data.
Space-filling designs such as scrambled-Hammersley, Latin Hypercube Sampling and Jittered Sampling have been proposed for fully parallel hyperparameter search, and were shown to be more effective than random or grid search. In this paper, we show that these designs only improve over random search by a constant factor. …
Hyperparameter optimization aims to find the optimal hyperparameter configuration of a machine learning model, which provides the best performance on a validation dataset. Manual search usually leads to get stuck in a local hyperparameter configuration, and heavily depends on human intuition and experience. A simple al…
Bayesian optimization improves classifier selection for acute infection and mortality.
The paper develops a stationary-distribution theory for Random Forest ensemble size selection.
Model classifies hand gestures with low EMG samples augmented with noise.
Determining the appropriate batch size for mini-batch gradient descent is always time consuming as it often relies on grid search. This paper considers a resizable mini-batch gradient descent (RMGD) algorithm based on a multi-armed bandit for achieving best performance in grid search by selecting an appropriate batch s…
Neural sequence-to-sequence models, particularly the Transformer, are the state of the art in machine translation. Yet these neural networks are very sensitive to architecture and hyperparameter settings. Optimizing these settings by grid or random search is computationally expensive because it requires many training r…
Learning rate annealing improves robustness in stochastic optimization.
Deep-n-Cheap automates deep learning model search for low complexity.
New method selects critical DER scenarios for distribution grid investment planning.
Paper reviews and compares methods for handling imbalanced data.
The paper examines linking numbers in grid models and finds polynomial moments.
Proposes a gradient-based bilevel optimization method for efficient hyperparameter tuning.
A hybrid ML model detects fraudulent transactions with high accuracy.
A new approach models exploration in continuous-time RL using random measures.
Systems based on artificial neural networks (ANNs) have achieved state-of-the-art results in many natural language processing tasks. Although ANNs do not require manually engineered features, ANNs have many hyperparameters to be optimized. The choice of hyperparameters significantly impacts models' performances. Howeve…
Tuning SVM and boosting models using optimization algorithms.
A new method uses SVM classification to efficiently compute confidence sets.
A new model framework called Realized Conditional Autoregressive Expectile (Realized-CARE) is proposed, through incorporating a measurement equation into the conventional CARE model, in a manner analogous to the Realized-GARCH model. Competing realized measures (e.g. Realized Variance and Realized Range) are employed a…
New method constructs moduli spaces of Lagrangian surfaces in CP^2 from grid diagrams.
New model solves PDEs using probabilistic random grids.
In recent years there have been many deep learning approaches towards the multi-speaker source separation problem. Most use Long Short-Term Memory - Recurrent Neural Networks (LSTM-RNN) or Convolutional Neural Networks (CNN) to model the sequential behavior of speech. In this paper we propose a novel network for source…
This work improves Bayesian Optimization for setting DNN hyper-parameters.
Paper studies kernel hyperparameters for clustering, proposing an efficient search method.
The study models insurance dependence using Bernstein copulas.
This work simplifies SVM parameter selection using S&S ratio.
Study on typical knots and links using grid diagrams, focusing on size, components, and writhe.
New high-order approximations for CIR process using random grids.
We consider the symmetric exclusion process on suitable random grids that approximate a compact Riemannian manifold. We prove that a class of random walks on these random grids converge to Brownian motion on the manifold. We then consider the empirical density field of the symmetric exclusion process and prove that it …
Scalable algorithm for sampling Gaussian processes using sparse grids and preconditioners.
Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The algorithm results may fluctuate dramatically under the different configuration of hyper-parameters. A…
Efficient search methods can outperform random search on challenging tasks.
Proposes a neural network for calibrating stochastic volatility models.
Setting regularization parameters for Lasso-type estimators is notoriously difficult, though crucial in practice. The most popular hyperparameter optimization approach is grid-search using held-out validation data. Grid-search however requires to choose a predefined grid for each parameter, which scales exponentially i…
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …
Modern smart grids rely on advanced metering infrastructure (AMI) networks for monitoring and billing purposes. However, such an approach suffers from electricity theft cyberattacks. Different from the existing research that utilizes shallow, static, and customer-specific-based electricity theft detectors, this paper p…
Neural architecture search (NAS) is a promising research direction that has the potential to replace expert-designed networks with learned, task-specific architectures. In this work, in order to help ground the empirical results in this field, we propose new NAS baselines that build off the following observations: (i) …