A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Global sensitivity analysis improves BNN hyperparameter selection for accurate uncertainty quantification.
A game-theoretic framework identifies influential hyperparameters for neural networks.
Method analyzes hyperparameters using HSIC for better neural network performance.
Study shows data attribution methods are sensitive to hyperparameters, making tuning costly.
Symbolic LSTM improves time series forecasting by reducing hyperparameter sensitivity.
Random forest hyperparameters affect variable selection in omics studies.
New method optimizes hyperparameters for randomized algorithms like random feature regression.
New benchmark protocol evaluates neural network optimizers for efficiency and data shift sensitivity.
slimTrain simplifies DNN training by separating features and adapting hyperparameters.
Automates RL with sample-efficient hyperparameter optimization.
Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradi…
Efficient auto-tuning for DR hyperparameters with BO.
Machine learning algorithms are very sensitive to the hyperparameters, and their evaluations are generally expensive. Users desperately need intelligent methods to quickly optimize hyperparameter settings according to known evaluation information, and thus reduce computational cost and promote optimization efficiency. …
The performance of policy gradient methods is sensitive to hyperparameter settings that must be tuned for any new application. Widely used grid search methods for tuning hyperparameters are sample inefficient and computationally expensive. More advanced methods like Population Based Training that learn optimal schedule…
Differentially private hyperparameter tuning improves privacy in machine learning.
Knowledge graph embeddings rank among the most successful methods for link prediction in knowledge graphs, i.e., the task of completing an incomplete collection of relational facts. A downside of these models is their strong sensitivity to model hyperparameters, in particular regularizers, which have to be extensively …
Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinf…
Field theory explains optimal scaling in ResNets for signal propagation.
Adversarial training shows promise as an approach for training models that are robust towards adversarial perturbation. In this paper, we explore some of the practical challenges of adversarial training. We present a sensitivity analysis that illustrates that the effectiveness of adversarial training hinges on the sett…
Improves neural architecture search methods to be more stable and efficient.
This paper re-evaluates hyperparameters for fine-tuning pre-trained models.
Working with any gradient-based machine learning algorithm involves the tedious task of tuning the optimizer's hyperparameters, such as its step size. Recent work has shown how the step size can itself be optimized alongside the model parameters by manually deriving expressions for "hypergradients" ahead of time. We sh…
New method upscales models and transfers hyperparameters efficiently.
Paper compiles ML algorithm performance benchmarks on OpenML datasets.
Modern deep learning methods are very sensitive to many hyperparameters, and, due to the long training times of state-of-the-art models, vanilla Bayesian hyperparameter optimization is typically computationally infeasible. On the other hand, bandit-based configuration evaluation approaches based on random search lack g…
Selecting an optimizer is a central step in the contemporary deep learning pipeline. In this paper, we demonstrate the sensitivity of optimizer comparisons to the hyperparameter tuning protocol. Our findings suggest that the hyperparameter search space may be the single most important factor explaining the rankings obt…
The performance of deep neural networks is highly sensitive to the choice of the hyperparameters that define the structure of the network and the learning process. When facing a new application, tuning a deep neural network is a tedious and time consuming process that is often described as a "dark art". This explains t…
Online Passive-Aggressive (PA) learning is a class of online margin-based algorithms suitable for a wide range of real-time prediction tasks, including classification and regression. PA algorithms are formulated in terms of deterministic point-estimation problems governed by a set of user-defined hyperparameters: the a…
Self-Tuning Actor-Critic improves reinforcement learning performance.
The performance of deep neural networks (DNN) is very sensitive to the particular choice of hyper-parameters. To make it worse, the shape of the learning curve can be significantly affected when a technique like batchnorm is used. As a result, hyperparameter optimization of deep networks can be much more challenging th…
We present a novel algorithm to train a deep Q-learning agent using natural-gradient techniques. We compare the original deep Q-network (DQN) algorithm to its natural-gradient counterpart, which we refer to as NGDQN, on a collection of classic control domains. Without employing target networks, NGDQN significantly outp…
Domain adaptation provides a powerful set of model training techniques given domain-specific training data and supplemental data with unknown relevance. The techniques are useful when users need to develop models with data from varying sources, of varying quality, or from different time ranges. We build CrossTrainer, a…
A two-step approach efficiently selects hyperparameters for FCMs.
A new survival analysis method eliminates hyperparameter tuning.
The behavior of many Bayesian models used in machine learning critically depends on the choice of prior distributions, controlled by some hyperparameters that are typically selected by Bayesian optimization or cross-validation. This requires repeated, costly, posterior inference. We provide an alternative for selecting…
Machine learning has witnessed tremendous success in solving tasks depending on a single hyperparameter. When considering simultaneously a finite number of tasks, multi-task learning enables one to account for the similarities of the tasks via appropriate regularizers. A step further consists of learning a continuum of…
A new method uses Gaussian Processes to solve power flow problems with uncertain renewable and load inputs.
This study evaluates different normalizing flow architectures for MCMC.
We develop a general variational inference method that preserves dependency among the latent variables. Our method uses copulas to augment the families of distributions used in mean-field and structured approximations. Copulas model the dependency that is not captured by the original variational distribution, and thus …
This paper enhances the Random Survival Forest model for better predictive maintenance.
A new stochastic method tackles bi-level optimization problems in deep learning.
A simple self-supervised model for tensor RPCA using deep unfolding.
New algorithm improves stability of optimization algorithms by adapting step-size.
Bayesian evidence helps compare models but can overfit.
New framework explains fast transfer of hyperparameters across model scales.
Unified GP model optimizes hyperparameters with conditional dependence.
In recent years, active subspace methods (ASMs) have become a popular means of performing subspace sensitivity analysis on black-box functions. Naively applied, however, ASMs require gradient evaluations of the target function. In the event of noisy, expensive, or stochastic simulators, evaluating gradients via finite …
New Muon and Momo variants improve neural network optimization robustness.