Statistical guarantees for hyperparameter selection
problem Hyperparameter selection in AI systems
method Learn-then-test framework
result Provable reliability and safety
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Hyperparameter Selection”
New method selects best offline RL policies from logged data.
problem Hyperparameter selection challenges offline RL.
method Offline hyperparameter selection for RL algorithms.
result Reliable ranking and selection of policies across hyperparameters.
AHS framework selects hyperparameters for FQE with error guarantees.
problem Hyperparameter selection for FQE is challenging and affects its utility.
method AHS framework defines optimality criteria without hyperparameters.
result Error bounds match empirical observations.
Paper tackles policy selection in offline RL without hyperparameters.
problem Selecting between policies and value functions in offline RL.
method Designs hyperparameter-free algorithms based on BVFT for policy selection.
result Demonstrates effectiveness in discrete-action benchmarks like Atari.
Develops a method to optimize hyperparameters for subsampling methods.
problem Optimizing hyperparameters for subsampling methods to improve estimator efficiency.
method Careful theoretical analysis leading to an optimal choice of hyperparameters.
result Improves the statistical efficiency of subsampling estimators without extra CPU time.
New optimizer improves privacy-protected hyperparameter tuning.
problem No practical methods for differentially private hyperparameter selection.
method Study honest hyperparameter selection under DP, show adaptive optimizers like DPAdam have an advantage.
result DPAdam optimizes hyperparameters more efficiently under DP constraints.
Efficient and Robust Bayesian Selection of Hyperparameters in Dimension Reduction for Visualizationstat.ML
Efficient auto-tuning for DR hyperparameters with BO.
problem Hyperparameter selection in DR for large-scale datasets.
method Bayesian optimization with surrogate model, normalization, subsampling.
result Robust and efficient hyperparameter selection for DR algorithms.
Both feature selection and hyperparameter tuning are key tasks in machine learning. Hyperparameter tuning is often useful to increase model performance, while feature selection is undertaken to attain sparse models. Sparsity may yield better model interpretability and lower cost of data acquisition, data handling and m…
LLMs learn to recommend models and hyperparameters from dataset metadata.
problem Model and hyperparameter selection in machine learning is challenging and resource-intensive.
method Converted datasets into metadata and prompted LLMs to recommend models and hyperparameters.
result LLMs can recommend competitive models and hyperparameters without search.
Random forest hyperparameters affect variable selection in omics studies.
problem Impact of hyperparameters on variable selection in random forests.
method Two simulation studies using theoretical and empirical data.
result Hyperparameters influence variable selection more than the splitting strategy and sample fraction.
Nested cross-validation when selecting classifiers is overzealous for most practical applicationscs.LG
When selecting a classification algorithm to be applied to a particular problem, one has to simultaneously select the best algorithm for that dataset \emph{and} the best set of hyperparameters for the chosen model. The usual approach is to apply a nested cross-validation procedure; hyperparameter selection is performed…
The success of machine learning on a given task dependson, among other things, which learning algorithm is selected and its associated hyperparameters. Selecting an appropriate learning algorithm and setting its hyperparameters for a given data set can be a challenging task, especially for users who are not experts in …
We propose a new algorithm for hyperparameter selection in machine learning algorithms. The algorithm is a novel modification of Harmonica, a spectral hyperparameter selection approach using sparse recovery methods. In particular, we show that a special encoding of hyperparameter space enables a natural group-sparse re…
aLTT selects hyperparameters efficiently with statistical guarantees.
problem Statistical validity and efficiency in hyperparameter selection.
method Sequential data-dependent multiple hypothesis testing with early termination.
result Reduces testing rounds while maintaining statistical validity.
A new framework tackles CASH problem with alternating optimization and Rising Bandits.
problem Efficiently solving the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem.
method Alternating optimization framework using BO for HPO and Rising Bandits for algorithm selection.
result Demonstrated superiority over competitive baselines in extensive experiments.
Empirical comparison of 18 hyperparameter tuning algorithms for SVM.
problem Tuning hyperparameters and for SVM with RBF kernel.
method Compared 18 search algorithms on 115 real-life data sets.
result Trees of Parzen estimators and particle swarm optimization perform similarly to grid search.
A two-step approach efficiently selects hyperparameters for FCMs.
problem Efficiently selecting hyperparameters for FCMs in a computationally expensive process.
method Two-step sequential approach: first estimate context length k, then estimate α.
result The proposed method achieves comparable compression performance to exhaustive search but with reduced computational cost.
Proposes FMS for more efficient neural network hyperparameter optimization.
problem Efficient hyperparameter optimization for deep learning models.
method Uses logged checkpoints of trained weights to guide hyperparameter selections.
result Proposes Forecasting Model Search (FMS) method.
New method optimizes hyperparameters in deep learning models efficiently.
problem Manual hyperparameter tuning in deep learning models is inefficient and requires expertise.
method Introduces lower bounds to the linearized Laplace approximation of the marginal likelihood using neural tangent kernels.
result Optimization of hyperparameters can be significantly accelerated using the method.
Transfer learning improves prediction quality in high-dimensional sparse regression.
problem Selecting hyperparameters for Lasso-based transfer learning algorithms.
method Asymptotic analysis using the replica method.
result Ignoring one type of transferred information has minimal impact on performance.
Many algorithms for data analysis exist, especially for classification problems. To solve a data analysis problem, a proper algorithm should be chosen, and also its hyperparameters should be selected. In this paper, we present a new method for the simultaneous selection of an algorithm and its hyperparameters. In order…
GP-TS optimizes TLM pre-training hyperparameters efficiently.
problem Resource inefficiency in TLM pre-training.
method Bayesian optimization with Thompson sampling and Gaussian process.
result GP-TS achieves lower MLM loss in fewer epochs.
MementoML: Performance of selected machine learning algorithm configurations on OpenML100 datasetscs.LG
Paper compiles ML algorithm performance benchmarks on OpenML datasets.
problem Finding optimal hyperparameters for ML algorithms efficiently.
method Generated benchmark data for 7 ML algorithms on 39 datasets, fixed hyperparameters before testing.
result Comprehensive dataset of ML algorithm performance sensitivity.
Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retrainingcs.LG
Study shows data attribution methods are sensitive to hyperparameters, making tuning costly.
problem Hyperparameter sensitivity in data attribution methods makes tuning impractical.
method Theoretical analysis and lightweight procedure for selecting regularization value without retraining.
result Proposes a lightweight procedure for selecting regularization value without model retraining.
Auto-CASH uses Deep Q-Network to automatically select machine learning algorithms.
problem Automatic selection of machine learning algorithms and hyperparameters.
method Pre-trained model based on meta-learning using Deep Q-Network.
result Auto-CASH achieves better performance with shorter time compared to classical and state-of-the-art methods.
This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.
problem Optimizing hyperparameters of non-smooth convex models.
method Implicit differentiation of proximal gradient and coordinate descent methods.
result Implicit differentiation can speed up hyperparameter optimization, especially for non-smooth problems.
Hyperparameter tuning is an omnipresent problem in machine learning as it is an integral aspect of obtaining the state-of-the-art performance for any model. Most often, hyperparameters are optimized just by training a model on a grid of possible hyperparameter values and taking the one that performs best on a validatio…
BOOST automates kernel and acquisition function selection in Bayesian optimization.
problem Inappropriate kernel and acquisition function combinations lead to poor performance in Bayesian optimization.
method BOOST uses offline evaluation to predict and select the best kernel-acquisition function pair.
result BOOST consistently improves over fixed-hyperparameter BO and is competitive with state-of-the-art adaptive methods.
LMs perform poorly in true few-shot learning without held-out examples.
problem Evaluating few-shot performance of language models without access to held-out examples.
method Evaluated two model selection criteria (cross-validation and minimum description length) for choosing LM prompts and hyperparameters in true few-shot learning.
result Selection criteria often prefer models that perform worse than random selection, suggesting overestimation of few-shot ability.
Pipeline selects best RL policy from limited data.
problem Selecting best RL policy from small datasets.
method Task- and method-agnostic pipeline using multiple data splits.
result Pipeline produces higher-performing policies.
New algorithms for model selection in off-policy evaluation of reinforcement learning.
problem Hyperparameter tuning for off-policy evaluation methods in reinforcement learning.
method Developed new model-free and model-based selectors with theoretical guarantees and a new experimental protocol.
result New model-free selector, LSTD-Tournament, demonstrates promising empirical performance.
Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have been proposed for improving the efficiency of model selection, however their adapta…
Many deep models have been recently proposed for anomaly detection. This paper presents comparison of selected generative deep models and classical anomaly detection methods on an extensive number of non--image benchmark datasets. We provide statistical comparison of the selected models, in many configurations, archite…
Global sensitivity analysis improves BNN hyperparameter selection for accurate uncertainty quantification.
problem Difficulties in obtaining accurate uncertainty quantification with Bayesian Neural Networks (BNNs).
method Global sensitivity analysis of BNN performance under varying hyperparameter settings.
result Many hyperparameters interact to affect both predictive accuracy and uncertainty quantification.
New method estimates marginal likelihood for deep learning models using training data alone.
problem Estimation difficulties in marginal likelihood for model selection in deep learning.
method Scalable marginal likelihood estimation based on Laplace's method and Gauss-Newton approximations.
result Estimate outperforms cross-validation and manual tuning on various datasets.
Selecting hyperparameters for unsupervised learning problems is challenging in general due to the lack of ground truth for validation. Despite the prevalence of this issue in statistics and machine learning, especially in clustering problems, there are not many methods for tuning these hyperparameters with theoretical …
A new IL framework estimates invariant predictors with single domain data.
problem Deep networks inherit spurious correlations and fail on unseen domains.
method Assumes multiple labeled domains for higher-level tasks, uses single domain for target task, employs cross-validation for hyperparameter selection.
result Empirically demonstrates effectiveness and correctness of hyperparameter selection.
A new principle for optimizer selection improves training speed and performance.
problem Finding the best optimizer hyperparameters for faster training.
method Formulate optimizer selection as maximizing the expected drop rate in loss, treating gradients and updates as signals and an optimizer as a causal filter.
result Greedy optimizer selection yields stable and effective momentum rules.
Auto-Surprise automates recommender system selection and optimization.
problem Finding the best algorithm and hyperparameters for recommender systems.
method Extends Surprise library with TPE optimization for algorithm selection and hyperparameter tuning.
result Significantly faster in finding optimal hyperparameters compared to grid search.
Adaptive speculative decoding framework for LLMs using bandit algorithms.
problem Adaptive speculative decoding for LLMs to balance speed and quality.
method Formulated as a Multi-Armed Bandit problem, proposed UCBSpec and EXP3Spec algorithms.
result UCBSpec algorithm achieves optimal regret performance up to universal constants.
A new RNN architecture reduces model size and improves performance.
problem Overparameterization and resource limitations in RNNs.
method Jointly encodes weight matrices using tensor-train factorization.
result Reduces model size by several orders of magnitude without sacrificing performance.
We present a new method for searching optimal hyperparameters among several tasks and several criteria. Multi-Task Multi Criteria method (MTMC) provides several Pareto-optimal solutions, among which one solution is selected with given criteria significance coefficients. The article begins with a mathematical formulatio…
Revisits online Laplace methods for neural networks, showing they are sound under certain conditions.
problem Online Laplace methods violate the Laplace approximation's critical assumption.
method Re-derives online Laplace methods, showing they target a variational bound on a mode-corrected variant of the Laplace evidence.
result Online Laplace and its mode-corrected counterpart share stationary points that satisfy the Laplace method's assumption.
The behavior of many Bayesian models used in machine learning critically depends on the choice of prior distributions, controlled by some hyperparameters that are typically selected by Bayesian optimization or cross-validation. This requires repeated, costly, posterior inference. We provide an alternative for selecting…
Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this work, we extend the span error bound theory to weighted SVM and we introduce effective hyperparamete…
Paper proposes an efficient bandit-based algorithm for hyperparameter optimization.
problem Efficiently evaluating hyperparameters in deep learning models with large search spaces.
method Sub-Sampling (SS) algorithm combined with Bayesian Optimization (BOSS).
result Theoretical proof of optimality and empirical validation of superior performance.
Convolutional Neural Networks and Deep Learning classification systems in general have been shown to be vulnerable to attack by specially crafted data samples that appear to belong to one class but are instead classified as another, commonly known as adversarial examples. A variety of attack strategies have been propos…
Novel approach simplifies VI problems with faster performance.
problem Black-box VI optimization problems.
method Sample Average Approximation (SAA) combined with quasi-Newton methods and line search.
result Achieves faster performance than existing methods.