Evolutionary Strategies optimize hyper-parameters for off-policy learning.
problem Hyper-parameter sensitivity in off-policy learning.
method Application of Evolutionary Strategies for online hyper-parameter tuning.
result Our method outperforms state-of-the-art baselines.
Study benchmarks RL algorithms on real robots, revealing their performance and hyper-parameter sensitivity.
problem Lack of benchmark tasks and source code for reinforcement learning on physical robots.
method Introduced benchmark tasks with multiple robots, tested 4 RL algorithms, analyzed hyper-parameter sensitivity.
result Some RL implementations can be applied to physical robots with proper setup, but hyper-parameters need re-tuning.
This paper proposes an efficient autoHPO method based on data-to-hyper-parameter mapping.
problem Manual hyper-parameter tuning is costly and dependent.
method The approach is based on mapping from data to hyper-parameters using a sophisticated network structure and effective construction algorithms.
result The proposed approach significantly outperforms state-of-the-art methods.
SGE-Kriging reduces high-dimensional surrogate modelling costs.
problem High-dimensional function approximation for expensive models.
method Splitting training data into slices, using sliced likelihood function, and learning hyper-parameters from sensitivity indices.
result SGE-Kriging achieves comparable accuracy to standard GE-Kriging but with lower training costs.
New method improves anomaly detection in acoustic signals.
problem Poor anomaly detection performance in existing acoustic signal-based unsupervised methods.
method Deep autoencoding Gaussian mixture model with hyper-parameter optimization.
result Significantly improved anomaly detection performance compared to previous methods.
An online algorithm improves cPCA for efficient, interpretable data analysis.
problem Efficiently finding informative low-dimensional representations in large datasets.
method Developed an online algorithm for a modified cPCA* method.
result The online algorithm for cPCA* shows improved performance and interpretability.
Symbolic LSTM improves time series forecasting by reducing hyperparameter sensitivity.
problem High sensitivity to hyperparameters and random initialization in numerical time series forecasting.
method Combining LSTM with a dimension-reducing symbolic representation.
result Symbolic representation alleviates forecasting problems and speeds up training.
Bayesian optimization is a powerful tool for fine-tuning the hyper-parameters of a wide variety of machine learning models. The success of machine learning has led practitioners in diverse real-world settings to learn classifiers for practical problems. As machine learning becomes commonplace, Bayesian optimization bec…
Recurrent neural networks are a powerful tool, but they are very sensitive to their hyper-parameter configuration. Moreover, training properly a recurrent neural network is a tough task, therefore selecting an appropriate configuration is critical. Varied strategies have been proposed to tackle this issue. However, mos…
This paper proposes an automatic neural network compression method.
problem Reducing resource requirements for deep neural networks on resource-constrained devices.
method Jointly prunes and quantizes neural networks without manual hyper-parameter tuning.
result Significant reduction in model size with minimal accuracy loss.
Environmental acoustic sensing involves the retrieval and processing of audio signals to better understand our surroundings. While large-scale acoustic data make manual analysis infeasible, they provide a suitable playground for machine learning approaches. Most existing machine learning techniques developed for enviro…
This work examines aggregation functions in Deep Set learning.
problem The sensitivity of Deep Set networks to aggregation function choices.
method Investigation of alternative aggregation functions, including learnable recurrent ones.
result Learnable aggregations improve performance, reduce hyper-parameter sensitivity, and generalize better.
FROST speeds up and stabilizes one-shot semi-supervised learning.
problem Slow training and sensitivity to labeled data choices in semi-supervised learning.
method Combines semi-supervised learning with a one-stage, single network self-training approach.
result FROST trains up to 10x faster and is more robust to labeled data choices.
Proposes debiasing strategy for ill-posed regression problems.
problem Estimating functions with conditional moment restrictions, especially when estimators are sensitive to misspecification.
method Debiased estimation using influence function of modified mean squared error.
result Demonstrates finite-sample convergence rate and robustness to misspecification.
Stage-based hyper-parameter optimization reduces GPU-hours and training time.
problem Efficiently executing hyper-parameter optimization for deep learning models.
method Stage-based execution strategy to remove redundant computations.
result Stage-based execution outperforms trial-based method by up to 6.60 times in GPU-hours and 4.13 times in training time.
Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.
problem Efficient tuning of hyper-parameters for stochastic optimization algorithms.
method Modeling hyper-parameter tuning as a Markov decision process and using policy gradient algorithms.
result The proposed framework significantly reduces the time required for hyper-parameter tuning compared to Bayesian optimization.
New algorithm tightens sensitivity bounds for coresets, reducing size by up to 10x.
problem Finding efficient coresets for least-mean-squares problems.
method Algorithm based on iterative sensitivity computation and reduction to higher-dimensional subspaces.
result Significantly smaller coresets with data-dependent sensitivity bounds.
PRESTO maps latent representations across diverse ML models.
problem Understanding variability in latent representations across different ML models.
method Uses persistent homology to characterize latent spaces and measure their pairwise similarity.
result Preserves desirable properties and enables sensitivity analysis of latent representations.
RLgraph separates RL tasks into modular components for stability and efficiency.
problem Algorithmic instability, hyper-parameter sensitivity, and distributed communication patterns in RL tasks.
method Introduces RLgraph, a library for RL tasks in static and define-by-run paradigms.
result Robust, testable, and high-performance implementations across different frameworks and backends.
This work improves Bayesian Optimization for setting DNN hyper-parameters.
problem Manual setting of DNN hyper-parameters is error-prone and computationally expensive.
method Combines Bayesian Optimization with tuning rules to reduce search space and improve accuracy.
result Improves efficiency and accuracy of hyper-parameter tuning for deep neural networks.
Hippo optimizes deep learning hyper-parameters by reducing redundant trials.
problem Redundant hyper-parameter trials in hyper-parameter optimization.
method Hippo breaks down hyper-parameter sequences into stages and executes them in a tree structure.
result Hippo reduces GPU-hours and training time significantly compared to existing methods.
Evaluation often aims to reduce the correctness or error characteristics of a system down to a single number, but that always involves trade-offs. Another way of dealing with this is to quote two numbers, such as Recall and Precision, or Sensitivity and Specificity. But it can also be useful to see more than this, and …
Study examines the cost of tuning hyper-parameters in regression models.
problem Estimating optimal hyper-parameters for regression models.
method Established finite-sample oracle inequalities for hyper-parameter selection.
result Generalization error of selected models shrinks at nearly a parametric rate.
While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-slow convergence, sensitivity to the settings of hyper-parameters such as learning rate, stagnation at high training errors, and difficulty …
Two-tier approach optimizes RL hyper-parameters for better agent learning.
problem Optimizing hyper-parameters in reinforcement learning to improve agent performance.
method Two-step optimization: first categorical hyper-parameters, then solution-level hyper-parameters.
result Promising results in simulated control tasks, suggesting user-independent reinforcement learning applications.
Federated learning improves with adaptive hyper-parameters and representation matching.
problem Heterogeneous client data leads to divergent local models in federated learning.
method Representation matching and adaptive hyper-parameters.
result Significant performance and robustness improvements in federated learning.
This paper explores hyperparameter optimization for machine learning models.
problem Finding the best hyper-parameters for machine learning models.
method Introduces state-of-the-art optimization techniques and discusses their application.
result Comparison of different optimization methods on benchmark datasets.
A new method selects algorithms and optimizes their hyper-parameters efficiently.
problem Redundant hyper-parameter search space in AutoML.
method Cascaded algorithm selection and hyper-parameter optimization with ER-UCB bandit.
result ER-UCB strategy achieves optimal regret bound for algorithm selection.
KD technique improves QDNN performance with reduced hyper-parameters.
problem Restoring performance loss in QDNNs due to quantization.
method Applied KD with reduced hyper-parameters, including a new coefficient reduction technique.
result Achieved 92.7% test accuracy on CIFAR-10 and 67.0% on CIFAR-100 with 2-bit weights.
Paper stabilizes DARTS algorithm for better neural architecture search.
problem Weak stability of DARTS algorithm leading to unreliable results.
method Amended gradient estimation method to bridge optimization gap.
result Significant improvement in search stability and larger search spaces explored.
Generative approach speeds hyperparameter tuning for machine learning models.
problem Computational infeasibility of cross-validation and difficulty of fully Bayesian hyper-parameter learning.
method Combines optimization-based approximations and amortization techniques.
result Rapid evaluation of hyper-parameters over grids or ranges, supporting predictive tuning and uncertainty quantification.
The quality of an induced model by a learning algorithm is dependent on the quality of the training data and the hyper-parameters supplied to the learning algorithm. Prior work has shown that improving the quality of the training data (i.e., by removing low quality instances) or tuning the learning algorithm hyper-para…
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
DSPG improves SPSA for distributed optimization with wireless delays.
problem Optimizing global functions in multi-agent systems with wireless delays and errors.
method Cross-entropy based distributed stochastic approximation algorithm (DSPG) using simultaneous perturbation.
result DSPG reduces biases due to communication delays and maintains convergence rate.
BCV method helps estimate clusters and hyper-parameters in large data sets.
problem Determining the number of clusters in large-scale data.
method Bi-cross validation (BCV) for spectral clustering.
result BCV directly applies to spectral clustering for estimating clusters and hyper-parameters.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Paper proposes efficient GP hyper-parameter optimization methods.
problem Efficient hyper-parameter optimization for Gaussian process regression.
method Cross-validation and ADMM for O(n2) complexity. result Proposed methods outperform traditional ML-based routines.
In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models. In this work, we investigate this tension between complexity and gen…
In this paper, a generalized multivariate Student-t mixture model is developed for classification and clustering of Low Probability of Intercept radar waveforms. A Low Probability of Intercept radar signal is characterized by a pulse compression waveform which is either frequency-modulated or phase-modulated. The propo…
Bayesian optimisation has gained great popularity as a tool for optimising the parameters of machine learning algorithms and models. Somewhat ironically, setting up the hyper-parameters of Bayesian optimisation methods is notoriously hard. While reasonable practical solutions have been advanced, they can often fail to …
Paper presents an efficient method for selecting machine learning algorithms and hyper-parameters.
problem Efficient selection of machine learning algorithms and hyper-parameters is challenging for large datasets.
method Progressive sampling-based Bayesian optimization
result Significantly reduces search time, classification error rate, and error rate variability.
ABPS improves RL training efficiency by sharing policies and evolving hyper-params.
problem Data inefficiency in training deep RL models for real-world applications.
method ABPS: adaptive behavior policy sharing; ABPS-PBT: hybridizing ABPS with PBT for evolving hyper-params.
result ABPS achieves superior performance and reduced variance compared to conventional hyper-parameter tuning.
XGBoost model improves business risk classification with feature selection and Bayesian hyper-parameter optimization.
problem Improving business risk classification models using advanced machine learning techniques.
method XGBoost, feature selection (FS), Bayesian hyper-parameter optimization (TPE and RS), 10-fold cross-validation.
result Bayesian TPE optimization outperforms random search and achieves higher accuracy and AUC, recall, and F1 score.
Optimizes SGD for text classification using hyper-parameter tuning.
problem Improving performance of SGD in text classification.
method Fine-tuning hyper-parameters using Grid-Search approach.
result Optimized SGD classification leads to better accuracy and execution time.
Bayesian optimisation tackles stochastic MPC hyper-parameter tuning.
problem Fine-tuning hyper-parameters in stochastic MPC models.
method Heteroscedastic Bayesian optimisation framework.
result Framework effectively tunes hyper-parameters in control problems.
Budgeted hyper-parameter tuning algorithm improves performance.
problem Optimizing hyper-parameters with resource constraints.
method Sequential decision making, Bayesian model, action-value function.
result Superior performance across various budgets.
SIBRE boosts reinforcement learning convergence by rewarding improvement over past performance.
problem Improving the rate of convergence in reinforcement learning.
method SIBRE is a reward shaping approach that rewards improvement over the agent's own past performance.
result SIBRE converges faster and more stably to the optimal policy compared to baseline RL algorithms.
Paper proposes CNN-LSTM model for multi-speaker speech separation.
problem Multi-speaker source separation using deep learning.
method Parallel CNN-LSTM architecture with Bayesian hyperparameter optimization.
result Parallel CNN-LSTM model outperforms LSTM-only and CNN-only models.