Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.
problem Efficient tuning of hyper-parameters for stochastic optimization algorithms.
method Modeling hyper-parameter tuning as a Markov decision process and using policy gradient algorithms.
result The proposed framework significantly reduces the time required for hyper-parameter tuning compared to Bayesian optimization.
Paper proposes a new algorithm for efficient hyper-parameter optimization.
problem Efficient hyper-parameter tuning for machine learning models.
method Information geometric optimization with stochastic natural gradient for discrete search domains.
result The proposed algorithm achieves faster optimization than existing methods without manual tuning.
Stage-based hyper-parameter optimization reduces GPU-hours and training time.
problem Efficiently executing hyper-parameter optimization for deep learning models.
method Stage-based execution strategy to remove redundant computations.
result Stage-based execution outperforms trial-based method by up to 6.60 times in GPU-hours and 4.13 times in training time.
Two-tier approach optimizes RL hyper-parameters for better agent learning.
problem Optimizing hyper-parameters in reinforcement learning to improve agent performance.
method Two-step optimization: first categorical hyper-parameters, then solution-level hyper-parameters.
result Promising results in simulated control tasks, suggesting user-independent reinforcement learning applications.
Bayesian Optimization improves Neural Network parameter tuning for XOR function.
problem Optimizing model parameters for neural networks, especially for large models.
method Bayesian Optimization with Gaussian Process Priors.
result Achieved higher prediction accuracy for the XOR function.
Develops a parameter-free SGD algorithm with optimal convergence rate.
problem Optimizing parameters in stochastic convex optimization.
method A novel parameter-free algorithm for SGD with high-probability guarantees and adaptive properties.
result Achieves optimal convergence rate with only a double-logarithmic factor increase compared to known-parameter settings.
New method achieves optimal performance without needing problem parameters.
problem Parameter-free stochastic optimization in non-convex and convex settings.
method Simple hyperparameter search technique for non-convex setting, and method with stochastic gradients for convex setting.
result Fully parameter-free methods can outperform state-of-the-art algorithms in both non-convex and convex settings.
Hippo optimizes deep learning hyper-parameters by reducing redundant trials.
problem Redundant hyper-parameter trials in hyper-parameter optimization.
method Hippo breaks down hyper-parameter sequences into stages and executes them in a tree structure.
result Hippo reduces GPU-hours and training time significantly compared to existing methods.
This paper explores hyperparameter optimization for machine learning models.
problem Finding the best hyper-parameters for machine learning models.
method Introduces state-of-the-art optimization techniques and discusses their application.
result Comparison of different optimization methods on benchmark datasets.
This work improves Bayesian Optimization for setting DNN hyper-parameters.
problem Manual setting of DNN hyper-parameters is error-prone and computationally expensive.
method Combines Bayesian Optimization with tuning rules to reduce search space and improve accuracy.
result Improves efficiency and accuracy of hyper-parameter tuning for deep neural networks.
Algorithm learns optimal parameters from infinite space for computational resource optimization.
problem Finding nearly-optimal parameters from an infinite space of tunable parameters.
method Learn a finite set of promising parameters from an infinite set using a data-independent discretization approach.
result Algorithm can help compile a configuration portfolio or select input to a configuration algorithm for finite parameter spaces.
This paper proposes an efficient autoHPO method based on data-to-hyper-parameter mapping.
problem Manual hyper-parameter tuning is costly and dependent.
method The approach is based on mapping from data to hyper-parameters using a sophisticated network structure and effective construction algorithms.
result The proposed approach significantly outperforms state-of-the-art methods.
Optimizes parameter reconstruction for optical scatterometry using Gaussian process regression.
problem Efficiently reconstructing geometry parameters of micro/nanostructures from scatterometry measurements.
method Bayesian optimization with Gaussian-process regression to find optimal parameter values.
result Gaussian process regression accelerates the optimization process for numerical simulations.
Improved Bayesian optimization for conditional parameter spaces.
problem Efficient global optimization of expensive-to-evaluate functions in conditional parameter spaces.
method Additive tree-structured covariance function for conditional parameter optimization.
result Significantly improved sample-efficiency and wider applicability compared to existing methods.
Unified GP model optimizes hyperparameters with conditional dependence.
problem Efficient tuning of hyperparameters in neural networks.
method Unified Bayesian optimization framework based on a new Gaussian process (GP) model.
result Higher prediction accuracy and better optimization efficiency observed.
A note on setting swap parameters for traders.
problem Determining optimal slippage parameters and trade size for wealth swapping.
method Theoretical solution and framework for optimal slippage parameters and trade size.
result Offers a method to solve optimal slippage parameters and trade size for wealth swapping.
Study optimizes sensor placement for accurate parameter estimation in complex systems.
problem Challenges in parameter estimation with limited or noisy data.
method Physics-Informed Neural Networks (PINNs) for optimal sensor placement and parameter estimation.
result PINNs-based framework achieves higher accuracy in parameter estimation compared to random sensor placements.
KOVA optimizes value functions using Kalman filtering, improving parameter uncertainty.
problem Improving parameter uncertainty in value function approximation.
method KOVA uses a trust region approach with a Bayesian perspective and Kalman filtering.
result KOVA provides more reliable parameter estimates and value function approximations.
New black-box reductions simplify online learning algorithms.
problem Designing adaptive and parameter-free online learning algorithms.
method Introducing black-box reductions to simplify analysis and improve regret guarantees.
result Improved regret bounds for parameter-free learning.
Paper uses Bayesian optimization to find best Supertrend indicator settings.
problem Finding optimal trading parameters for the Supertrend indicator.
method Bayesian optimization to automate parameter selection.
result BO-optimized Supertrend strategy yields higher profits in backtesting.
Scout-Nd optimizes parameters of stochastic simulators efficiently.
problem Optimizing parameters of stochastic, computationally expensive simulators.
method Scout-Nd algorithm, reducing gradient noise, multi-fidelity schemes.
result Demonstrates better performance compared to existing methods.
Develops parameter-free online mirror descent for optimal dynamic regret.
problem Optimal online linear optimization in unbounded domains.
method Modified online mirror descent framework for parameter-free algorithms.
result First unconstrained online linear optimization achieving optimal dynamic regret.
Two simulation-based methods improve optimal sampling design in systems biology.
problem Optimal selection of sampling points for accurate parameter estimation in dynamical systems.
method E-optimal-ranking (EOR) and LSTM neural network-based methods.
result Simulation studies show the proposed methods outperform random selection and classical E-optimal design.
Study optimal dividends in dual risk model under extreme parameter conditions.
problem Optimal dividends in a dual risk model with extreme parameters.
method Asymptotic analysis of optimal control problem.
result Insights into optimal strategies and values under extreme parameter conditions.
BALSON optimizes parameters with Bayesian approach and Dirichlet distribution.
problem Data fitting with nonnegative L1-norm constraints.
method Bayesian approach, Gaussian likelihood, Dirichlet distribution, sampling methods.
result BALSON outperforms conventional methods in polynomial fitting.
Integrates estimation and optimization for uncertain parameters.
problem Optimizing with uncertain parameters whose distributions can be estimated.
method Integrated Conditional Estimation-Optimization (ICEO) framework.
result Asymptotically consistent and provides finite performance guarantees.
Optimal strategies in stochastic control problems with two parameters are identified.
problem Optimal control strategies in stochastic processes with two parameters.
method First, parameters are chosen by continuous/smooth fit conditions. Then, optimality is shown using verification arguments.
result Optimal strategies can be concisely expressed via scale functions.
New framework optimizes random forest parameters for stability and cost.
problem Optimizing random forest parameters for industrial applications.
method Bayesian optimization framework considering error, stability, and cost.
result Parameter settings that balance error, stability, and cost.
Combines NES and PPO to enhance exploration in various environments.
problem Improving exploration in reinforcement learning environments.
method Parameter transfer and parameter space noise methods for combining NES and PPO.
result PPO benefits from both NES methods in discrete and continuous control tasks.
GoBOED optimizes experiments for specific decision-making objectives, improving downstream outcomes.
problem Reducing parameter uncertainty does not always improve decision-making in critical settings.
method Combines variational posterior surrogate and differentiable convex decision layer for gradient-based design optimization.
result GoBOED identifies designs that better align with specific decision objectives and reveals wider optimal design windows.
Parameter-free online convex optimization with sub-exponential noise achieves optimal regret.
problem Online convex optimization with sub-exponential noise, especially when subgradients are unbounded.
method Designing a novel parameter-free algorithm BANCO via a reduction to betting on noisy coins.
result BANCO achieves the optimal regret rate in the problem of unconstrained online convex optimization with sub-exponential noise.
Convex optimization models predict outputs from inputs via optimization problems.
problem Predicting outputs from inputs using convex optimization models.
method Proposed a heuristic for learning parameters of convex optimization models from datasets.
result Demonstrated the effectiveness of the proposed method on three model classes.
In practical Bayesian optimization, we must often search over structures with differing numbers of parameters. For instance, we may wish to search over neural network architectures with an unknown number of layers. To relate performance data gathered for different architectures, we define a new kernel for conditional p…
DLOPT simplifies deep learning hyper-parameter optimization.
problem Finding optimal network configurations in deep learning.
method Introduces a novel library for hyper-parameter optimization.
result Demonstrates effectiveness through use examples.
Sparse reduced-rank regression selects variables and ranks via manifold optimization.
problem Traditional rank selection fails when true rank is high.
method Sparse regularization and manifold optimization for rank and variable selection.
result Accurate estimation of coefficient parameter with high true rank.
Bayesian model selection optimizes data augmentation for improved machine learning robustness.
problem Choosing optimal data augmentation parameters is challenging and often done through trial and error.
method Interprets augmentation parameters as model hyperparameters and uses Bayesian model selection to optimize them.
result Our approach improves calibration and robust performance on various tasks.
XGBoost model improves business risk classification with feature selection and Bayesian hyper-parameter optimization.
problem Improving business risk classification models using advanced machine learning techniques.
method XGBoost, feature selection (FS), Bayesian hyper-parameter optimization (TPE and RS), 10-fold cross-validation.
result Bayesian TPE optimization outperforms random search and achieves higher accuracy and AUC, recall, and F1 score.
A parameter-free PGD algorithm for convex optimization.
problem Minimizing convex functions over convex sets.
method A fully adaptive AdaGrad variant of PGD without parameters or restarts.
result Optimal convergence rates for cumulative regret.
Machine learning speeds up LSM parameter optimization and uncertainty quantification.
problem Optimizing and assessing uncertainty of LSM parameters.
method Combining MCMC with Gaussian process regression.
result 50,000 times faster than direct MCMC application.
Bayesian optimization tunes distributed SGD parameters for faster convergence.
problem Finding efficient configurations to balance load in distributed SGD.
method Bayesian optimization with a probabilistic model of distributed SGD.
result Optimizer converges to efficient configurations within ten iterations.
New neural network approach for optimizing latent variable models.
problem Stability issues in marginalizing Gaussian Bayesian networks.
method Developed a new graphical structure and a neural network algorithm.
result Established a duality between parameter optimization and neural network training.
New Riemannian optimization improves variance estimation in mixed models.
problem Challenges in estimating variance parameters in linear mixed models due to constraints.
method Formulated as an optimization problem on a Riemannian manifold, using Riemannian gradient and Hessian.
result Yields higher quality variance parameter estimates compared to existing methods.
The study calibrates neural networks' parameters through optimal contraction in prediction problems.
problem Ensuring the existence and uniqueness of optimal parameters in neural networks.
method Transforming RNNs into contractions and solving matrix equations involving Sylvester equations.
result Optimal parameters exist, are unique, and can be found through an algorithm with desired precision.
New deep learning method simplifies parameter estimation design.
problem Optimal experimental design for parameter estimation with non-linear systems.
method Training a deep network as a Likelihood Free Estimator to simplify design process.
result Deep design improves parameter recovery quality and simplifies design process.
This paper shows using sub-sample estimates can improve optimization results in large-scale problems.
problem Large-scale optimization problems with uncertain parameters often lead to suboptimal solutions due to mis-specifications or extreme sample characteristics.
method The paper introduces the use of sub-sample estimates to reduce errors in stochastic optimization models, providing theoretical analysis and numerical examples.
result Sub-sample optimization can achieve improved results over full-sample solution estimates in large-scale problems.
Novel prediction method using string invariants with evolutionary optimization.
problem Optimal prediction parameters for string invariants.
method Evolutionary algorithm for parameter optimization.
result Method performs well in single step prediction but needs improvement for multiple steps.
Natural gradient optimization improves model parameter estimation in graphical models.
problem Estimating model parameters in graphical models.
method Reformulated as an information geometric optimization problem, introduced natural gradient descent strategy.
result Natural gradient strategy leads to optimal parameter learning without fitting an incorrect distribution.
NeAda solves nonconvex minimax optimization by balancing primal and dual variables adaptively.
problem Nonconvex minimax optimization challenges with parameter-agnostic adaptive algorithms.
method Nested Adaptive (NeAda) framework with inner and outer loops for primal and dual variables.
result Achieves near-optimal convergence rates for nonconvex-strongly-concave problems.