Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.
problem Efficient tuning of hyper-parameters for stochastic optimization algorithms.
method Modeling hyper-parameter tuning as a Markov decision process and using policy gradient algorithms.
result The proposed framework significantly reduces the time required for hyper-parameter tuning compared to Bayesian optimization.
Paper proposes a new algorithm for efficient hyper-parameter optimization.
problem Efficient hyper-parameter tuning for machine learning models.
method Information geometric optimization with stochastic natural gradient for discrete search domains.
result The proposed algorithm achieves faster optimization than existing methods without manual tuning.
Stage-based hyper-parameter optimization reduces GPU-hours and training time.
problem Efficiently executing hyper-parameter optimization for deep learning models.
method Stage-based execution strategy to remove redundant computations.
result Stage-based execution outperforms trial-based method by up to 6.60 times in GPU-hours and 4.13 times in training time.
Two-tier approach optimizes RL hyper-parameters for better agent learning.
problem Optimizing hyper-parameters in reinforcement learning to improve agent performance.
method Two-step optimization: first categorical hyper-parameters, then solution-level hyper-parameters.
result Promising results in simulated control tasks, suggesting user-independent reinforcement learning applications.
Develops a parameter-free SGD algorithm with optimal convergence rate.
problem Optimizing parameters in stochastic convex optimization.
method A novel parameter-free algorithm for SGD with high-probability guarantees and adaptive properties.
result Achieves optimal convergence rate with only a double-logarithmic factor increase compared to known-parameter settings.
New method achieves optimal performance without needing problem parameters.
problem Parameter-free stochastic optimization in non-convex and convex settings.
method Simple hyperparameter search technique for non-convex setting, and method with stochastic gradients for convex setting.
result Fully parameter-free methods can outperform state-of-the-art algorithms in both non-convex and convex settings.
This paper explores hyperparameter optimization for machine learning models.
problem Finding the best hyper-parameters for machine learning models.
method Introduces state-of-the-art optimization techniques and discusses their application.
result Comparison of different optimization methods on benchmark datasets.
Hippo optimizes deep learning hyper-parameters by reducing redundant trials.
problem Redundant hyper-parameter trials in hyper-parameter optimization.
method Hippo breaks down hyper-parameter sequences into stages and executes them in a tree structure.
result Hippo reduces GPU-hours and training time significantly compared to existing methods.
This work improves Bayesian Optimization for setting DNN hyper-parameters.
problem Manual setting of DNN hyper-parameters is error-prone and computationally expensive.
method Combines Bayesian Optimization with tuning rules to reduce search space and improve accuracy.
result Improves efficiency and accuracy of hyper-parameter tuning for deep neural networks.
Algorithm learns optimal parameters from infinite space for computational resource optimization.
problem Finding nearly-optimal parameters from an infinite space of tunable parameters.
method Learn a finite set of promising parameters from an infinite set using a data-independent discretization approach.
result Algorithm can help compile a configuration portfolio or select input to a configuration algorithm for finite parameter spaces.
This paper proposes an efficient autoHPO method based on data-to-hyper-parameter mapping.
problem Manual hyper-parameter tuning is costly and dependent.
method The approach is based on mapping from data to hyper-parameters using a sophisticated network structure and effective construction algorithms.
result The proposed approach significantly outperforms state-of-the-art methods.
Optimizes parameter reconstruction for optical scatterometry using Gaussian process regression.
problem Efficiently reconstructing geometry parameters of micro/nanostructures from scatterometry measurements.
method Bayesian optimization with Gaussian-process regression to find optimal parameter values.
result Gaussian process regression accelerates the optimization process for numerical simulations.
Improved Bayesian optimization for conditional parameter spaces.
problem Efficient global optimization of expensive-to-evaluate functions in conditional parameter spaces.
method Additive tree-structured covariance function for conditional parameter optimization.
result Significantly improved sample-efficiency and wider applicability compared to existing methods.
Unified GP model optimizes hyperparameters with conditional dependence.
problem Efficient tuning of hyperparameters in neural networks.
method Unified Bayesian optimization framework based on a new Gaussian process (GP) model.
result Higher prediction accuracy and better optimization efficiency observed.
A note on setting swap parameters for traders.
problem Determining optimal slippage parameters and trade size for wealth swapping.
method Theoretical solution and framework for optimal slippage parameters and trade size.
result Offers a method to solve optimal slippage parameters and trade size for wealth swapping.
Study optimizes sensor placement for accurate parameter estimation in complex systems.
problem Challenges in parameter estimation with limited or noisy data.
method Physics-Informed Neural Networks (PINNs) for optimal sensor placement and parameter estimation.
result PINNs-based framework achieves higher accuracy in parameter estimation compared to random sensor placements.
KOVA optimizes value functions using Kalman filtering, improving parameter uncertainty.
problem Improving parameter uncertainty in value function approximation.
method KOVA uses a trust region approach with a Bayesian perspective and Kalman filtering.
result KOVA provides more reliable parameter estimates and value function approximations.
Paper uses Bayesian optimization to find best Supertrend indicator settings.
problem Finding optimal trading parameters for the Supertrend indicator.
method Bayesian optimization to automate parameter selection.
result BO-optimized Supertrend strategy yields higher profits in backtesting.
Scout-Nd optimizes parameters of stochastic simulators efficiently.
problem Optimizing parameters of stochastic, computationally expensive simulators.
method Scout-Nd algorithm, reducing gradient noise, multi-fidelity schemes.
result Demonstrates better performance compared to existing methods.
Develops parameter-free online mirror descent for optimal dynamic regret.
problem Optimal online linear optimization in unbounded domains.
method Modified online mirror descent framework for parameter-free algorithms.
result First unconstrained online linear optimization achieving optimal dynamic regret.
When applying Machine Learning techniques to problems, one must select model parameters to ensure that the system converges but also does not become stuck at the objective function's local minimum. Tuning these parameters becomes a non-trivial task for large models and it is not always apparent if the user has found th…
We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…
Two simulation-based methods improve optimal sampling design in systems biology.
problem Optimal selection of sampling points for accurate parameter estimation in dynamical systems.
method E-optimal-ranking (EOR) and LSTM neural network-based methods.
result Simulation studies show the proposed methods outperform random selection and classical E-optimal design.
Integrates estimation and optimization for uncertain parameters.
problem Optimizing with uncertain parameters whose distributions can be estimated.
method Integrated Conditional Estimation-Optimization (ICEO) framework.
result Asymptotically consistent and provides finite performance guarantees.
Combines NES and PPO to enhance exploration in various environments.
problem Improving exploration in reinforcement learning environments.
method Parameter transfer and parameter space noise methods for combining NES and PPO.
result PPO benefits from both NES methods in discrete and continuous control tasks.
GoBOED optimizes experiments for specific decision-making objectives, improving downstream outcomes.
problem Reducing parameter uncertainty does not always improve decision-making in critical settings.
method Combines variational posterior surrogate and differentiable convex decision layer for gradient-based design optimization.
result GoBOED identifies designs that better align with specific decision objectives and reveals wider optimal design windows.
Parameter-free online convex optimization with sub-exponential noise achieves optimal regret.
problem Online convex optimization with sub-exponential noise, especially when subgradients are unbounded.
method Designing a novel parameter-free algorithm BANCO via a reduction to betting on noisy coins.
result BANCO achieves the optimal regret rate in the problem of unconstrained online convex optimization with sub-exponential noise.
Convex optimization models predict outputs from inputs via optimization problems.
problem Predicting outputs from inputs using convex optimization models.
method Proposed a heuristic for learning parameters of convex optimization models from datasets.
result Demonstrated the effectiveness of the proposed method on three model classes.
In practical Bayesian optimization, we must often search over structures with differing numbers of parameters. For instance, we may wish to search over neural network architectures with an unknown number of layers. To relate performance data gathered for different architectures, we define a new kernel for conditional p…
Sparse reduced-rank regression selects variables and ranks via manifold optimization.
problem Traditional rank selection fails when true rank is high.
method Sparse regularization and manifold optimization for rank and variable selection.
result Accurate estimation of coefficient parameter with high true rank.
Bayesian model selection optimizes data augmentation for improved machine learning robustness.
problem Choosing optimal data augmentation parameters is challenging and often done through trial and error.
method Interprets augmentation parameters as model hyperparameters and uses Bayesian model selection to optimize them.
result Our approach improves calibration and robust performance on various tasks.
A parameter-free PGD algorithm for convex optimization.
problem Minimizing convex functions over convex sets.
method A fully adaptive AdaGrad variant of PGD without parameters or restarts.
result Optimal convergence rates for cumulative regret.
XGBoost model improves business risk classification with feature selection and Bayesian hyper-parameter optimization.
problem Improving business risk classification models using advanced machine learning techniques.
method XGBoost, feature selection (FS), Bayesian hyper-parameter optimization (TPE and RS), 10-fold cross-validation.
result Bayesian TPE optimization outperforms random search and achieves higher accuracy and AUC, recall, and F1 score.
Machine learning speeds up LSM parameter optimization and uncertainty quantification.
problem Optimizing and assessing uncertainty of LSM parameters.
method Combining MCMC with Gaussian process regression.
result 50,000 times faster than direct MCMC application.
New neural network approach for optimizing latent variable models.
problem Stability issues in marginalizing Gaussian Bayesian networks.
method Developed a new graphical structure and a neural network algorithm.
result Established a duality between parameter optimization and neural network training.
New Riemannian optimization improves variance estimation in mixed models.
problem Challenges in estimating variance parameters in linear mixed models due to constraints.
method Formulated as an optimization problem on a Riemannian manifold, using Riemannian gradient and Hessian.
result Yields higher quality variance parameter estimates compared to existing methods.
The study calibrates neural networks' parameters through optimal contraction in prediction problems.
problem Ensuring the existence and uniqueness of optimal parameters in neural networks.
method Transforming RNNs into contractions and solving matrix equations involving Sylvester equations.
result Optimal parameters exist, are unique, and can be found through an algorithm with desired precision.
New deep learning method simplifies parameter estimation design.
problem Optimal experimental design for parameter estimation with non-linear systems.
method Training a deep network as a Likelihood Free Estimator to simplify design process.
result Deep design improves parameter recovery quality and simplifies design process.
This paper shows using sub-sample estimates can improve optimization results in large-scale problems.
problem Large-scale optimization problems with uncertain parameters often lead to suboptimal solutions due to mis-specifications or extreme sample characteristics.
method The paper introduces the use of sub-sample estimates to reduce errors in stochastic optimization models, providing theoretical analysis and numerical examples.
result Sub-sample optimization can achieve improved results over full-sample solution estimates in large-scale problems.
Natural gradient optimization improves model parameter estimation in graphical models.
problem Estimating model parameters in graphical models.
method Reformulated as an information geometric optimization problem, introduced natural gradient descent strategy.
result Natural gradient strategy leads to optimal parameter learning without fitting an incorrect distribution.
NeAda solves nonconvex minimax optimization by balancing primal and dual variables adaptively.
problem Nonconvex minimax optimization challenges with parameter-agnostic adaptive algorithms.
method Nested Adaptive (NeAda) framework with inner and outer loops for primal and dual variables.
result Achieves near-optimal convergence rates for nonconvex-strongly-concave problems.
A new method selects algorithms and optimizes their hyper-parameters efficiently.
problem Redundant hyper-parameter search space in AutoML.
method Cascaded algorithm selection and hyper-parameter optimization with ER-UCB bandit.
result ER-UCB strategy achieves optimal regret bound for algorithm selection.
Investigates optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.
problem Optimal consumption and investment strategies in non-Markovian markets with unbounded parameters.
method Martingale optimal principle and quadratic BSDEs with exponential moment.
result Establishes optimal strategies for consumption and investment.
Evolutionary Strategies optimize hyper-parameters for off-policy learning.
problem Hyper-parameter sensitivity in off-policy learning.
method Application of Evolutionary Strategies for online hyper-parameter tuning.
result Our method outperforms state-of-the-art baselines.
Paper refines PAC-Bayes bounds for bandit problems.
problem Improving probabilistic bounds for off-policy learning.
method Optimizes PAC-Bayesian bounds using a new parameter optimization approach.
result Provides two parameter-free PAC-Bayes bounds that nearly match optimal rates.
Optimal B-robust estimate is constructed for multidimensional parameter in drift coefficient of diffusion type process with small noise. Optimal mean-variance robust (optimal V -robust) trading strategy is find to hedge in mean-variance sense the contingent claim in incomplete financial market with arbitrary informatio…
Study investigates how errors in reinsurance parameters degrade optimal solutions.
problem Effectiveness of optimal reinsurance solutions degraded by errors in parameters and models.
method Asymptotic and numerical studies, including Value at Risk criteria and Bayesian integration.
result Rate of degradation often O ( 1 / n ) O(1/n) O ( 1/ n ) , but can be O ( 1 / n ) O(1/\sqrt{n}) O ( 1/ n ) for Value at Risk. Generative Adversarial Networks optimize model parameters for image matching.
problem Optimizing model parameters for accurate image matching.
method Model-Assisted Generative Adversarial Network (GAN) to produce fake images matching true images.
result Best match model parameter values can minimize bias in image recognition.