Study improves stock index prediction accuracy using TPE-GRNN models.
problem Enhancing prediction of stock index prices in volatile markets.
method Gated recurrent neural networks (LSTM, GRU) combined with TPE Bayesian optimization.
result TPE-LSTM method shows lowest MAPE (best accuracy) for NIFTY 50 index prediction.
This paper aims to explore models based on the extreme gradient boosting (XGBoost) approach for business risk classification. Feature selection (FS) algorithms and hyper-parameter optimizations are simultaneously considered during model training. The five most commonly used FS methods including weight by Gini, weight b…
SMAC method optimizes tree-boosting hyperparameters best.
problem Optimizing hyperparameters for tree-boosting to improve model accuracy.
method Compared and evaluated various hyperparameter optimization methods.
result SMAC method outperforms other methods for hyperparameter tuning.
Improved Bayesian optimization for conditional parameter spaces.
problem Efficient global optimization of expensive-to-evaluate functions in conditional parameter spaces.
method Additive tree-structured covariance function for conditional parameter optimization.
result Significantly improved sample-efficiency and wider applicability compared to existing methods.
Novel covariance function improves Bayesian optimization efficiency.
problem Efficient global optimization of expensive black-box functions.
method Additive tree-structured covariance function and parallel optimization algorithm.
result Significantly outperforms state-of-the-art methods in conditional parameter optimization.
Auto-Surprise automates recommender system selection and optimization.
problem Finding the best algorithm and hyperparameters for recommender systems.
method Extends Surprise library with TPE optimization for algorithm selection and hyperparameter tuning.
result Significantly faster in finding optimal hyperparameters compared to grid search.
This study improves hyperparameter optimization for categorical and non-normal data.
problem Bayesian hyperparameter optimization struggles with categorical hyperparameters and non-normal data.
method Integrates conformalized quantile regression to address estimation weaknesses and provides robust calibration guarantees.
result Quantile surrogate architectures and acquisition functions yield superior performance compared to existing methods.
A new Bayesian optimization method using Poisson process for better noise robustness.
problem Estimating relative rankings of candidates in noisy environments.
method Poisson process-based ranking surrogate model and tailored acquisition functions.
result PoPBO framework shows lower computation costs and better robustness to noise compared to GP-BO.
Novel Bayesian optimization framework improves portfolio management stability and efficiency.
problem Stable and sample-efficient optimization for black-box portfolio models under limited observation budgets.
method TPE-AS framework with adaptive scheduling and importance sampling.
result Demonstrated effectiveness across four backtest settings with three distinct models.
Automatically searching for optimal hyperparameter configurations is of crucial importance for applying deep learning algorithms in practice. Recently, Bayesian optimization has been proposed for optimizing hyperparameters of various machine learning algorithms. Those methods adopt probabilistic surrogate models like G…
Bayesian framework improves variance component estimation in MET data.
problem Inaccurate estimation of variance components in MET data.
method Proposes a Bayesian updating framework using historical data.
result Stabilizes variance component estimation and quantifies uncertainty.
Post-ADC inference corrects bias in statistical inference after active data collection.
problem Bias in inference after active data collection.
method Post-ADC inference framework that corrects bias from both ADC process and data-driven target construction.
result Valid inference for data collected by SMBO methods like GP-UCB and TPE.
Automl is the key technology for machine learning problem. Current state of art hyperparameter optimization methods are based on traditional black-box optimization methods like SMBO (SMAC, TPE). The objective function of black-box optimization is non-smooth, or time-consuming to evaluate, or in some way noisy. Recent y…
LLMs struggle to optimize hyperparameters efficiently, but hybrid methods can improve performance.
problem Optimizing hyperparameters of small language models using LLMs.
method Comparison of classical HPO algorithms and LLM-based methods, introducing Centaur hybrid approach.
result Hybrid Centaur approach achieves best results, outperforming classical and pure LLM methods.
Model-based reinforcement learning (MBRL) has been proposed as a promising alternative solution to tackle the high sampling cost challenge in the canonical reinforcement learning (RL), by leveraging a learned model to generate synthesized data for policy training purpose. The MBRL framework, nevertheless, is inherently…
One of the goals of the ICML workshop on representation and learning is to establish benchmark scores for a new data set of labeled facial expressions. This paper presents the performance of a "Null" model consisting of convolutions with random weights, PCA, pooling, normalization, and a linear readout. Our approach fo…
Bayesian optimization with cost-awareness using Gittins index.
problem Optimizing unknown functions with limited data evaluations and costs.
method Developed a connection between cost-aware Bayesian optimization and the Pandora's Box problem, using the Gittins index as an acquisition function.
result The Gittins index-based acquisition function performs well in cost-aware Bayesian optimization, especially in high dimensions.
SCoreBO improves Bayesian optimization by learning hyperparameters and self-correcting.
problem Efficient hyperparameter tuning for Gaussian process models in Bayesian optimization.
method Introduces SAL and SCoreBO, which prioritize hyperparameter learning and perform simultaneous optimization and learning.
result SCoreBO outperforms state-of-the-art methods on traditional benchmarks and atypical tasks.
Bayesian optimization sped up with scalable Gaussian processes.
problem Optimizing functions with derivative information and large datasets.
method Combines derivative acceleration and scalable Gaussian process models.
result Significant speedup in optimization convergence for large datasets.
When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesian optimization that is a generalization of two existing transfer learning extens…
Bayesian optimization simplifies bioprocess engineering experiments.
problem Complex biological systems and experimental uncertainty.
method Adapts classical Bayesian optimization for bioprocess engineering.
result Provides accessible introduction to Bayesian optimization for practitioners.
Unified Bayesian Optimization framework for model selection balancing effectiveness and training efficiency.
problem Balancing model effectiveness and training efficiency in machine learning model selection.
method Proposes a unified Bayesian Optimization framework to jointly optimize model effectiveness and training efficiency.
result Models selected using the proposed framework significantly improve training efficiency while maintaining strong effectiveness.
Proposes OBS, a method to adaptively combine Bayesian models online.
problem Learning optimal combinations of Bayesian models in online learning.
method Empirical Bayes lens, Online Bayesian Stacking (OBS).
result Establishes a novel connection between OBS and portfolio selection.
Bayesian optimization has become a popular method for high-throughput computing, like the design of computer experiments or hyperparameter tuning of expensive models, where sample efficiency is mandatory. In these applications, distributed and scalable architectures are a necessity. However, Bayesian optimization is mo…
Bayesian optimization method tackles combinatorial spaces, scalable for large data.
problem Optimization over combinatorial categorical spaces in natural sciences.
method Combines variational optimization and continuous relaxations for gradient-based optimization.
result Method performs comparably to state-of-the-art methods while scaling well.
This paper presents a Bayesian optimization method with exponential convergence without the need of auxiliary optimization and without the delta-cover sampling. Most Bayesian optimization methods require auxiliary optimization: an additional non-convex global optimization problem, which can be time-consuming and hard t…
Postprocessing reduces Bayesian optimization steps for global optima.
problem Slow convergence in Bayesian optimization for high-dimensional problems.
method Prohibits duplicated samples in the dataset postprocessing method.
result Significantly reduces the number of sequential steps to find the global optimum.
SILBO optimizes high-dimensional Bayesian optimization using semi-supervised embedding learning.
problem Bayesian optimization struggles with high-dimensional search spaces.
method SILBO uses semi-supervised dimension reduction to find a low-dimensional space for iterative optimization.
result SILBO outperforms existing methods on high-dimensional Bayesian optimization tasks.
Finding an optimal parameter of a black-box function is important for searching stable material structures and finding optimal neural network structures, and Bayesian optimization algorithms are widely used for the purpose. However, most of existing Bayesian optimization algorithms can only handle vector data and canno…
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consuming and expensive model training / validation, is not a trivial challenge. Bayesian optimization is a…
An adaptive dropout approach improves high-dimensional Bayesian optimization.
problem High-dimensional black-box optimization problems.
method Adaptive dropout of variables in the acquisition function.
result AdaDropout effectively tackles high-dimensional challenges and improves solution quality.
BARK optimizes black-box functions using Bayesian Additive Regression Trees.
problem Bayesian optimization of complex, black-box functions with uncertainty quantification.
method BART Kernel using tree agreement for posterior over piecewise-constant functions, explored using MCMC.
result BARK obtains samples of Gaussian processes for function distributions, enabling acquisition functions for optimization.
Bayesian optimization has shown to be a fundamental global optimization algorithm in many applications: ranging from automatic machine learning, robotics, reinforcement learning, experimental design, simulations, etc. The most popular and effective Bayesian optimization relies on a surrogate model in the form of a Gaus…
Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a c…
Bayesian optimization enhanced with conformal prediction for better outcome reliability.
problem Uncertainty and model misspecification in Bayesian optimization.
method Conformal prediction to provide coverage guarantees and Bayesian optimization to select queries.
result Significant improvement in query coverage without sacrificing sample-efficiency.
Bayesian optimization technique scaled using Vecchia approximations.
problem Scalability issue with Gaussian process surrogate models in Bayesian optimization.
method Adapted Vecchia approximation from spatial statistics to Gaussian processes, developed improvements and extensions.
result Methods compared favorably to state-of-the-art on various test functions and reinforcement learning problems.
Bayesian optimization is an effective method to efficiently optimize unknown objective functions with high evaluation costs. Traditional Bayesian optimization algorithms select one point per iteration for single objective function, whereas in recent years, Bayesian optimization for multi-objective optimization or multi…
Automatic Chemical Design is a framework for generating novel molecules with optimized properties. The original scheme, featuring Bayesian optimization over the latent space of a variational autoencoder, suffers from the pathology that it tends to produce invalid molecular structures. First, we demonstrate empirically …
Proposes MCBO for causal Bayesian optimization with model learning and regret bounds.
problem Maximizing downstream variables in unknown structural models.
method Model-based causal Bayesian optimization (MCBO) that learns full system models and trades off exploration and exploitation.
result First non-asymptotic bounds for CBO and practical implementation showing superior performance.
Bayesian optimization reduces computational effort in aircraft design optimization.
problem High computational cost in industrial aircraft design optimization.
method Constrained Bayesian optimization (Super Efficient Global Optimization with Mixture of Experts)
result Significant computational efficiency improvements over existing Isight optimizers.
NUBO simplifies Bayesian optimization for researchers.
problem Optimizing expensive functions like experiments and simulations.
method Bayesian optimization using Gaussian processes and acquisition functions.
result Transparency and user-friendly design for easy access.
Local Bayesian optimization shows strong performance and converges well, contrary to folklore.
problem Understanding the behavior and convergence of local Bayesian optimization methods.
method Studied the behavior of local optimization strategies and rigorously analyzed a specific algorithm.
result Local Bayesian optimization algorithms converge well and perform strongly, contrary to the folklore.
In typical applications of Bayesian optimization, minimal assumptions are made about the objective function being optimized. This is true even when researchers have prior information about the shape of the function with respect to one or more argument. We make the case that shape constraints are often appropriate in at…
The aim of black-box optimization is to optimize an objective function within the constraints of a given evaluation budget. In this problem, it is generally assumed that the computational cost for evaluating a point is large; thus, it is important to search efficiently with as low budget as possible. Bayesian optimizat…
The paper improves Bayesian optimization by calibrating uncertainty estimates.
problem Improper uncertainty estimates in Bayesian optimization when data is non-stationary.
method Proposes online learning algorithms to maintain calibration on non-i.i.d. data and integrates them into Bayesian optimization.
result Calibrated Bayesian optimization converges to better optima in fewer steps.
Bayesian optimization improves with transfer learning for aircraft design.
problem Cold start problem in Bayesian optimization for aircraft design.
method Ensemble of surrogate models using transfer learning in a constrained Bayesian optimization framework.
result Significant improvement in convergence and prediction accuracy.
Unified Bayesian-AI framework improves epidemiological risk prediction and uncertainty quantification.
problem Lack of calibrated uncertainty in machine learning models for epidemiology.
method Combines Bayesian prediction with Bayesian hyperparameter optimization using logistic regression and Gaussian-process Bayesian optimization.
result Unified Bayesian-AI framework provides reliable coverage and improved calibration, enhancing epidemiological decision making.