Spaces over BO are equivalent to thickened manifolds.
problem Understanding embeddings of manifolds in higher dimensions.
method Formal identification of manifolds with their thickened versions, using geometric constructions.
result The infinity-category of thickened smooth manifolds is equivalent to the infinity-category of finite spaces over BO.
MORBO improves multi-objective BO for high-dimensional problems.
problem Optimizing multiple objectives in high-dimensional spaces with expensive evaluations.
method Parallel local BO in multiple regions with coordinated strategy.
result Significant improvement in sample efficiency for high-dimensional problems.
BOSS optimizes string inputs using string kernels and genetic algorithms.
problem Optimizing string inputs with constraints.
method Bayesian optimization over string kernels and genetic algorithms.
result Significantly improved optimization across various string constraints.
Bayesian Optimisation (BO) refers to a suite of techniques for global optimisation of expensive black box functions, which use introspective Bayesian models of the function to efficiently search for the optimum. While BO has been applied successfully in many applications, modern optimisation tasks usher in new challeng…
Enhances BO in high dimensions with Newton methods.
problem Challenges in scaling BO to high-dimensional spaces.
method Construct multiple local quadratic models using gradients and Hessians from a global GP, and select new sample points by solving bound-constrained quadratic programs.
result Outperforms existing high-dimensional BO techniques on synthetic and real-world applications.
LOL-BO improves latent space Bayesian optimization over structured inputs.
problem Optimizing complex functions over high-dimensional, structured search spaces.
method Adapting trust regions from high-dimensional to structured settings, using a DAE to map inputs into a latent space.
result Achieves up to 20x improvement over state-of-the-art methods.
GIBBON unifies Bayesian optimization for various problem types.
problem Bayesian optimization across noisy, multi-fidelity, and batch optimisations.
method Novel approximation of information gain for a single framework.
result Significantly lower computational overhead compared to existing approaches.
Bayesian optimization tackles expensive discrete and mixed parameter spaces.
problem Optimizing expensive functions with discrete and mixed parameters.
method Probabilistic reparameterization to maximize expectation of AF over continuous parameters.
result Our approach provably converges to a maximizer of the AF and enjoys the same regret bounds as standard BO.
New framework analyzes regret in guided diffusion for optimizing structured inputs.
problem Understanding regret behavior in guided-diffusion black-box optimization for structured design problems.
method Developed a certificate-based expected simple-regret framework that avoids assumptions breaking down in modern diffusion BO pipelines.
result Explains how exponential and polynomial convergence can arise from mass lift in near-optimal designs.
Rank-based Bayesian Optimization improves molecule selection in chemical systems.
problem Optimizing chemical compounds using traditional regression models.
method Introducing Rank-based Bayesian Optimization (RBO) using ranking models.
result RBO outperforms regression-based BO, especially for rough landscapes and activity cliffs.
A new framework tackles CASH problem with alternating optimization and Rising Bandits.
problem Efficiently solving the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem.
method Alternating optimization framework using BO for HPO and Rising Bandits for algorithm selection.
result Demonstrated superiority over competitive baselines in extensive experiments.
KPCA-BO improves BO for high-dimensional optimization problems by learning a non-linear sub-manifold.
problem High-dimensional optimization problems where Gaussian Process regression requires too much data and computation.
method KPCA-BO embeds a non-linear sub-manifold in the search space, learning a GPR model on this sub-manifold.
result KPCA-BO outperforms vanilla BO in convergence speed, especially as dimensionality increases.
πBO augments BO with user beliefs for better hyperparameter optimization.
problem BO ignores user beliefs, reducing its appeal to practitioners.
method Proposes πBO, an acquisition function that incorporates user-provided prior beliefs. result πBO outperforms competing approaches and deep learning tasks. Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.
problem Efficient optimisation of expensive black-box functions in unknown search spaces.
method Expands search space over iterations based on a hyperharmonic series, scales to high dimensions.
result Sub-linear regret growth for both algorithms.
New BO method efficiently optimizes high-dimensional functions by automatically selecting variables.
problem Efficiently optimizing functions with high-dimensional domains.
method Exploits variable selection to automatically learn sub-spaces without pre-specified dimensions.
result Empirically validated on synthetic and real problems, demonstrating efficiency.
This paper uses Bayesian optimization to efficiently identify stochastic dynamical systems.
problem Efficiently identifying linear stochastic dynamical systems with unknown coefficients and noise variances.
method Adaptive Bayesian optimization with ensemble Gaussian processes (EGP) and Kalman filter recursion.
result BO-based estimator achieves RMSE below the Cramer-Rao bound, improving robustness and consistency.
Bayesian optimization for high-dimensional combinatorial spaces using embeddings.
problem Optimizing expensive functions over large, complex input spaces.
method Dictionary-based ordinal embeddings for high-dimensional combinatorial structures, using Gaussian process models.
result The proposed method outperforms state-of-the-art BO methods on diverse real-world benchmarks.
ARCO-BO optimizes multi-agent design under heterogeneity, improving efficiency and performance.
problem Heterogeneous multi-agent optimization challenges in resource use and information sharing.
method ARCO-BO integrates a consensus mechanism, budget-aware sampling, and partial input sharing for heterogeneous design spaces.
result ARCO-BO outperforms independent and collaborative BO methods in complex multi-agent settings.
Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic program (DP) that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-sp…
New method optimizes multiple points in Bayesian optimization efficiently.
problem Optimizing multiple points in expensive black-box functions.
method Reformulated BO as probability measure optimization, using convex gradient flows.
result Demonstrated effectiveness on various benchmarks compared to state-of-the-art methods.
Proposes a new method for faster function optima through integrated model selection and Bayesian optimization.
problem Efficient model selection and optimization in Bayesian optimization.
method Integrates model selection and Bayesian optimization by moving back and forth between model space and function space, using a score function to guide model selection.
result Significant improvement in convergence compared to standard Bayesian optimization, with improved sample efficiency.
BOE reformulates BO as a classifier for scalable batch optimisation.
problem Scalable batch optimisation of expensive functions.
method Reformulates BO as density-ratio estimation, removing need for explicit function prior.
result Theoretical guarantees and improved uncertainty estimates for batch optimisation.
BOFiP optimizes high-dimensional functions by distributing them into sub-spaces and using game theory.
problem Optimizing high-dimensional black box functions with computational complexity.
method BOFiP decomposes high-dimensional space into sub-spaces, searches within sub-spaces, and updates beliefs using game theory.
result BOFiP outperforms competitors in high-dimensional optimization problems.
Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-box functions. In this …
Bayesian optimization improves molecule design by addressing three pitfalls.
problem Bayesian optimization pitfalls cause poor performance in molecule design.
method Identified and addressed three pitfalls: incorrect prior width, over-smoothing, and inadequate acquisition function maximization.
result Basic BO setup achieves highest performance on PMO benchmark.
BO method identifies sparse subspaces for efficient high-dimensional optimization.
problem Efficient optimization of high-dimensional black-box functions.
method Sparse Gaussian process surrogate models on axis-aligned subspaces with Hamiltonian Monte Carlo inference.
result SAASBO achieves excellent performance on synthetic and real-world problems.
A new method optimizes material discovery by balancing exploration and exploitation.
problem Substantial experimental costs and lengthy development periods in material discovery.
method Threshold-Driven UCB-EI Bayesian Optimization (TDUE-BO) method.
result TDUE-BO significantly outperforms traditional BO methods in material discovery.
MOCA-HESP optimizes high-dimensional combinatorial and mixed spaces using hyper-ellipsoid partitioning.
problem Challenges in optimizing high-dimensional, combinatorial and mixed spaces.
method MOCA-HESP uses hyper-ellipsoid space partitioning with different categorical encoders and multi-armed bandit for adaptive selection.
result MOCA-HESP outperforms existing methods on various synthetic and real-world benchmarks.
In-BO optimizes complex constrained domains using SIn-GP surrogate models.
problem Optimizing in complex constrained domains with irregular shapes.
method Sparse Intrinsic Gaussian Processes (SIn-GP) on manifolds with heat kernel estimation.
result In-BO outperforms traditional BO in complex constrained domains.
Improves Bayesian optimization efficiency for mixed variable spaces.
problem Boosting sample efficiency in Bayesian optimization for mixed variable spaces.
method Proposes frequency modulated (FM) kernels to model complex dependencies across different types of variables.
result BO-FM outperforms competitors in various optimization problems.
Study compares high-dimensional BO algorithms on 24 functions.
problem Challenges in optimizing high-dimensional problems with BO.
method Comparison of five BO algorithms on 24 BBOB functions at varying dimensions.
result BO outperforms CMA-ES for limited evaluations, trust regions show promise.
Bayesian optimization (BO) based on Gaussian process models is a powerful paradigm to optimize black-box functions that are expensive to evaluate. While several BO algorithms provably converge to the global optimum of the unknown function, they assume that the hyperparameters of the kernel are known in advance. This is…
Simple linear models outperform complex BO methods in high dimensions.
problem Overcoming the curse of dimensionality in Bayesian optimization.
method Bayesian linear regression with linear kernels, applied to high-dimensional search spaces.
result Simple linear models match or outperform state-of-the-art BO methods in high-dimensional tasks.
Bayesian optimization (BO) is a powerful approach for seeking the global optimum of expensive black-box functions and has proven successful for fine tuning hyper-parameters of machine learning models. However, BO is practically limited to optimizing 10--20 parameters. To scale BO to high dimensions, we usually make str…
A new method for high-dimensional Bayesian optimization.
problem Challenges in extending BO to high dimensions.
method Expected Coordinate Improvement (ECI) criterion for high-dimensional Bayesian optimization.
result Significantly better results than standard BO and competitive results with state-of-the-art methods.
Bayesian optimization (BO) is a popular approach to optimize expensive-to-evaluate black-box functions. A significant challenge in BO is to scale to high-dimensional parameter spaces while retaining sample efficiency. A solution considered in existing literature is to embed the high-dimensional space in a lower-dimensi…
Computational models in fields such as computational neuroscience are often evaluated via stochastic simulation or numerical approximation. Fitting these models implies a difficult optimization problem over complex, possibly noisy parameter landscapes. Bayesian optimization (BO) has been successfully applied to solving…
New method optimizes costly evaluations in Bayesian optimization.
problem Costly evaluations in BO methods, especially in hyperparameter tuning.
method Formulated as a CMDP, developed rollout approximation for optimal policy.
result Improved cost efficiency over standard BO methods.
The performance of deep neural networks (DNN) is very sensitive to the particular choice of hyper-parameters. To make it worse, the shape of the learning curve can be significantly affected when a technique like batchnorm is used. As a result, hyperparameter optimization of deep networks can be much more challenging th…
The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete doma…
One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs). We address this problem by taking advantage of recent advances in scientific machine learning and the dynamically orthogonal (DO) and bi-orthogonal (BO) methods for representing …
Bayesian Optimization with a Prior for the Optimum (BOPrO) improves efficiency and accuracy.
problem Bayesian Optimization's standard priors are not intuitive for domain experts.
method BOPrO injects expert knowledge into the optimization process using priors about the optimum.
result BOPrO is 6.67x faster than state-of-the-art methods and achieves new state-of-the-art performance.
Simplifies BO by directly sampling from posterior, achieving 35x efficiency.
problem Optimizing expensive functions with Bayesian Optimization.
method Direct sampling from posterior using a pre-trained generative model.
result Achieves 35x efficiency gain over Gaussian process-based BO.
Develops a framework for cost-efficient Bayesian optimization with constraints.
problem Optimizing designs with minimal cost in constrained search spaces.
method Constrained multi-fidelity Bayesian optimization (CMFBO) with automatic stopping criterion.
result Minimizes overall sampling costs while ensuring feasibility.
HyperBO+ pre-trains a universal prior for Bayesian optimization across different domains.
problem Bayesian optimization requires domain-specific priors, limiting its applicability.
method Two-step pre-training method for hierarchical Gaussian processes.
result HyperBO+ achieves lower regrets on unseen search spaces.
Cost-aware BO minimizes function evaluations with varying costs.
problem Optimization with varying evaluation costs in expensive functions.
method Cost Apportioned Bayesian Optimization (CArBO) combining initial and cost-cooled phases.
result CArBO finds better hyperparameter configurations with the same cost budget.
Bayesian optimization (BO) has become an effective approach for black-box function optimization problems when function evaluations are expensive and the optimum can be achieved within a relatively small number of queries. However, many cases, such as the ones with high-dimensional inputs, may require a much larger numb…
We study the moduli space of handlebodies diffeomorphic to (Dn+1×Sn)♮g, i.e. the classifying space BDiff((Dn+1×Sn)♮g,D2n) of the group of diffeomorphisms that restrict to the identity near a 2n-dimensional disk embedded in the boundary, $\partial(D^{n+1}\times S^n)^…