Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.
problem Efficient optimisation of expensive black-box functions in unknown search spaces.
method Expands search space over iterations based on a hyperharmonic series, scales to high dimensions.
result Sub-linear regret growth for both algorithms.
Bayesian optimisation tackles expensive black-box functions with constraints.
problem Optimizing constrained black-box functions in machine learning and simulation.
method Proposes a new Knowledge Gradient acquisition function for constrained Bayesian optimisation.
result Demonstrates superior performance over four state-of-the-art constrained Bayesian optimisation algorithms.
We introduce a means of automating machine learning (ML) for big data tasks, by performing scalable stochastic Bayesian optimisation of ML algorithm parameters and hyper-parameters. More often than not, the critical tuning of ML algorithm parameters has relied on domain expertise from experts, along with laborious hand…
BOE reformulates BO as a classifier for scalable batch optimisation.
problem Scalable batch optimisation of expensive functions.
method Reformulates BO as density-ratio estimation, removing need for explicit function prior.
result Theoretical guarantees and improved uncertainty estimates for batch optimisation.
A new approach reduces computational time in Bayesian Optimisation Algorithm (BOA).
problem Time-consuming PGM updates in BOA.
method Proposes FBOA, a new BOA-based optimisation approach that avoids frequent PGM updates.
result FBOA presents competitive results while significantly saving computational time.
Bayesian optimisation framework for graph functions.
problem Optimizing functions on graph structures efficiently.
method Learning suitable kernels and local modelling.
result Demonstrated effectiveness on synthetic and real-world graphs.
Bayesian optimisation tackles uncertain inputs in noisy function evaluations.
problem Optimizing functions with uncertain query locations and noisy outcomes.
method Proposes a UCB algorithm for BO with uncertain inputs, using Gaussian process models.
result Theoretical and experimental results show the proposed algorithm outperforms conventional methods.
Bayesian optimisation is improved by incorporating expert prior through space warping.
problem Cold start phase in expensive function optimisation.
method Prior distribution warps the search space around high probability regions of function optimum.
result Improves optimisation performance through acquisition agnostic approach.
SOBER optimizes and quadrates efficiently in parallel for diverse tasks.
problem Scalability of batch Bayesian optimization and quadrature for expensive functions.
method Reformulates batch selection as a quadrature problem, balancing exploitation and exploration.
result SOBER outperforms 11 baselines on 12 tasks.
Bayesian optimisation has gained great popularity as a tool for optimising the parameters of machine learning algorithms and models. Somewhat ironically, setting up the hyper-parameters of Bayesian optimisation methods is notoriously hard. While reasonable practical solutions have been advanced, they can often fail to …
Bayesian optimization guided by experimenter intuition and beliefs.
problem Finding optimal functions with experimenter's beliefs incorporated.
method Sequential Subspace Search using Gaussian Process.
result Algorithm converges in sub-linear time with finite effective dimension.
BOP-Elites uses Bayesian Optimisation for QD search, improving efficiency and insight.
problem Finding diverse high-performing points from an objective function.
method Bayesian Optimisation and Gaussian Processes to model quality and diversity.
result Significantly more sample efficient and better at identifying niche solutions.
GACBO optimizes unknown causal graphs with interventions.
problem Optimizing a target variable on an unknown causal graph with interventions.
method Graph Agnostic Causal Bayesian Optimisation (GACBO) seeks to balance exploitation and exploration of causal structures and functions.
result GACBO outperforms baselines in simulated and real-world applications.
Bayesian optimisation framework considers only variable orderings to handle ill-conditioned objectives.
problem Bayesian optimisation fails with ill-conditioned or discontinuous objectives.
method Proposes a new framework that considers only the ordering of variables in input and output spaces, fitting a Gaussian process in a latent space.
result Proves optimal performance under the measure of regret for an optimistic strategy in the latent space.
New algorithm optimizes PAC-Bayes bound without surrogate loss.
problem Mismatch between optimisation objective and generalisation bound in stochastic neural networks.
method Proposes a novel training algorithm that optimizes the PAC-Bayesian bound directly.
result Empirical results show improved performance over existing PAC-Bayesian training methods.
New approach separates VAE and GP for better molecular optimisation.
problem Optimizing complex structured domains like molecular spaces using VAEs.
method Decouples VAE for structure generation and GP for predictive modelling, combining them with a Bayesian update rule.
result Improves identification of high-potential candidates in molecular optimisation.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
Bayesian method finds robust optima in expensive black-box functions.
problem Optimizing expensive black-box functions with sensitivity to inputs.
method Bayesian optimisation using Gaussian process prior and evolutionary algorithm for sampling and evaluation.
result Locating a region of design space with relatively insensitive performance to inputs.
Dragonfly optimizes complex functions without needing grad students.
problem Challenging real-world optimisation tasks where conventional BO methods fail.
method Scalable and robust Bayesian Optimisation with multiple advanced methods.
result Significant improvements in BO performance when integrating new methods.
Cost-aware multi-objective Bayesian optimization for non-uniformly expensive functions.
problem Non-uniform cost of function evaluations in Bayesian optimization.
method Introduces cost-aware constraints and a new acquisition function to optimize multi-objective functions with varying costs.
result Demonstrates improved optimization in hyperparameter tuning of neural networks and random forests.
This paper explores optimising acquisition functions in Bayesian optimisation.
problem Optimising acquisition functions in Bayesian optimisation is challenging due to their non-convex nature.
method The authors derive compositional forms for acquisition functions and use them to recast maximisation as a compositional optimisation problem.
result The compositional approach to maximising acquisition functions shows empirical advantages across various tasks.
Bayesian optimisation outperforms standard ML-II in small samples.
problem Standard ML-II fails in small-sample trials.
method Adopting fully Bayesian optimisation (FBO) as an alternative.
result FBO is more robust and practical than ML-II.
A new method learns priors for Bayesian optimisation to improve performance.
problem Bayesian optimisation tasks often assume strong similarity, which is violated in many cases.
method Replace strong similarity assumption with shape similarity, learn priors for hyperparameters.
result PLeBO and prior transfer find good inputs in fewer evaluations.
Paper improves robustness of Optimisation Monte Carlo method.
problem Overconfident approximations in Optimisation Monte Carlo.
method Robust Optimisation Monte Carlo (Robust OMC) method.
result Corrects overconfident approximations by collapsing regions of similar likelihood.
Proposes PE-GP-UCB for time-varying Bayesian optimisation.
problem Time-varying Gaussian process bandits with unknown prior.
method PE-GP-UCB algorithm, relying on consistency of function values with priors.
result Regret bound provided for the proposed algorithm.
Wind farm layout optimisation tackles space constraints with Bayesian multi-objective approach.
problem Optimizing wind farm layout due to limited space and conflicting objectives.
method Set-based multi-objective Bayesian optimisation using Gaussian process.
result Demonstrates potential of set-based Bayesian multi-objective optimisation for wind farm layout.
This paper proposes using neural networks for Bayesian optimisation in machine learning.
problem Efficiently choosing the best model and its hyperparameters in machine learning applications.
method Uses neural networks to model distributions over functions, reformulating density-ratio estimation for approximate inference.
result Demonstrates the efficiency and tractability of using neural networks in Bayesian optimisation.
Information-theoretic Bayesian optimisation techniques have demonstrated state-of-the-art performance in tackling important global optimisation problems. However, current information-theoretic approaches require many approximations in implementation, introduce often-prohibitive computational overhead and limit the choi…
Bayesian Adam method improves neural network uncertainty estimation.
problem Improving uncertainty estimation in neural networks.
method Probabilistic interpretation of adaptive optimisation algorithms.
result Bayesian Adam method correctly relates learnt uncertainties to weights' predictive capabilities.
Unified approach to Bayesian inference with guarantees on covariance matrices.
problem Approximate Bayesian inference with PSD guarantees.
method Bayes-Newton methods extending Newton's method for optimisation.
result Novel algorithms with PSD covariance matrices.
Bayesian optimisation (BO) is a well-known efficient algorithm for finding the global optimum of expensive, black-box functions. The current practical BO algorithms have regret bounds ranging from O ( l o g N N ) \mathcal{O}(\frac{logN}{\sqrt{N}}) O ( N l o g N ) to O ( e − N ) \mathcal O(e^{-\sqrt{N}}) O ( e − N ) , where N N N is the number of evaluations. This paper exp…
Bayesian optimization adapted for experiments with changing environmental conditions.
problem Optimizing experiments influenced by uncontrollable environmental factors.
method Extends Bayesian optimization to handle both controllable and uncontrollable parameters, fitting a global surrogate model and optimizing only controllable parameters conditionally on measurements of uncontrollable variables.
result The proposed ENVBO algorithm finds solutions for the full domain of the environmental variable more efficiently and cost-effectively than traditional methods.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
New greedy algorithms improve Bayesian optimisation performance.
problem Optimizing continuous functions with exploration vs exploitation trade-offs.
method Introduced two novel ε-greedy acquisition functions and compared them with conventional methods.
result ε-greedy algorithms generally outperform conventional methods, especially in higher dimensions.
A new method optimises problems with both continuous and categorical inputs.
problem Optimising black-box problems with mixed continuous and categorical inputs.
method Continuous and Categorical Bayesian Optimisation (CoCaBO) combining multi-armed bandits and Bayesian optimisation.
result CoCaBO outperforms existing methods on synthetic and real-world tasks.
Unified Bayesian Optimisation for mixed variables improves performance.
problem Efficient optimisation of problems with both categorical and continuous variables.
method Derive value proposals from the Expected Improvement criterion to optimise both categorical and continuous variables under a single acquisition metric.
result Unified approach significantly outperforms existing methods across mixed-variable tasks.
New bounds for model generalization under deterministic gradient descent.
problem Establishing generalization bounds for models trained with gradient descent methods.
method PAC-Bayesian bounds for deterministic optimisation algorithms.
result Fully computable bounds that depend on initial distribution and Hessian.
We consider the problem of approximate Bayesian parameter inference in non-linear state-space models with intractable likelihoods. Sequential Monte Carlo with approximate Bayesian computations (SMC-ABC) is one approach to approximate the likelihood in this type of models. However, such approximations can be noisy and c…
New framework converts multi-objective to single-objective optimisation.
problem Solving multi-objective optimisation problems.
method Formalises scalarisation into mathematical framework, uses R2 utilities.
result R2 utilities are monotone and submodular, optimised by greedy algorithms.
Paper proposes a new Bayesian optimisation method to handle aleatoric uncertainty.
problem Representing and minimizing aleatoric noise in Bayesian optimisation.
method Heteroscedastic Gaussian process (GP) surrogate model with AEI and ANPEI acquisition functions.
result Improved performance on toy problems and real-world datasets compared to homoscedastic methods.
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
In-BO optimizes complex constrained domains using SIn-GP surrogate models.
problem Optimizing in complex constrained domains with irregular shapes.
method Sparse Intrinsic Gaussian Processes (SIn-GP) on manifolds with heat kernel estimation.
result In-BO outperforms traditional BO in complex constrained domains.
Unified method for balancing simulation and data collection.
problem Finding optimal solutions with uncertain inputs.
method Bayesian Information Collection and Optimisation (BICO) procedure.
result Algorithm automatically determines optimal balance between optimisation and data collection.
Bayesian optimization algorithm with preference constraints on objectives.
problem Finding Pareto-optimal solutions with user-defined preference over objectives.
method A multi-objective Bayesian optimization algorithm that incorporates user-defined preference constraints on objectives. The algorithm selects Pareto-optimal points that satisfy these constraints and uses a new acquisition function based on expected improvement in dominated hypervolume (EHI).
result The algorithm efficiently explores the Pareto front satisfying user-defined preference constraints.
New method optimises worst-case risk under model uncertainty.
problem Minimizing expected risk under posterior beliefs leads to sub-optimal decisions due to model uncertainty.
method Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS)
result Improved out-of-sample robustness in the Newsvendor problem.
Optimizes data power control in cell-free networks for better spectral efficiency.
problem Maximizing overall spectral efficiency in cell-free networks with multi-objective optimisation.
method Applied scalable multi-objective Bayesian optimisation to solve convergence-time limitations.
result Improved radio resource management in cell-free networks.
Optimising black-box functions is important in many disciplines, such as tuning machine learning models, robotics, finance and mining exploration. Bayesian optimisation is a state-of-the-art technique for the global optimisation of black-box functions which are expensive to evaluate. At the core of this approach is a G…
MEMe efficiently approximates large-scale ML problems with hundreds of moments.
problem Efficient approximation in large-scale machine learning.
method Maximum entropy algorithm with hundreds of moments for computationally efficient approximations.
result Superior to existing approaches in fast log determinant estimation and Bayesian optimisation.