arXiv research

Local Bayesian optimization shows strong performance and converges well, contrary to folklore.

We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step. This is combined with a novel stopping condition based on expected regret. This pairing allows us to obtain the best characteristics of both lo…

LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.

Differentially private hyperparameter tuning improves privacy in machine learning.

Global optimization finds applications in a wide range of real world problems. The multi-start methods are a popular class of global optimization techniques, which are based on the ideas of conducting local searches at multiple starting points. In this work we propose a new multi-start algorithm where the starting poin…

LCBO tackles constrained optimization in high dimensions, offering a polynomial convergence rate.

LOL-BO improves latent space Bayesian optimization over structured inputs.

Pseudo-Bayesian Optimization improves black-box function optimization using simple local regression.

Bayesian Optimization (BO) has become a core method for solving expensive black-box optimization problems. While much research focussed on the choice of the acquisition function, we focus on online length-scale adaption and the choice of kernel function. Instead of choosing hyperparameters in view of maximum likelihood…

Local PBO methods improve preferential BO in high-dimensional problems.

Bayesian optimization has shown to be a fundamental global optimization algorithm in many applications: ranging from automatic machine learning, robotics, reinforcement learning, experimental design, simulations, etc. The most popular and effective Bayesian optimization relies on a surrogate model in the form of a Gaus…

Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive wi…

Improves Bayesian optimization using Gaussian process Thompson sampling.

Bayesian optimization sped up to linear time.

BOinG optimizes HPO problems by focusing on promising local regions.

Novel method for high-dimensional BO using CMA to define local regions.

The paper refines NOTEARS for learning Bayesian networks, improving accuracy and efficiency.

Bayesian optimization uses triangulation candidates for better performance.

We empirically evaluate a stochastic annealing strategy for Bayesian posterior optimization with variational inference. Variational inference is a deterministic approach to approximate posterior inference in Bayesian models in which a typically non-convex objective function is locally optimized over the parameters of t…

AdaScale-TuRBO improves high-dimensional Bayesian optimization by dynamically scaling the GP lengthscale.

BALLET filters a high-confidence region of interest for Bayesian optimization.

BO limits search to lower dimensions with LGPR, improving efficiency and accuracy.

A novel machine learning optimization process coined Restrictive Federated Model Selection (RFMS) is proposed under the scenario, for example, when data from healthcare units can not leave the site it is situated on and it is forbidden to carry out training algorithms on remote data sites due to either technical or pri…

Bayesian optimization improves with nonstationary covariance functions.

Kalman filters are routinely used for many data fusion applications including navigation, tracking, and simultaneous localization and mapping problems. However, significant time and effort is frequently required to tune various Kalman filter model parameters, e.g. process noise covariance, pre-whitening filter models f…

Enhances BO in high dimensions with Newton methods.

We address challenges in collaborative black-box optimization through three frameworks.

VCSMC improves efficiency in Bayesian phylogenetic inference.

Bayesian optimization is a sample-efficient method for finding a global optimum of an expensive-to-evaluate black-box function. A global solution is found by accumulating a pair of query point and its function value, repeating these two procedures: (i) modeling a surrogate function; (ii) maximizing an acquisition funct…

New method improves multi-fidelity Bayesian optimization by accounting for local correlations and varying noise.

Paper presents a new method for optimizing hyperparameters in machine learning models.

Bayesian Algorithm Execution uses mutual information to infer properties of black-box functions efficiently.

Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.

Bayesian optimization offers a flexible framework to optimize an objective function that is expensive to be evaluated. A Bayesian optimizer iteratively queries the function values on its carefully selected points. Subsequently, it makes a sensible recommendation about where the optimum locates based on its accumulated …

CAGES optimizes expensive RL problems by efficiently learning gradients from multiple sources.

Bayesian optimization has become a fundamental global optimization algorithm in many problems where sample efficiency is of paramount importance. Recently, there has been proposed a large number of new applications in fields such as robotics, machine learning, experimental design, simulation, etc. In this paper, we foc…

Focalized GP improves Bayesian optimization for large datasets.

Simple linear models outperform complex BO methods in high dimensions.

Bayesian optimization improves policy search in reinforcement learning.

BKTF uses tensor factorization for Bayesian optimization of complex functions.

Gradient-informed BNNs improve Bayesian optimization performance.

Optical scatterometry is a method to measure the size and shape of periodic micro- or nanostructures on surfaces. For this purpose the geometry parameters of the structures are obtained by reproducing experimental measurement results through numerical simulations. We compare the performance of Bayesian optimization to …

When applying Machine Learning techniques to problems, one must select model parameters to ensure that the system converges but also does not become stuck at the objective function's local minimum. Tuning these parameters becomes a non-trivial task for large models and it is not always apparent if the user has found th…

This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES co…

Bayesian approach for estimating heterogeneous treatment effects in RDD designs.

Bayesian structure learning is the NP-hard problem of discovering a Bayesian network that optimally represents a given set of training data. In this paper we study the computational worst-case complexity of exact Bayesian structure learning under graph theoretic restrictions on the super-structure. The super-structure …

SA-REMBO adapts to nonstationary high-dimensional optimization.

A novel hierarchical Bayesian approach to Federated Learning reduces data exposure and improves convergence rates.