Bayesian optimization is a sample-efficient approach to solving global optimization problems. Along with a surrogate model, this approach relies on theoretically motivated value heuristics (acquisition functions) to guide the search process. Maximizing acquisition functions yields the best performance; unfortunately, t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “acquisition functions”
This paper optimizes Bayesian acquisition functions in Gaussian Processes for better optimization.
problem Improving the efficiency of Bayesian optimization methods.
method Analysis of different acquisition functions and optimizers for optimizing Bayesian acquisition functions.
result Optimization of acquisition functions leads to faster and more accurate sampling points.
Bayesian optimization is a sample-efficient approach to global optimization that relies on theoretically motivated value heuristics (acquisition functions) to guide its search process. Fully maximizing acquisition functions produces the Bayes' decision rule, but this ideal is difficult to achieve since these functions …
This paper explores optimising acquisition functions in Bayesian optimisation.
problem Optimising acquisition functions in Bayesian optimisation is challenging due to their non-convex nature.
method The authors derive compositional forms for acquisition functions and use them to recast maximisation as a compositional optimisation problem.
result The compositional approach to maximising acquisition functions shows empirical advantages across various tasks.
Inexact acquisition solutions in BO lead to sublinear cumulative regret.
problem Inexact maximization of acquisition functions in Bayesian optimization.
method Define inaccuracy measure, establish cumulative regret bounds for GP-UCB and GP-TS.
result Inexact BO algorithms can achieve sublinear cumulative regret under appropriate inaccuracy conditions.
Unified framework connects EI and information-theoretic acquisition functions.
problem Distinguish between Expected Improvement and information-theoretic acquisition functions.
method Introduces Variational Entropy Search (VES) to unify EI and information-theoretic approaches.
result EI can be seen as a variational inference approximation of Max-value Entropy Search (MES).
Proposes a new acquisition function for batched Bayesian optimization.
problem Intractability of acquisition functions for batched Bayesian optimization.
method Statistical physics inspired acquisition function for Gaussian processes.
result Demonstrates competitive performance on various problems.
This paper optimizes kernel and acquisition functions for high-dimensional Bayesian Optimization.
problem Bayesian Optimization struggles with high-dimensional problems due to computational inefficiency.
method The paper leverages the additionality of the objective function to map kernel and acquisition functions in lower-dimensional subspaces, improving efficiency.
result Efficient optimization of acquisition function in high-dimensional problems.
Bayesian optimization is a sample-efficient method for finding a global optimum of an expensive-to-evaluate black-box function. A global solution is found by accumulating a pair of query point and its function value, repeating these two procedures: (i) modeling a surrogate function; (ii) maximizing an acquisition funct…
This paper optimizes sampling policies for Bayesian optimization to improve exploration and exploitation.
problem Improving the balance between exploration and exploitation in Bayesian optimization.
method Developed efficient methods to estimate and optimize non-myopic acquisition functions using rollout policies and stochastic gradient optimization.
result Efficient optimization of sampling policies leads to better performance in Bayesian optimization.
New acquisition function improves batch Bayesian active learning.
problem BatchBALD conflates epistemic and aleatoric uncertainty, leading to suboptimal performance.
method Focus on predictive probabilities to separate epistemic uncertainty, leading to better performance and faster evaluation.
result The new acquisition function performs better and allows for larger batches.
We present Acquisition Thompson Sampling (ATS), a novel technique for batch Bayesian Optimization (BO) based on the idea of sampling multiple acquisition functions from a stochastic process. We define this process through the dependency of the acquisition functions on a set of model hyper-parameters. ATS is conceptuall…
Bayesian optimization (BO) is a class of sample-efficient global optimization methods, where a probabilistic model conditioned on previous observations is used to determine future evaluations via the optimization of an acquisition function. Most acquisition functions are myopic, meaning that they only consider the impa…
An adaptive dropout approach improves high-dimensional Bayesian optimization.
problem High-dimensional black-box optimization problems.
method Adaptive dropout of variables in the acquisition function.
result AdaDropout effectively tackles high-dimensional challenges and improves solution quality.
We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multiv…
Bayesian optimization surveys information-theoretic acquisition functions.
problem Optimizing noisy, expensive, non-convex functions with unknown gradients.
method Bayesian optimization using Gaussian process surrogate models and information-theoretic acquisition functions.
result Information-theoretic acquisition functions outperform others in real scenarios.
Optimizes data acquisition in high-dimensional Bayesian optimization.
problem Suboptimal data acquisition in high-dimensional Bayesian optimization tasks.
method Utility-calibrated variational inference to align approximations with BO goals.
result Optimal data acquisition decisions under a limited computational budget.
A simple method improves batch active learning without high compute.
problem Efficient batch active learning in machine learning.
method Adapting standard single-point acquisition strategies to batch.
result Simple strategy performs as well as advanced batch methods.
Optimizes information acquisition to reduce estimation risk and maximize utility.
problem Estimation risk in investor decision-making.
method Derives closed-form value functions using CARA and CRRA utility functions, employs variational methods to explore optimal acquisition.
result Acquiring information earlier is more valuable in reducing estimation risk and achieving higher utility.
BOOST automates kernel and acquisition function selection in Bayesian optimization.
problem Inappropriate kernel and acquisition function combinations lead to poor performance in Bayesian optimization.
method BOOST uses offline evaluation to predict and select the best kernel-acquisition function pair.
result BOOST consistently improves over fixed-hyperparameter BO and is competitive with state-of-the-art adaptive methods.
qEUBO optimizes decision-making with noisy feedback.
problem Optimizing decision-making with noisy preference feedback.
method Introduces qEUBO as a novel acquisition function for preferential Bayesian optimization.
result qEUBO is one-step Bayes optimal and enjoys an approximation guarantee under noise.
Bayesian optimization methods improved for min max optimization problems.
problem Min-max optimization for unknown functions.
method Extended Bayesian optimization to min-max problems with new acquisition functions.
result Improved acquisition functions lead to better solutions.
Bayesian Optimization (BO) is a framework for black-box optimization that is especially suitable for expensive cost functions. Among the main parts of a BO algorithm, the acquisition function is of fundamental importance, since it guides the optimization algorithm by translating the uncertainty of the regression model …
New acquisition functions improve Bernoulli LSE.
problem Efficiently estimating regions where a Bernoulli function is above or below a threshold.
method Developed new look-ahead acquisition functions for Gaussian process classification models.
result Demonstrated clear benefits of new acquisition functions on benchmark and real-world tasks.
A corrected EI acquisition function handles noisy observations in Bayesian optimization.
problem Noisy observations in Bayesian optimization.
method Proposes a modified expected improvement (EI) acquisition function that incorporates covariance information from the Gaussian Process model.
result Achieves a sublinear convergence rate on cumulative regret bound under heteroscedastic observation noise.
GOIMDA selects inputs to maximize expected influence on a goal functional, reducing data acquisition needs.
problem Challenges in active data acquisition for learning and optimization tasks in deep neural networks.
method GOIMDA uses inverse curvature and goal gradient to select inputs maximizing expected influence on a specified goal functional.
result GOIMDA achieves target performance with fewer labeled samples or function evaluations compared to baselines.
Boundary effects inflate variance in Gaussian processes, leading to acquisition bias.
problem Boundary-induced acquisition bias in Gaussian processes.
method Traced root cause to geometric mechanism of kernel truncation at domain boundaries.
result Boundary effects create distortion that worsens with dimensionality, affecting acquisition behavior.
New method tackles constrained optimization in multi-fidelity Bayesian optimization.
problem Efficiently identifying feasible regions in constrained optimization problems.
method Proposes CMFBO method with novel acquisition functions.
result Demonstrates effectiveness on synthetic problems and real-world ICF and joint design problems.
Simulated annealing improves candidate optimization for multi-objective Bayesian optimization.
problem Efficient candidate optimization for multi-objective acquisition functions in Bayesian optimization.
method Simulated annealing-based approach for batch acquisition function optimization.
result Simulated annealing outperforms SLSQP in most multi-objective optimization problems, achieving higher hypervolume values and better convergence characteristics.
LogEI improves Bayesian optimization by simplifying numerical computation of EI and related functions.
problem Numerical pathologies in optimizing EI and related acquisition functions.
method Proposes LogEI, a family of acquisition functions that simplify numerical optimization.
result LogEI members improve optimization performance and match or exceed state-of-the-art methods.
Proposes efficient FOBO algorithms for global maxima of expensive functions.
problem Finding global maxima of expensive-to-evaluate functions.
method Uses gradient information from Gaussian process models to identify potential query points.
result Proposed algorithms outperform state-of-the-art FOBO algorithms.
This paper proposes automatic tuning of Bayesian Optimization's acquisition function.
problem Optimizing black-box functions with noisy, expensive evaluations and hyperparameter tuning.
method Exploring heuristics to automatically tune acquisition functions in Bayesian Optimization.
result Demonstrates effectiveness of heuristics in automatic Bayesian Optimization.
A new acquisition function RMES improves Bayesian optimization performance.
problem Improper evaluation of mutual information in MES leads to suboptimal performance.
method Developed rectified MES (RMES) and used stochastic gradient ascent with reparameterization.
result RMES shows consistent improvement over MES in benchmarks and real-world problems.
A new method extends Bayesian optimization to more models and utilities.
problem Extending Bayesian optimization to a broader class of models and utilities.
method Likelihood-free Bayesian Optimization (LFBO) which directly models the acquisition function without separate inference.
result LFBO outperforms state-of-the-art black-box optimization methods on real-world problems.
BOtied optimizes multiple objectives using copulas and CDF indicators.
problem Joint optimization of multiple competing objectives.
method Proposes BOtied, a new acquisition function based on the CDF.
result BOtied outperforms state-of-the-art MOBO acquisition functions.
Efficiently optimizes constrained problems with two-step lookahead BO.
problem Optimizing constrained problems with limited computational resources.
method Two-step lookahead Bayesian optimization with inequality constraints, using a novel unbiased gradient estimator.
result Significantly improves query efficiency over previous methods.
New methods improve global optimisation for expensive functions using lookahead strategies.
problem Optimising expensive functions without gradient info in high dimensions.
method Nonmyopic acquisition strategies based on approximate dynamic programming.
result Nonmyopic methods outperform myopic approaches in various applications.
Optimizes function networks by selectively evaluating parts of the network, reducing costs.
problem Optimizing expensive function networks where full evaluations are costly.
method Knowledge gradient acquisition function for cost-aware partial evaluations.
result Outperforms existing BOFN methods and benchmarks across various problems.
FunBO uses LLMs to discover effective acquisition functions for Bayesian optimization.
problem Designing optimal acquisition functions for Bayesian optimization across diverse problems.
method FunBO leverages FunSearch, an LLM, to learn and evaluate new acquisition functions.
result FunBO discovers acquisition functions that generalize well and outperform existing methods.
πBO augments BO with user beliefs for better hyperparameter optimization.
problem BO ignores user beliefs, reducing its appeal to practitioners.
method Proposes BO, an acquisition function that incorporates user-provided prior beliefs.
result BO outperforms competing approaches and deep learning tasks.
Proposes qPO, a new acquisition strategy for batched Bayesian optimization that maximizes the probability of including the optimum.
problem Efficiently identifying top-performing compounds from a large chemical library.
method qPO (multipoint Probability of Optimality) acquisition strategy that maximizes the probability of including the true optimum.
result Empirical evidence shows that qPO is competitive with and complements other state-of-the-art methods in batched Bayesian optimization.
Model selection is treated as a standard performance boosting step in many machine learning applications. Once all other properties of a learning problem are fixed, the model is selected by grid search on a held-out validation set. This is strictly inapplicable to active learning. Within the standardized workflow, the …
We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning. BatchBALD is a greedy linear-time -approximate a…
Bayesian optimization is a sample-efficient method for black-box global optimization. How- ever, the performance of a Bayesian optimization method very much depends on its exploration strategy, i.e. the choice of acquisition function, and it is not clear a priori which choice will result in superior performance. While …
Bayesian optimization is a powerful global optimization technique for expensive black-box functions. One of its shortcomings is that it requires auxiliary optimization of an acquisition function at each iteration. This auxiliary optimization can be costly and very hard to carry out in practice. Moreover, it creates ser…
Improves graph-based active learning for non-Gaussian models.
problem Efficiently selecting data points for labeling in graph-based semi-supervised learning.
method Approximates non-Gaussian distributions, introduces rank-one update and model change acquisition function.
result Enhanced active learning for graph-based SSL under non-Gaussian models.
Novel Hilbert space Gaussian process improves sequential design accuracy and efficiency.
problem Efficiently implementing Gaussian process acquisition functions for expensive simulations.
method Proposed a truncated eigenbasis representation for closed-form evaluation of IMSE acquisition function.
result Significantly lower prediction error and reduced computation time compared to benchmarks.
Improved Thompson Sampling outperforms existing Bayesian optimization methods.
problem Thompson Sampling's performance in Bayesian optimization is suboptimal compared to other methods.
method Developed Stagger Thompson Sampler (STS), which more precisely samples the optimal arm with less computation.
result STS outperforms TS, PSS, and other acquisition methods in various optimization tasks.