Paper derives a simplified formula for Expected Improvement using log-transformed data.
problem Challenges in enhancing Bayesian optimization with Expected Improvement.
method Derives a closed form of Expected Improvement for Gaussian process trained on log-transformed objective.
result Provides a simplified formula for Expected Improvement.
Expected signatures map data streams to lower dimensions, improving ML performance.
problem Leveraging model-free embeddings for domain-agnostic machine learning.
method Expected signatures map data streams to lower dimensions, with convergence results bridging empirical and theoretical estimators.
result A modified expected signature estimator with lower mean squared error for martingale processes.
New scalarizing functions improve multi-objective Bayesian optimisation.
problem Improving multi-objective Bayesian optimisation efficiency.
method Comparing two infill criteria based on hypervolume improvement.
result Effective scalarizing functions enhance hypervolume maximisation.
Paper analyzes convergence rate of noisy Bayesian Optimization with Expected Improvement.
problem Theoretical convergence behaviors and rates of Expected Improvement (EI) in Bayesian optimization.
method Analyzes Expected Improvement (EI) under Gaussian process (GP) prior assumption, considering noisy observations.
result Established asymptotic error bound and rate for GP-EI with noisy observations.
State-level minimum Bayes risk (sMBR) training has become the de facto standard for sequence-level training of speech recognition acoustic models. It has an elegant formulation using the expectation semiring, and gives large improvements in word error rate (WER) over models trained solely using cross-entropy (CE) or co…
We propose an extension of the concept of Expected Improvement criterion commonly used in Kriging based optimization. We extend it for more complex Kriging models, e.g. models using derivatives. The target field of application are CFD problems, where objective function are extremely expensive to evaluate, but the theor…
LogEI improves Bayesian optimization by simplifying numerical computation of EI and related functions.
problem Numerical pathologies in optimizing EI and related acquisition functions.
method Proposes LogEI, a family of acquisition functions that simplify numerical optimization.
result LogEI members improve optimization performance and match or exceed state-of-the-art methods.
Since their introduction a year ago, distributional approaches to reinforcement learning (distributional RL) have produced strong results relative to the standard approach which models expected values (expected RL). However, aside from convergence guarantees, there have been few theoretical results investigating the re…
New method optimizes costly functions with unknown costs and budget constraints.
problem Optimizing functions with unknown and heterogeneous evaluation costs under a budget constraint.
method Budgeted multi-step expected improvement acquisition function.
result Our method outperforms existing approaches in various synthetic and real problems.
Posterior sampling-based EI achieves sublinear regret bounds for expensive function optimization.
problem Theoretical analysis of expected improvement (EI) in Bayesian optimization.
method Randomized posterior sampling of EI.
result Achieves sublinear Bayesian cumulative regret bounds.
A new method for high-dimensional Bayesian optimization.
problem Challenges in extending BO to high dimensions.
method Expected Coordinate Improvement (ECI) criterion for high-dimensional Bayesian optimization.
result Significantly better results than standard BO and competitive results with state-of-the-art methods.
CEI achieves convergence rates for constrained Bayesian optimization.
problem Constrained Bayesian optimization with theoretical convergence rates.
method Analyzing simple regret upper bound for CEI in RKHS and Gaussian process settings.
result CEI achieves convergence rates of t−21log2d+1(t) and t2ν+d−νlog2ν+dν(t) for squared exponential and Matérn kernels, respectively. This work deals with parallel optimization of expensive objective functions which are modeled as sample realizations of Gaussian processes. The study is formalized as a Bayesian optimization problem, or continuous multi-armed bandit problem, where a batch of q > 0 arms is pulled in parallel at each iteration. Several a…
The expected improvement (EI) algorithm is a popular strategy for information collection in optimization under uncertainty. The algorithm is widely known to be too greedy, but nevertheless enjoys wide use due to its simplicity and ability to handle uncertainty and noise in a coherent decision theoretic framework. To pr…
A new parallel BO method with exact gradients for multi-objective optimization.
problem Efficiently optimizing multiple objectives in a sample-efficient manner.
method Derive q-Expected Hypervolume Improvement (qEHVI) for parallel, constrained evaluation.
result qEHVI is computationally tractable and outperforms state-of-the-art methods.
VES-Gamma adapts EI using information-theoretic principles.
problem Optimizing black-box functions using Bayesian optimization.
method Variational Entropy Search (VES) and VES-Gamma algorithm.
result VES-Gamma improves EI by incorporating information-theoretic concepts.
In this paper, we firstly give a brief introduction of expectation maximization (EM) algorithm, and then discuss the initial value sensitivity of expectation maximization algorithm. Subsequently, we give a short proof of EM's convergence. Then, we implement experiments with the expectation maximization algorithm (We im…
Optimizes target value in stochastic black box functions.
problem Finding input to minimize expected squared error to target value.
method Derives acquisition functions for expected improvement, probability of improvement, and lower confidence bound, assuming Gaussian aleatoric effects.
result Acquisition functions can outperform classical Bayesian optimization under certain conditions.
New methods improve uncertainty in machine learning predictions for asset returns.
problem Uncertainty in machine learning predictions for asset returns.
method Developed new methods to construct forecast confidence intervals for expected returns from neural networks.
result Neural network forecasts of expected returns have the same asymptotic distribution as classic nonparametric methods, enabling standard error calculation.
A new EM algorithm improves inference from large datasets.
problem Efficient inference in latent variable models with large datasets.
method Introduces SPIDER-EM, a novel EM algorithm using SPIDER estimator.
result Finite-time complexity bounds for smooth non-convex likelihood.
We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multiv…
We consider optimization of composite objective functions, i.e., of the form f(x)=g(h(x)), where h is a black-box derivative-free expensive-to-evaluate function with vector-valued outputs, and g is a cheap-to-evaluate real-valued function. While these problems can be solved with standard Bayesian optimization, we…
Generative AI predicts economic activity from corporate transcripts.
problem Predicting economic activity using existing measures like surveys.
method Extracted managerial expectations from transcripts using generative AI.
result AI Economy Score predicts economic activity up to 10 quarters ahead.
Paper proposes a new MIMO detection algorithm using Gaussian Mixture Expectation Propagation.
problem Challenges in MIMO detection due to interference and noise in high-order high-dimensional systems.
method The approach uses a Gaussian Mixture Model (GMM) approximation for Belief Propagation (BP) and Expectation Propagation (EP) messages to improve detection accuracy.
result The proposed algorithm outperforms state-of-the-art detection algorithms while maintaining low computational complexity.
This paper optimizes Bayesian acquisition functions in Gaussian Processes for better optimization.
problem Improving the efficiency of Bayesian optimization methods.
method Analysis of different acquisition functions and optimizers for optimizing Bayesian acquisition functions.
result Optimization of acquisition functions leads to faster and more accurate sampling points.
Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.
problem Analyzing cumulative regret of GP-EI with different incumbents in noisy Bayesian optimization.
method Analyzes GP-EI with three incumbents (BPMI, BSPMI, BOI) in both SE and Matérn kernels, proving no-regret for BPMI and BSPMI.
result GP-EI with BPMI and BSPMI is a no-regret algorithm for both SE and Matérn kernels, providing theoretical guidance for choosing incumbents.
Paper analyzes nonconvex bandit problems with improved adaptive methods.
problem Continuous armed bandit problems for nonconvex cost functions.
method Simple and adaptive bin splitting methods.
result Adaptive method achieves locally minimax optimal expected cumulative regret.
Study improves accuracy of risk measures using advanced algorithms.
problem Computing accurate risk measures for financial losses.
method Nested stochastic approximation and multilevel acceleration.
result Established central limit theorems for estimation errors.
Develops a new option pricing model under G-expectation framework.
problem Modeling uncertainty in financial markets and robust valuation under model uncertainty.
method G-expectation framework, logarithmic transformation, finite difference schemes.
result Unified risk-neutral valuation approach yielding G-Black-Scholes equation.
We revisit the challenge of designing online algorithms for the bandit convex optimization problem (BCO) which are also scalable to high dimensional problems. Hence, we consider algorithms that are \textit{projection-free}, i.e., based on the conditional gradient method whose only access to the feasible decision set, i…
State-of-the-art forecasting methods using Recurrent Neural Net- works (RNN) based on Long-Short Term Memory (LSTM) cells have shown exceptional performance targeting short-horizon forecasts, e.g given a set of predictor features, forecast a target value for the next few time steps in the future. However, in many appli…
This paper analyzes regret bounds for Gaussian process Thompson sampling.
problem Analyzing the performance of Gaussian process Thompson sampling (GP-TS) in Bayesian optimization.
method The paper derives several regret bounds for GP-TS, including a lower bound, upper bounds on the second moment of cumulative regret, expected lenient regret, and improved cumulative regret.
result The paper provides improved regret upper bounds for GP-TS, showing that it suffers from a polynomial dependence on 1/δ with probability δ. Binary classification models get more efficient predictive probabilities.
problem Computing predictive probabilities in Bayesian probit models is computationally challenging.
method Use of expectation propagation (EP) to find a closed-form expression for predictive probabilities.
result Closed-form predictive probabilities improve over existing methods.
New EP variants improve inference stability and efficiency.
problem Inference stability and efficiency issues in EP.
method Motivated by natural-gradient optimization, new EP variants are introduced that are robust to Monte Carlo noise and efficient with single samples.
result Improved stability and efficiency in inference tasks.
We present an objective function for learning with unlabeled data that utilizes auxiliary expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (…
A new method extends Bayesian optimization to more models and utilities.
problem Extending Bayesian optimization to a broader class of models and utilities.
method Likelihood-free Bayesian Optimization (LFBO) which directly models the acquisition function without separate inference.
result LFBO outperforms state-of-the-art black-box optimization methods on real-world problems.
Unified framework connects EI and information-theoretic acquisition functions.
problem Distinguish between Expected Improvement and information-theoretic acquisition functions.
method Introduces Variational Entropy Search (VES) to unify EI and information-theoretic approaches.
result EI can be seen as a variational inference approximation of Max-value Entropy Search (MES).
MLMC boosts Bayesian optimization's look-ahead efficiency.
problem Efficiently computing nested expectations in Bayesian optimization.
method Multilevel Monte Carlo (MLMC) for nested operations.
result MLMC achieves MC convergence rate for nested operations, improving BO performance.
We present an adaptive approach to the construction of Gaussian process surrogates for Bayesian inference with expensive-to-evaluate forward models. Our method relies on the fully Bayesian approach to training Gaussian process models and utilizes the expected improvement idea from Bayesian global optimization. We adapt…
Much recent research has been conducted in the area of Bayesian learning, particularly with regard to the optimization of hyper-parameters via Gaussian process regression. The methodologies rely chiefly on the method of maximizing the expected improvement of a score function with respect to adjustments in the hyper-par…
Paper proposes using tree-based surrogate models for efficient Shapley computation.
problem Efficient computation of Shapley values using conditional expectations.
method Surrogate model-based tree for approximating Shapley and SHAP values.
result The proposed algorithm improves accuracy and unifies global interpretation.
Efficient global optimization is the problem of minimizing an unknown function f, using as few evaluations f(x) as possible. It can be considered as a continuum-armed bandit problem, with noiseless data and simple regret. Expected improvement is perhaps the most popular method for solving this problem; the algorithm pe…
Efficient EP algorithm improves smoothing distribution inference in financial models.
problem Computational intractability of smoothing distribution in high dimensions.
method Adapted expectation propagation (EP) algorithms for the unified skew-normal family.
result Accuracy gains in financial illustrations over existing approximate algorithms.
New bounds for transfer learning in linear models, improving generalization.
problem Understanding when auxiliary data helps in improving generalization in linear models.
method Derivation of exact error bounds and optimal task weights for linear regression and linear neural networks.
result First non-vacuous sufficient conditions for beneficial auxiliary learning in linear neural networks.
A corrected EI acquisition function handles noisy observations in Bayesian optimization.
problem Noisy observations in Bayesian optimization.
method Proposes a modified expected improvement (EI) acquisition function that incorporates covariance information from the Gaussian Process model.
result Achieves a sublinear convergence rate on cumulative regret bound under heteroscedastic observation noise.
Introduces expected eligibility traces for more efficient credit assignment in reinforcement learning.
problem Efficiently assigning credit to states and actions in reinforcement learning.
method Introduces expected eligibility traces, allowing updates to counterfactual sequences.
result Substantial improvements in temporal-difference learning can be achieved with expected traces.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
CAESar improves risk forecasting by combining VaR and ES estimates.
problem Lack of tail risk measures in financial risk management.
method Conditional Autoregressive Expected Shortfall model, combining VaR and ES estimates.
result CAESar outperforms existing methods in risk forecasting.