Max-value Entropy Search improves Bayesian optimization efficiency.
problem Expensive computation in maximizing entropy for Bayesian optimization.
method MES, a new criterion that uses information about the maximum function value.
result MES maintains or improves empirical performance while significantly reducing computational cost.
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of t…
JES optimizes expensive functions by considering joint entropy over input and output spaces.
problem Optimizing expensive functions with limited evaluations.
method Joint Entropy Search (JES) considers joint entropy over input and output spaces.
result JES outperforms other information-theoretic methods in Bayesian optimization.
The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves …
PESMOC optimizes multiple expensive functions with constraints using entropy reduction.
problem Simultaneous optimization of multiple expensive functions with constraints.
method Iterative entropy reduction strategy based on predictive entropy search.
result PESMOC provides better recommendations with fewer evaluations than random search.
Bayesian search optimizes exploration of feasible solutions under expensive constraints.
problem Identifying feasible solutions in computationally expensive constraint spaces.
method Bayesian models with an acquisition function for efficient exploration and exploitation.
result The proposed acquisition function improves the prediction of feasibility.
VES-Gamma adapts EI using information-theoretic principles.
problem Optimizing black-box functions using Bayesian optimization.
method Variational Entropy Search (VES) and VES-Gamma algorithm.
result VES-Gamma improves EI by incorporating information-theoretic concepts.
Extends entropy search for more efficient robotic movement adaptation.
problem Efficiently learning and adapting robotic movement to different conditions.
method Active contextual policy search using entropy search.
result Learning successful behavior with fewer trials.
Protein Thoughts interprets protein interactions with clear reasoning, improving prediction accuracy.
problem Lack of mechanistic justification in protein-protein interaction predictions.
method Interpretable search problem reformulation, hypothesis-guided entropy-regularized Tree-of-Thoughts search, embedding-space flow matching.
result Improves mean best-binder rank from 47.7 to 11.2 on SHS148k benchmark.
PFES uses entropy of Pareto-frontier for multi-objective Bayesian optimization.
problem Bayesian optimization for multi-objective problems, especially trade-off among objectives.
method Pareto-frontier entropy search (PFES) incorporating trade-off relation.
result PFES effectively incorporates dependency among objectives conditioned on Pareto-frontier.
Trust-region methods and natural gradients are equivalent in certain policy search scenarios.
problem Improving policy search methods in continuous control tasks.
method Introducing compatible policy search (COPOS) that uses natural parameterization and compatible value function approximation to control entropy loss.
result COPOS yields state-of-the-art results in challenging tasks and reduces entropy loss.
A new method for multi-objective Bayesian optimization using entropy search and variational lower bound maximization.
problem Efficiently optimizing multiple objectives in continuous domains.
method Approximates the Pareto-frontier using a mixture distribution and optimizes the balance through variational lower bound maximization.
result Demonstrated effectiveness especially with many objective functions.
Paper optimizes energy-based controller for swinging up a pendulum using entropy search.
problem Finding optimal parameters for energy-based controllers is hard.
method Bayesian optimization (Entropy Search) applied to energy-based controller design.
result Optimal controller improves performance of a swinging pendulum.
Parallel BO method for multi-objective optimization with constraints.
problem Optimizing multiple objectives under constraints with expensive evaluations.
method PPESMOC, a batch method for simultaneous optimization of black-box functions.
result Empirical evidence shows PPESMOC is effective for multi-objective optimization with constraints.
New algorithms predict reinforcement learning values efficiently.
problem Predicting reinforcement learning values with linear function approximation.
method Multi-timescale stochastic approximation of cross entropy method.
result Proved convergence and achieved good performance in experiments.
Unified framework connects EI and information-theoretic acquisition functions.
problem Distinguish between Expected Improvement and information-theoretic acquisition functions.
method Introduces Variational Entropy Search (VES) to unify EI and information-theoretic approaches.
result EI can be seen as a variational inference approximation of Max-value Entropy Search (MES).
A new method for constrained Bayesian optimization using Max-Value Entropy Search.
problem Optimizing expensive functions with unknown constraints.
method Constrained Max-value Entropy Search (cMES), a novel acquisition function.
result cMES outperforms prior work on constrained hyperparameter optimization problems.
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
problem Reward-free learning in high-dimensional, continuous-control domains.
method Maximum Entropy POLicy optimization (MEPOL) algorithm that maximizes a non-parametric state entropy estimate.
result MEPOL learns a maximum-entropy exploration policy that facilitates learning various reward-based tasks.
PPES selects batches of points for parallel optimization of expensive functions.
problem Optimizing expensive, black-box functions efficiently.
method Parallel Predictive Entropy Search (PPES) for batch Bayesian optimization.
result PPES outperforms greedy batch selection in optimization performance.
EENMF improves e-commerce sponsored search efficiency and effectiveness.
problem Improving efficiency and effectiveness of e-commerce sponsored search.
method End-to-end neural matching framework (EENMF) for vector-based ad retrieval and neural pre-ranking.
result Significantly outperforms baseline in real e-commerce traffic.
Paper introduces MPES for top-k ranking BO with preferential observations.
problem Handling top-k ranking and tie/indifference observations in Bayesian optimization.
method Designs a surrogate model and introduces MPES acquisition function.
result MPES outperforms existing acquisition functions in handling preferential observations.
Study improves materials discovery for high-entropy alloys using sparse linear models.
problem Inefficient materials discovery due to combinatorial explosion in alloy compositions.
method Sparse mixed linear modeling with anchor-based guidance for feature selection and prediction.
result Developed a method that balances predictive performance and interpretability for materials discovery.
LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.
problem Optimizing large, complex design spaces is infeasible and unnecessary.
method LES uses Bayesian optimization to target solutions reachable by iterative optimizers.
result LES achieves strong sample efficiency compared to existing methods.
Unknown constraints arise in many types of expensive black-box optimization problems. Several methods have been proposed recently for performing Bayesian optimization with constraints, based on the expected improvement (EI) heuristic. However, EI can lead to pathologies when used with constraints. For example, in the c…
A new algorithm learns MAGs from data more efficiently using entropy.
problem Learning MAGs from data is unstable and computationally expensive.
method Uses entropy estimation and refined Markov property to score MAGs.
result Algorithm is polynomial in number of nodes and outperforms existing methods.
Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
problem Bayesian Optimization
method Bayesian Optimization with Conditional Diffusion Models
result DMS outperforms standard BO baselines
RES improves robustness in Bayesian optimization.
problem Finding robust solutions in Bayesian optimization with adversarial perturbations.
method Robust Entropy Search (RES) acquisition function.
result RES reliably finds robust optima, outperforming state-of-the-art algorithms.
A method for policy search with high-dimensional context variables.
problem Learning from high-dimensional context variables like camera images is challenging.
method Model-based relative entropy stochastic search framework with integrated dimensionality reduction.
result The proposed method outperforms naive dimensionality reduction methods.
A framework for constrained Bayesian optimization using information-based search.
problem Efficiently solving global black-box optimization problems with constraints.
method Information-theoretic framework and Predictive Entropy Search with Constraints (PESC).
result PESC is an effective algorithm that provides a promising direction for constrained Bayesian optimization.
A framework for efficient multi-objective optimization using entropy search.
problem Optimizing expensive black-box functions with multiple objectives.
method Output space entropy search (OSE) to minimize resource cost.
result Improves efficiency and accuracy in multi-objective optimization.
Hybridizes CEM and gradient descent for efficient model-predictive control.
problem Efficiently planning optimal action sequences in high-dimensional spaces.
method Interleaves Cross-Entropy Method (CEM) and gradient descent steps.
result Faster convergence and avoidance of local optima compared to CEM.
MESMOC optimizes constrained multi-objective problems efficiently.
problem Constrained multi-objective optimization with expensive function evaluations.
method Max-value Entropy Search in the output space.
result MESMOC selects high-quality Pareto solutions efficiently.
NES improves robust optimization with noisy inputs.
problem Finding robust optima in problems with input and measurement noise.
method Noisy-Input Entropy Search (NES) acquisition function based on Gaussian process modeling.
result NES reliably finds robust optima, outperforming existing methods.
Statistical mechanics helps understand sparse linear regression limits.
problem Understanding limits of sparse linear regression solutions.
method Replica method from statistical mechanics.
result Wide parameter region where local search algorithms can find ground state.
Improves neural network search in combinatorial spaces of mathematical symbols.
problem Early commitment and initialization bias limit exploration in neural network search.
method Entropy regularization and distribution initialization methods.
result Improves performance, increases sample efficiency, lowers solution complexity.
Deep CNN improves search recall for eBay categories.
problem Improving search recall for eBay classifieds platforms.
method Transform queries to numeric vectors, apply fully-connected deep CNN for classification.
result Achieves very high classification accuracy compared to other techniques.
Improved MESMOC+ optimizes constrained multi-objective problems efficiently.
problem Optimizing constrained multi-objective problems with expensive evaluations.
method Minimizes entropy of Pareto frontier to guide search, using linear cost and decoupled evaluation.
result Significantly faster than alternatives, with more accurate entropy estimation.
A new acquisition function RMES improves Bayesian optimization performance.
problem Improper evaluation of mutual information in MES leads to suboptimal performance.
method Developed rectified MES (RMES) and used stochastic gradient ascent with reparameterization.
result RMES shows consistent improvement over MES in benchmarks and real-world problems.
DE-QT detects optimal Q-learning stopping points.
problem Information loss in Q-learning during prolonged training.
method Introducing DE-QT to detect entropy changes in Q-tables.
result DE-QT identifies the best stopping point for Q-learning.
Bayesian optimization method predicts high costs for unstable robot controllers.
problem Time-consuming and challenging learning robot controllers with unknown penalties.
method Proposes a Bayesian model that predicts high costs in unstable regions.
result Improves robot learning by guiding exploration toward stable regions.
This paper distills Bayesian posterior expectations for deep neural networks.
problem Improving deep neural network performance and uncertainty quantification.
method Develops a framework for distilling expectations from Bayesian posterior distributions using Monte Carlo samples.
result The framework successfully distills posterior predictive distribution and expected entropy.
This paper generalizes BO uncertainty measures using decision-theoretic entropies.
problem Efficiently inferring optima of expensive black-box functions.
method Introduces a generalized entropy measure from statistical decision theory to optimize Bayesian optimization.
result Demonstrates strong empirical performance across various sequential decision-making tasks.
ESPRESSO segments time-series data for better human activity recognition.
problem Segmenting high-dimensional time-series data for applications like HAR.
method ESPRESSO combines entropy and shape analysis for multi-dimensional time-series segmentation.
result ESPRESSO outperforms four state-of-the-art methods across seven datasets.
Framework uses optimal transport for neural architecture search.
problem Optimizing neural architectures in deep learning.
method Semi-discrete optimization using optimal transport.
result Gradient flow and minimizing movement scheme converge to reaction-diffusion equations.
Ancestry improves genealogy search results by ranking diverse record types.
problem Ranking diverse genealogy records equitably from various sources.
method Customized Coordinate Ascent, Stochastic Search, Normalized Cumulative Entropy.
result Demonstrated effectiveness of algorithms in improving relevance and diversity.
A new method for faster expensive function evaluations using multi-fidelity Bayesian optimization.
problem Expensive function evaluations in Bayesian optimization.
method Max-value entropy search (MES) for multi-fidelity Bayesian optimization (MFBO), with parallelization.
result Significant reduction in computational complexity for multi-fidelity Bayesian optimization.
PESMO optimizes multi-objective Bayesian optimization by reducing entropy.
problem Efficiently identifying the Pareto set of multi-objective optimization problems with expensive evaluations.
method PESMO uses entropy minimization to select evaluation points, decomposing the multi-objective acquisition function into objective-specific components.
result PESMO achieves better recommendations with fewer evaluations compared to other methods.
Saha seeks physics models to manage income inequality.
problem Minimizing income inequality in society.
method Develops statistical physics models to constrain income distribution entropy.
result Models aim to reduce income inequality.