MINs learn inverse mappings for high-dimensional optimization problems.
problem Data-driven optimization with high-dimensional inputs and valid subsets.
method Model Inversion Networks (MINs) learn an inverse mapping from scores to inputs.
result MINs can scale to high-dimensional input spaces and handle both offline and active data.
AIRBO optimizes robustly under uncertain inputs.
problem Challenges in Bayesian Optimization due to input uncertainty.
method AIRBO models arbitrary input uncertainty with MMD and accelerates posterior inference.
result AIRBO achieves state-of-the-art performance under various input uncertainties.
New BO method optimizes multiple objectives under input noise.
problem Optimizing multiple performance metrics in manufacturing processes subject to random input noise.
method Formalizes optimization of multivariate value-at-risk (MVaR) using random scalarizations.
result Significantly outperforms alternative methods in identifying robust designs.
Framework optimizes multiple objectives considering input uncertainty.
problem Efficiently optimizing multiple objectives with input uncertainty.
method Robust Gaussian Process model and two-stage Bayesian optimization process.
result Found a robust Pareto frontier considering input uncertainty.
NES improves robust optimization with noisy inputs.
problem Finding robust optima in problems with input and measurement noise.
method Noisy-Input Entropy Search (NES) acquisition function based on Gaussian process modeling.
result NES reliably finds robust optima, outperforming existing methods.
Transforms input design for probabilistic models into optimal control of a Hamiltonian system.
problem Designing inputs for probabilistic models with intractable posterior distributions.
method Representing posterior as Hamiltonian system trajectories, solving optimal control problem.
result Parameter posterior concentrates around true parameter values.
New method certifies neural network robustness under random input noise.
problem Certifying neural network robustness against random input noise.
method Chance-constrained optimization problem reformulated with input-output samples, convex conditions developed.
result Proposed method certifies robustness against various input noise regimes over larger uncertainty regions.
Bayesian optimization for set inputs using approximate set kernels.
problem Permutation-invariant optimization over sets with black-box functions.
method Developed a Bayesian optimization method with set kernel, efficient approximate set kernel, and constrained acquisition function.
result Our method outperforms other methods in numerical experiments.
Optimizes data collection for ranking and selection problems.
problem Identifying the best system from multiple solutions with limited data.
method Sequential sampling algorithm with MPB estimator and kernel ridge regression.
result OSAR achieves optimal sampling ratios almost surely in the limit.
FFBO optimizes functions as inputs and outputs, improving on existing BO methods.
problem Optimizing functions as both inputs and outputs in complex systems.
method Function-on-function Gaussian process (FFGP) model with a separable operator-valued kernel, scalar upper confidence bound (UCB) acquisition function, and scalable functional gradient ascent algorithm (FGA).
result FFBO outperforms existing methods in synthetic and real-world data.
New method in Bayesian optimization finds optimal inputs knowing the optimal outputs.
problem Finding optimal inputs when the optimal outputs are known in advance.
method Transform Gaussian process surrogate using known optimum output; propose two acquisition functions.
result Our approaches give quantitatively better performance than standard BO methods.
Algorithm optimally estimates linear dynamical systems with active input selection.
problem Estimating parameters of linear dynamical systems efficiently.
method Active learning with adaptive input selection.
result Finite time bound and asymptotic optimality proven.
New method certifies neural network robustness to random input noise.
problem Certifying neural networks' robustness to random input noise.
method Chance-constrained optimization problem reformulated using input-output samples.
result Certifies a uniform infinity-norm uncertainty region with a radius 50 times larger.
Convex optimization models predict outputs from inputs via optimization problems.
problem Predicting outputs from inputs using convex optimization models.
method Proposed a heuristic for learning parameters of convex optimization models from datasets.
result Demonstrated the effectiveness of the proposed method on three model classes.
LOL-BO improves latent space Bayesian optimization over structured inputs.
problem Optimizing complex functions over high-dimensional, structured search spaces.
method Adapting trust regions from high-dimensional to structured settings, using a DAE to map inputs into a latent space.
result Achieves up to 20x improvement over state-of-the-art methods.
Proposes a method to optimize budget allocation for collecting and analyzing streaming data.
problem Optimizing resource allocation for collecting and analyzing streaming data.
method Formulates optimization problems to allocate budgets for collecting input data and running simulations, characterizes asymptotic behavior of performance estimators, and develops a multi-stage simultaneous budget allocation procedure.
result Demonstrates competitive performance of the proposed procedure through numerical studies.
Optimal kernel learning improves GP regression for high-dimensional inputs.
problem High computational costs and low prediction accuracy in GP models with many inputs.
method Approximates GP covariance with a convex combination of kernel functions, identifying active variables.
result Improves prediction accuracy and correctly identifies active input variables.
HyCNNs improve convex function learning and optimal transport.
problem Learning and optimizing convex functions efficiently.
method Combining Maxout networks and ICNNs to create a new neural architecture.
result HyCNNs require fewer parameters and outperform existing methods in convex tasks.
Aims to optimize complex multivariate systems with constraints.
problem Optimizing force-field systems in physics with large-scale simulations.
method Combines machine learning and experimental design to find feasible input combinations.
result Locates multiple good regions in the input space.
This research reverses feature visualization in neural networks to optimize for specific feature objectives.
problem The invertibility of feature visualization in neural networks is not well understood.
method The approach involves optimizing for the feature objective that generates the input used in feature visualization, using the gradient of a specific objective function.
result A closed-form solution is found to minimize the gradient, providing an alternative view on network sensitivity.
JES optimizes expensive functions by considering joint entropy over input and output spaces.
problem Optimizing expensive functions with limited evaluations.
method Joint Entropy Search (JES) considers joint entropy over input and output spaces.
result JES outperforms other information-theoretic methods in Bayesian optimization.
The paper examines how neural network topology affects adversarial robustness.
problem Understanding how neural network topology influences adversarial robustness.
method Investigated the graph of input traversing all layers of a neural network, comparing clean and adversarial inputs.
result Under-optimized edges in neural network graphs are a source of adversarial vulnerability and can be used to detect adversarial inputs.
A new method reduces both input and output dimensions for better goal-oriented analysis.
problem Simultaneous reduction of input and output dimensions for more accurate analysis.
method Coupled input-output dimension reduction, optimizing gradient-based bounds.
result Determine most informative sensors and influential parameters efficiently.
New method selects inputs for Bayesian regression with few samples.
problem Optimal experimental design for high-dimensional inputs.
method Output-weighted optimal sampling using Bayesian regression.
result New criterion considers output values of existing samples.
Sparse codes improve optimal control tasks with correlated inputs.
problem Optimal control tasks with correlated feature inputs.
method Used a sparse code to represent natural images in an optimal control task solved with neuro-dynamic programming.
result An over-complete sparse code increases memory capacity and learning speed beyond a complete code.
BOSS optimizes string inputs using string kernels and genetic algorithms.
problem Optimizing string inputs with constraints.
method Bayesian optimization over string kernels and genetic algorithms.
result Significantly improved optimization across various string constraints.
Bayesian optimization improves performance with common random numbers.
problem Optimizing expensive stochastic functions with common random numbers.
method Proposes a novel Gaussian process model and Knowledge Gradient for Common Random Numbers.
result Significant performance improvements with moderate computational cost.
We consider the problem of learning a high-dimensional multi-task regression model, under sparsity constraints induced by presence of grouping structures on the input covariates and on the output predictors. This problem is primarily motivated by expression quantitative trait locus (eQTL) mapping, of which the goal is …
New attack improves on existing methods for ReLU networks.
problem Finding optimal adversarial inputs for ReLU networks is NP-hard.
method Gradient-free optimization on linear regions of ReLU networks.
result Improves over existing methods in most experiments.
A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.
problem Optimizing robustness measures under input uncertainty.
method Randomized robustness measure GP-UCB (RRGP-UCB) that samples β from a chi-squared-based distribution.
result RRGP-UCB provides tight bounds on expected regret.
Develops a new approach to optimal control of stochastic systems.
problem Optimal control of stochastic nonlinear dynamical systems is challenging.
method Formulates optimal control as input estimation, using probabilistic inference and Expectation Maximization.
result Extracts time-varying linear Gaussian feedback controllers from the joint state-action distribution.
PAC-MOO optimizes constrained multi-objective problems with preferences.
problem Optimizing with constraints and practitioner preferences over objectives.
method Preference-aware constrained multi-objective Bayesian optimization.
result Efficacy demonstrated on real-world analog circuit design problems.
Unified method for balancing simulation and data collection.
problem Finding optimal solutions with uncertain inputs.
method Bayesian Information Collection and Optimisation (BICO) procedure.
result Algorithm automatically determines optimal balance between optimisation and data collection.
WE constructs GP kernels for mixed inputs using weighted EDMs.
problem Limitation of standard GP models in handling categorical variables.
method WEGP constructs kernel function using weighted EDMs for categorical inputs.
result WEGP improves GP model accuracy in both synthetic and real-world optimization problems.
The paper tackles stable maxima optimization for expensive functions.
problem Finding stable maxima of expensive functions with input variations.
method Uses multiple gradient Gaussian Process models to estimate stability and guide optimization.
result Demonstrates effective finding of stable maxima on synthetic and real-world problems.
Bayesian optimization technique scaled using Vecchia approximations.
problem Scalability issue with Gaussian process surrogate models in Bayesian optimization.
method Adapted Vecchia approximation from spatial statistics to Gaussian processes, developed improvements and extensions.
result Methods compared favorably to state-of-the-art on various test functions and reinforcement learning problems.
Graph neural networks improve combinatorial optimization by leveraging inductive bias.
problem Combinatorial optimization problems often arise from related data distributions.
method Using graph neural networks to enhance or solve combinatorial tasks.
result Graph neural networks effectively encode combinatorial and relational input.
New method optimizes protein design by sampling from realistic inputs.
problem Optimizing properties of interest in design problems, especially with black box predictive models.
method Conditioning by Adaptive Sampling, using model-based adaptive sampling to estimate conditional input distributions.
result Achieves state-of-the-art results on protein fluorescence problem.
Scalable algorithm for computing Wasserstein-2 barycenters without bias.
problem Computing Wasserstein-2 barycenters efficiently and accurately.
method Input convex neural networks and cycle-consistency regularization.
result Our approach avoids introducing bias and does not require minimax optimization.
Proposes a method to handle missing inputs in Bayesian optimization.
problem Missing values in historical data and function evaluations.
method Impute missing values using probability distributions and develop a new acquisition function.
result Improves performance of Bayesian optimization by handling missing inputs effectively.
LMGPs extend GPs to handle mixed data, offering better accuracy and interpretability.
problem Handling mixed data types (quantitative and qualitative) in metamodeling.
method Introduce LMGPs that learn a latent manifold for qualitative inputs, using a low-rank linear map.
result LMGPs outperform existing methods in accuracy and versatility.
DIET-SNN optimizes SNNs for faster, lower-energy image classification.
problem High inference latency and inefficient input encoding in SNNs.
method End-to-end backpropagation to optimize membrane leak and firing threshold.
result Achieves top-1 accuracy of 69% on ImageNet with 5 timesteps and 12x less compute energy.
We consider derivative-free black-box global optimization of expensive noisy functions, when most of the randomness in the objective is produced by a few influential scalar random inputs. We present a new Bayesian global optimization algorithm, called Stratified Bayesian Optimization (SBO), which uses this strong depen…
Generative adversarial networks benefit from optimal input dimension and adaptive generator architecture.
problem Minimizing generalization error in GANs through optimal input dimension.
method Introducing generalized GANs (G-GANs) with group penalty and architecture penalty for adaptive dimensionality reduction and network architecture identification.
result G-GANs achieve superior performance with 40%+ improvements in maximum mean discrepancy or Frechet inception distance compared to off-the-shelf methods.
The sparse pseudo-input Gaussian process (SPGP) is a new approximation method for speeding up GP regression in the case of a large number of data points N. The approximation is controlled by the gradient optimization of a small set of M `pseudo-inputs', thereby reducing complexity from N^3 to NM^2. One limitation of th…
Simplified DGPs training by fixing inducing inputs to subset of data.
problem Challenging training of deep Gaussian processes.
method Fixed subset of data for inducing inputs, variational sampling.
result Significant reduction in trainable parameters and computation cost without performance degradation.
Proposes deep collective learning to learn inputs and weights together in neural networks.
problem The need to optimize inputs alongside weights in deep neural networks.
method Introduces deep collective learning to learn inputs and weights jointly, using lookup tables.
result Demonstrates advantages and promise of Lookup-VNets and deep collective learning.
This paper presents a novel decentralized high-dimensional Bayesian optimization (DEC-HBO) algorithm that, in contrast to existing HBO algorithms, can exploit the interdependent effects of various input components on the output of the unknown objective function f for boosting the BO performance and still preserve scala…