Paper introduces CageBO for optimizing complex public policy problems.
problem Complex decision-making and implicit constraints in public policy.
method CageBO framework using conditional variational autoencoder.
result CageBO outperforms baselines in optimizing large-scale police redistricting.
ADMM solves constrained CASH problems by breaking them into smaller, manageable pieces.
problem Handling black-box constraints in CASH problems.
method Leverages ADMM optimization framework to decompose CASH problems.
result ADMM facilitates incorporation of black-box constraints.
A new method for optimizing black-box problems with constraints.
problem Optimizing black-box systems with multiple performance criteria and constraints.
method Developed a novel constrained Bayesian optimization approach based on the knowledge gradient method.
result A new acquisition function that balances optimality and feasibility.
Method reformulates constrained optimization as latent space inference.
problem Optimizing black-box functions with hard constraints.
method Posterior inference in latent space using flow-based models and diffusion models.
result Method achieves superior performance across various tasks.
Parallel BO method for multi-objective optimization with constraints.
problem Optimizing multiple objectives under constraints with expensive evaluations.
method PPESMOC, a batch method for simultaneous optimization of black-box functions.
result Empirical evidence shows PPESMOC is effective for multi-objective optimization with constraints.
Bayesian optimisation tackles expensive black-box functions with constraints.
problem Optimizing constrained black-box functions in machine learning and simulation.
method Proposes a new Knowledge Gradient acquisition function for constrained Bayesian optimisation.
result Demonstrates superior performance over four state-of-the-art constrained Bayesian optimisation algorithms.
Paper tackles constrained bandit problems with a new learning framework.
problem Optimizing a black-box reward function subject to a black-box constraint function over a continuous space.
method Rectified Pessimistic-Optimistic Learning (RPOL) framework, incorporating optimistic and pessimistic GP bandit learning.
result RPOL achieves sublinear regret and minimal cumulative constraint violation.
New method for robustly interpreting ML models using quantile constraints and Wasserstein projections.
problem Assessing robustness of black-box models to input misspecification.
method Quantile-constrained Wasserstein projections for robust interpretability.
result Analytical solution for perturbation problem and smooth perturbations.
Tree ensemble kernels improve Bayesian optimization for mixed features and constraints.
problem Optimizing over mixed-feature spaces with known constraints.
method Kernel interpretation of tree ensembles as Gaussian Process prior, compatible optimization formulation for acquisition function, integration of known constraints.
result Framework outperforms state-of-the-art methods for mixed-feature spaces and constraints.
GACEM optimizes complex multi-modal problems using neural networks.
problem Black-box optimization and constraint satisfaction in multi-modal environments.
method Modified Cross-Entropy Method with masked auto-regressive neural network.
result GACEM outperforms traditional CEM in diverse solutions, mode discovery, and sample efficiency.
Novel framework for Bayesian neural networks incorporating task-specific constraints.
problem Task-specific constraints in supervised model deployment.
method Introduces Output-Constrained BNN (OC-BNN) framework.
result OC-BNNs effectively incorporate prior expert knowledge and desiderata like safety and fairness.
We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervised learning pipelines. This black-box (gradient-free) optimization with mixed integer & continuous variables is a challenging problem. We pr…
This work presents PESMOC, Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints, an information-based strategy for the simultaneous optimization of multiple expensive-to-evaluate black-box functions under the presence of several constraints. PESMOC can hence be used to solve a wide range…
Adapts Bayesian optimization for mixed constraints in aircraft design.
problem Optimizing expensive black box functions with mixed constraints.
method Super efficient global optimization with upper trust bound for constraints, Gaussian process uncertainty, refinement procedure.
result Superior performance on aircraft design problem compared to state-of-the-art solvers.
Bayesian method optimizes uncertain constraints in black-box function optimization.
problem Optimizing black-box functions with uncertain environmental variables.
method Distributionally robust chance-constrained Bayesian optimization.
result The method can find accurate solutions with high probability in a finite number of trials.
SCBO tackles constrained optimization in high dimensions.
problem Optimizing complex, high-dimensional functions with constraints.
method Proposes scalable constrained Bayesian optimization algorithm.
result Demonstrates excellent performance on various benchmarks.
Recently, neural networks trained as optimizers under the "learning to learn" or meta-learning framework have been shown to be effective for a broad range of optimization tasks including derivative-free black-box function optimization. Recurrent neural networks (RNNs) trained to optimize a diverse set of synthetic non-…
PyBADS optimizes complex functions quickly and reliably.
problem Optimizing rough, noisy, and expensive functions with unknown gradients.
method Bayesian Adaptive Direct Search (BADS) algorithm.
result PyBADS performs well on both artificial and real-world problems.
HyperFair integrates fairness in recommender systems using probabilistic soft logic.
problem Ensuring fairness in recommender systems across diverse domains.
method Soft fairness constraints integrated as regularization in a joint inference objective function.
result HyperFair improves fairness of predictions from black-box models and hybrid systems.
Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general ca…
Improved MESMOC+ optimizes constrained multi-objective problems efficiently.
problem Optimizing constrained multi-objective problems with expensive evaluations.
method Minimizes entropy of Pareto frontier to guide search, using linear cost and decoupled evaluation.
result Significantly faster than alternatives, with more accurate entropy estimation.
New method improves black-box attacks using pre-trained models.
problem Efficiently attacking black-box models with limited information.
method EigenBA algorithm leveraging pre-trained white-box model's Jacobian matrix.
result Optimal perturbations are related to right singular vectors of Jacobian matrix.
We propose an intriguingly simple method for the construction of adversarial images in the black-box setting. In constrast to the white-box scenario, constructing black-box adversarial images has the additional constraint on query budget, and efficient attacks remain an open problem to date. With only the mild assumpti…
New framework limits testing algorithmic stability under computational constraints.
problem Testing algorithmic stability is computationally hard.
method Unified framework for quantifying stability hardness.
result Exhaustive search is the only universally valid mechanism for certifying stability.
We present an information-theoretic framework for solving global black-box optimization problems that also have black-box constraints. Of particular interest to us is to efficiently solve problems with decoupled constraints, in which subsets of the objective and constraint functions may be evaluated independently. For …
In this paper, we consider the problem of black box continuous submodular maximization where we only have access to the function values and no information about the derivatives is provided. For a monotone and continuous DR-submodular function, and subject to a bounded convex body constraint, we propose Black-box Contin…
GONs improve predictions of maximizers from noisy black-box functions.
problem Estimating maximizers of noisy black-box functions.
method Global Optimization Networks (GONs) composed of invertible and unimodal functions.
result GONs outperform convex fits, GPR, and DNNs in prediction accuracy.
Unknown constraints arise in many types of expensive black-box optimization problems. Several methods have been proposed recently for performing Bayesian optimization with constraints, based on the expected improvement (EI) heuristic. However, EI can lead to pathologies when used with constraints. For example, in the c…
Bayesian neural network (BNN) priors are defined in parameter space, making it hard to encode prior knowledge expressed in function space. We formulate a prior that incorporates functional constraints about what the output can or cannot be in regions of the input space. Output-Constrained BNNs (OC-BNN) represent an int…
Constraint-based learning reduces the burden of collecting labels by having users specify general properties of structured outputs, such as constraints imposed by physical laws. We propose a novel framework for simultaneously learning these constraints and using them for supervision, bypassing the difficulty of using d…
Adaptive algorithm for multi-objective optimization with binary constraints.
problem Optimization of black-box problems with binary constraints.
method Bayesian optimization using regression and classification models.
result Significantly faster expected hypervolume calculation.
Orthogonal deep models defend against black-box attacks by ensuring internal representations are nearly orthogonal.
problem Vulnerability of deep learning models to black-box adversarial attacks.
method Introduce a gradient regularization scheme to encourage deep models' internal representations to be orthogonal to another model's.
result Orthogonal deep models significantly boost robustness against transferable black-box adversarial attacks.
New method optimizes processes under constraints using bivariate Gaussian models.
problem Optimizing processes with constraints using traditional methods.
method Developed a constrained expected improvement acquisition function using bivariate Gaussian process models.
result Demonstrated improved performance in a manufacturing cure process optimization.
New framework for resilient bi-criteria optimization under noisy feedback.
problem Bi-criteria combinatorial optimization with noisy function evaluations.
method Introducing (α,β,δ,extttN)-resilience and developing a black-box framework. result Achieves sublinear regret and constraint violation for bi-criteria bandit problems.
New framework for variational coresets simplifies Bayesian inference for complex models.
problem Efficient Bayesian inference for complex models like neural networks.
method Black-box variational inference for coresets that handle intractable posterior distributions.
result Principled application of variational coresets to Bayesian neural networks.
Canonical correlation analysis was proposed by Hotelling [6] and it measures linear relationship between two multidimensional variables. In high dimensional setting, the classical canonical correlation analysis breaks down. We propose a sparse canonical correlation analysis by adding l1 constraints on the canonical vec…
Proposes a tabular transformer model to maintain feature effect intelligibility.
problem Losing marginal feature effects in deep tabular transformer networks.
method Adapts tabular transformer networks to identify marginal feature effects.
result The model accurately identifies marginal feature effects, matching black-box performance while maintaining intelligibility.
As machine learning algorithms are increasingly applied to high impact yet high risk tasks, such as medical diagnosis or autonomous driving, it is critical that researchers can explain how such algorithms arrived at their predictions. In recent years, a number of image saliency methods have been developed to summarize …
Image post-processing is used in clinical-grade ultrasound scanners to improve image quality (e.g., reduce speckle noise and enhance contrast). These post-processing techniques vary across manufacturers and are generally kept proprietary, which presents a challenge for researchers looking to match current clinical-grad…
This work investigates uncertainty quantification for black-box large language models in natural language generation.
problem Lack of trustworthiness in responses generated by black-box large language models.
method Differentiated uncertainty vs confidence, proposed and compared several confidence/uncertainty measures, applied to selective NLG.
result A simple measure for semantic dispersion can predict the quality of LLM responses.
Novel method for efficient optimization of noisy, expensive hybrid models.
problem Efficient optimization of hybrid models with noisy observations and constraints.
method Constrained Upper Quantile Bound (CUQB) method exploiting composite structure.
result Significantly improved sampling efficiency and theoretical guarantees.
In many machine learning applications, it is important to explain the predictions of a black-box classifier. For example, why does a deep neural network assign an image to a particular class? We cast interpretability of black-box classifiers as a combinatorial maximization problem and propose an efficient streaming alg…
The notion of expense in Bayesian optimisation generally refers to the uniformly expensive cost of function evaluations over the whole search space. However, in some scenarios, the cost of evaluation for black-box objective functions is non-uniform since different inputs from search space may incur different costs for …
Operator learning approximates complex mappings for PDEs and experimental data.
problem Approximating mappings between infinite-dimensional function spaces for scientific computing.
method Formalizing operator learning as function-to-function regression and incorporating physical constraints.
result Development of rigorous uncertainty quantification frameworks for operator learning.
The paper introduces a method to learn models with built-in explanations.
problem Lack of interpretability in deep learning models.
method Formalizes learning with explanation constraints and provides a learning theoretic framework.
result Models that satisfy these constraints have reduced Rademacher complexities, improving their performance.
A new error bound improves safety in Bayesian optimization.
problem Ensuring safety in Bayesian optimization with probabilistic models.
method Introducing a novel error bound using Wiener kernel regression for Gaussian processes and noise.
result The new error bound provides larger safety regions than previous methods.
MVRSM optimizes expensive functions with mixed variables, outperforming state-of-the-art methods.
problem Minimizing expensive functions with mixed continuous and integer variables.
method Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) using rectified linear units.
result MVRSM outperforms state-of-the-art methods on synthetic and real-life benchmarks.
Method for explaining machine learning survival models using counterfactuals.
problem Tackles the challenge of explaining survival models in machine learning.
method Introduces a condition based on the difference of mean times to event for counterfactual explanation. Reduces the problem to a convex optimization problem for Cox models and applies Particle Swarm Optimization for other models.
result Demonstrates the effectiveness of the proposed method through numerical experiments.