Proposes a method to refine PDE-driven high-dimensional rare-event simulation.
problem Challenges in constructing accurate surrogates for rare-event simulation.
method Adaptive importance sampling framework that refines a locally constructed surrogate.
result Achieves accuracy comparable to true-model adaptive importance sampling with fewer high-fidelity evaluations.
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learnin…
A method constructs a stochastic surrogate from dimensionality reduction results for high-dimensional uncertainty quantification.
problem High-dimensional uncertainty quantification with physics-based models.
method Constructs a stochastic surrogate model from dimensionality reduction results.
result Preserves convenience of sequential dimensionality reduction and Gaussian process regression while overcoming limitations.
LGV boosts adversarial attacks by improving surrogate models.
problem Improving the transferability of black-box adversarial attacks.
method LGV uses a pretrained surrogate model and multiple weight sets from additional training epochs to generate an effective surrogate ensemble.
result LGV outperforms other test-time transformations by significant margins.
SBBO optimizes complex spaces using sampling-based models.
problem Optimizing complex spaces with discrete variables.
method Simulation Based Bayesian Optimization (SBBO) using sampling-based surrogate models.
result Empirical effectiveness of SBBO in combinatorial optimization.
We study the interplay between surrogate methods for structured prediction and techniques from multitask learning designed to leverage relationships between surrogate outputs. We propose an efficient algorithm based on trace norm regularization which, differently from previous methods, does not require explicit knowled…
Optimizes hard-to-optimize metrics using adaptive surrogates.
problem Training models with black-box and hard-to-optimize metrics.
method Expresses metric as a function of surrogates, solves optimization problem over relaxed surrogate space.
result Approach performs on par with known methods and adds value when metric form is unknown.
Unified framework for sampling and approximating high-dimensional energy landscapes.
problem Sampling and approximating complex energy landscapes in physical systems with constraints and energy barriers.
method Formulates a minimax optimization problem that jointly adapts surrogate approximation and adaptive sampling.
result Demonstrates effectiveness in biomolecular systems with up to 30 collective variables.
The paper uses model-based trees to create interpretable surrogate models for complex machine learning models.
problem Interpreting complex machine learning models.
method Using model-based trees to partition feature space and create interpretable models.
result Model-based trees generate optimal surrogate models that balance interpretability and performance.
Seq2Seq models speed up epidemic model predictions.
problem Complex epidemic models are computationally expensive.
method Used deep seq2seq models as surrogates for complex models.
result Surrogates predict scenarios up to several thousand times faster.
EPGP surrogate outperforms finite elements in solving wave equations.
problem Benchmarking Gaussian Process surrogates vs. finite elements for wave equation solutions.
method EPGP uses penalized least squares and exponential-polynomial bases; CN-FEM employs Crank--Nicolson time stepping.
result EPGP achieves lower error than CN-FEM under matched degrees-of-freedom.
Unified framework for modeling hierarchical spaces in design problems.
problem Challenges in modeling hierarchical, conditional, heterogeneous, or tree-structured domains.
method Unified framework combining feature modeling and graph theory, introducing meta and partially-decreed variables.
result Demonstrated effectiveness on complex system design problems, including neural networks and green-aircraft.
SURGIN uses generative models to infer subsurface flow data efficiently.
problem Inefficient and task-specific inversion methods for subsurface multiphase flow.
method SURGIN integrates U-FNO surrogate with SGM for zero-shot conditional generation.
result Decent inference of heterogeneous geological fields and flow dynamics with uncertainty quantification.
Optimize black-box simulators with local generative models.
problem Optimizing non-differentiable, stochastic simulators with intractable likelihoods.
method Differentiable local surrogate models based on deep generative models.
result Local surrogates enable gradient-based optimization, faster than baseline methods.
A new beta-VAE based regression model accelerates oilfield optimization studies.
problem Computational expense of full-physics reservoir simulations.
method beta-VAE for interpretable latent space representation, probabilistic dense layers for uncertainty quantification.
result Interpretable latent representation and quantified uncertainty for optimization decisions.
The paper explores trading off consistency and dimensionality in convex surrogates for multiclass classification.
problem Designing consistent surrogate losses for multiclass classification with high-dimensional outcomes.
method Investigates embedding outcomes into convex polytopes and examining consistency under low-noise assumptions.
result Consistency can be achieved with less than n−1 dimensions, but hallucination occurs for some distributions. New bounds show polyhedral surrogates are optimal for generalization.
problem Proving generalization rates for polyhedral loss functions.
method Developed two general results for polyhedral surrogates.
result Polyhedral surrogates provide linear surrogate regret bounds, translating directly to target rates.
NoFAS combines variational inference and adaptive surrogate models for efficient inference of computationally expensive models.
problem Efficient inference of parameters from data with computationally expensive models.
method Variational inference with normalizing flow and adaptive surrogate model training.
result NoFAS reduces computational cost without sacrificing inferential accuracy.
Improved MALA method for neural networks uncertainty quantification.
problem Uncertainty quantification in Bayesian neural networks.
method Corrected Stochastic MALA (csMALA) with a simplified correction term.
result Improved surrogate posterior for quantifying uncertainties in neural networks.
A new method reduces high-dimensional parameter spaces for faster numerical tasks.
problem Efficiently reducing high-dimensional parameter spaces for numerical tasks.
method Local Active Subspaces (LAS) combining active subspaces with clustering techniques.
result Significant speed-up in numerical tasks through efficient dimension reduction.
Develops a transparent surrogate model for complex data.
problem Balancing accuracy and transparency in complex decision-making models.
method Partial dependence effects for feature engineering, smart segmentation, and GLM fitting.
result The maidrr GLM closely approximates a black box model and outperforms benchmarks.
Polynomial chaos surrogates handle intrinsic noise in stochastic models.
problem Handling intrinsic noise in stochastic models with parametric uncertainty.
method Developed a PCE surrogate on a joint space of intrinsic and parametric uncertainty using Rosenblatt transformations and Karhunen-Loeve expansion.
result Quantified intrinsic noise contribution to model output variance using PCE Sobol indices.
We propose a robust adversarial prediction framework for general multiclass classification. Our method seeks predictive distributions that robustly optimize non-convex and non-continuous multiclass loss metrics against the worst-case conditional label distributions (the adversarial distributions) that (approximately) m…
New method optimises learning via surrogate PAC-Bayes bounds.
problem Computational intractability of optimising generalisation bounds.
method Iteratively optimising surrogate training objectives derived from PAC-Bayes bounds.
result Iteratively optimising surrogates implies optimising original generalisation bounds.
The central task in modeling complex dynamical systems is parameter estimation. This task involves numerous evaluations of a computationally expensive objective function. Surrogate-based optimization introduces a computationally efficient predictive model that approximates the value of the objective function. The stand…
INP accelerates stochastic simulations using deep Bayesian active learning.
problem Computational expense of stochastic simulations at fine-grained resolution.
method Interactive Neural Process (INP) framework combining spatiotemporal surrogate model and active learning acquisition function.
result STNP outperforms baselines in accelerating stochastic simulations and LIG achieves state-of-the-art for Bayesian active learning.
This work improves surrogate models using low-fidelity data to enhance accuracy and efficiency.
problem Limited training data makes high-fidelity models unreliable.
method Uses low-fidelity data to augment input space and condition high-fidelity models.
result Increased predictive accuracy and reduced computational cost compared to existing methods.
Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the…
Bayesian neural networks with functional priors improve surrogate modeling in mechanics.
problem Challenges in integrating prior knowledge and quantifying uncertainties in high-dimensional NN parameter spaces.
method Anchored ensembling to integrate a priori information and learn low-rank correlations between NN parameters.
result Effective transfer of knowledge between function-space and parameter-space priors improves surrogate model accuracy and uncertainty estimation.
CCVFM uses coreset to improve generative models by refining residual flows.
problem Generating multimodal distributions from scratch is challenging.
method Augments hierarchical rectified flow with a data-informed source distribution using a coreset.
result CCVFM achieves competitive few-step generation without a learned noise-to-data map.
We learn a compact surrogate model for optimization problems to reduce training and inference time.
problem Solving optimization problems with unknown parameters is computationally expensive and may lead to suboptimal solutions.
method We represent the optimization problem in terms of meta-variables and learn a low-dimensional surrogate model end-to-end with the predictive model.
result We achieve a large reduction in training and inference time, and improved performance.
Develops a surrogate model for predicting system responses using GDMaps and geometric harmonics.
problem Predicting responses of engineering systems and complex physical phenomena with uncertainties.
method Grassmannian diffusion maps (GDMaps) and geometric harmonics for low-dimensional representation and function extension.
result Accurate predictions of system responses in various examples, demonstrating the technique's potential for uncertainty quantification.
We study consistency properties of surrogate loss functions for general multiclass learning problems, defined by a general multiclass loss matrix. We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning pro…
Improving in-context learning for latent space Bayesian optimization by adapting pretraining on molecular latent space.
problem Improving in-context learning for latent space Bayesian optimization.
method Adapting pretraining on molecular latent space.
result Achieving strong performance on held-out molecular optimization benchmarks.
The aim of the present paper is to develop a strategy for solving reliability-based design optimization (RBDO) problems that remains applicable when the performance models are expensive to evaluate. Starting with the premise that simulation-based approaches are not affordable for such problems, and that the most-probab…
Active sampling improves design space exploration for analog circuits.
problem Efficiently exploring the space of design features in analog circuits with many parameters.
method Combining drastic dimension reduction with sensitivity analysis and Bayesian surrogate modeling for active sampling.
result The proposed active sampling flow outperforms traditional Monte-Carlo sampling.
New method optimizes black-box functions using generative models and Wasserstein distance.
problem Optimizing black-box functions with stochastic responses in high dimensions.
method Deep generative surrogate models and Wasserstein distance for uncertainty estimation.
result Method outperforms state-of-the-art methods in robustness to function shape and stochasticity.
O3 optimizes samples from generative models without extra training.
problem Optimizing specific criteria within rich data distributions.
method Surrogate latent spaces for efficient black-box optimisation.
result Surrogate-space optimisation finds higher-scoring samples.
A new framework learns differentiable structured losses from data.
problem Learning effective losses for complex structured prediction tasks.
method Contrastive learning to learn differentiable structured losses from output data.
result Achieves similar or better performance than kernel-based methods.
REMAL: Residual Equilibrium Manifold Active Learning for Surrogate-Based Multidisciplinary Design Analysis
problem Multidisciplinary design analysis of coupled engineering systems requires solving equilibrium states where all disciplinary coupling variables are consistent.
method Residual manifold surrogate modeling framework for coupled systems.
result REMAL learns a surrogate model of the joint residual manifold via multitask Gaussian process models.
BoGA combines evolutionary search with Bayesian optimization for efficient protein design.
problem Designing novel proteins with specific characteristics is challenging due to sequence space complexity.
method BoGA integrates a genetic algorithm with Bayesian optimization to efficiently explore sequence space.
result BoGA accelerates discovery of high-confidence binders for diverse protein design objectives.
SAM minimizes loss sharpness, improving adversarial transferability.
problem Improving adversarial transferability of deep neural networks.
method Evaluating surrogate models trained with seven minimizers, focusing on loss sharpness and flat neighborhoods.
result SAM minimizes loss sharpness, leading to better adversarial transferability.
Deep neural networks improve surrogate models for non-smooth quantities in uncertain geometries.
problem Building accurate surrogates for non-smooth quantities in uncertain geometries.
method Deep neural networks for point evaluation of solutions to interface problems with geometric uncertainties.
result Neural networks provide good surrogates without suffering from the curse of dimensionality.
We present a framework for automatically structuring and training fast, approximate, deep neural surrogates of stochastic simulators. Unlike traditional approaches to surrogate modeling, our surrogates retain the interpretable structure and control flow of the reference simulator. Our surrogates target stochastic simul…
A new framework uses an Incremental Transformer to design geopolymer mixtures efficiently.
problem Designing geopolymer mixtures with limited data and physical constraints.
method Topology-aware surrogate framework guided by Incremental Transformer.
result The design space is redundant, with fewer effective mixture regimes.
Randomizing the Fourier-transform (FT) phases of temporal-spatial data generates surrogates that approximate examples from the data-generating distribution. We propose such FT surrogates as a novel tool to augment and analyze training of neural networks and explore the approach in the example of sleep-stage classificat…
Deep learning model approximates stochastic responses.
problem Approximating stochastic responses using neural networks.
method Generative neural network with conditional maximum mean discrepancy (CMMD) loss.
result Excellent performance on benchmark problems.
We formalize and study the natural approach of designing convex surrogate loss functions via embeddings, for problems such as classification, ranking, or structured prediction. In this approach, one embeds each of the finitely many predictions (e.g.\ rankings) as a point in Rd, assigns the original loss val…