Maximal Rate of Stepwise Uncertainty Reduction selects simulations to reduce uncertainty efficiently.
problem Efficiently estimating quantities of interest from multi-fidelity simulations.
method Bayesian sequential strategy that maximizes the ratio of expected uncertainty reduction to simulation cost.
result MR-SUR strategy unifies and provides principled approaches to develop new methods.
A new Bayesian optimization method tackles constrained optimization with uncertainties.
problem Optimizing functions with uncertain constraints.
method Bayesian optimization with a new acquisition criterion.
result The new criterion optimizes both objective function improvement and constraint reliability.
The current study proposes a dimension reduction method, stepwise support vector machine (SVM), to reduce the dimensions of large p small n datasets. The proposed method is compared with other dimension reduction methods, namely, the Pearson product difference correlation coefficient (PCCs), recursive feature eliminati…
Bayesian method for estimating inputs leading to specific probability outputs.
problem Estimating inputs for specific probability outputs of uncertain functions.
method Bayesian strategy using Gaussian process modeling and SUR principle.
result Surpassed performance of existing methods through numerical experiments.
Adaptive batching improves Gaussian process surrogates for noisy level set estimation.
problem Learning the level set of noisy simulator responses.
method Developed four novel adaptive batching schemes for Gaussian process metamodels.
result Adaptive batching brings significant computational speed-ups with minimal loss of modeling fidelity.
We study the conditions for a nilpotent Lie group to be foliated into subgroups that have square integrable (relative discrete series) unitary representations, that fit together to form a filtration by normal subgroups. Then we use that filtration to construct a class of "stepwise square integrable" representations on …
We propose and analyze sequential design methods for the problem of ranking several response surfaces. Namely, given L≥2 response surfaces over a continuous input space X, the aim is to efficiently find the index of the minimal response across the entire X. The response surfaces are not known and ha…
Game theory finds nowadays a broad range of applications in engineering and machine learning. However, in a derivative-free, expensive black-box context, very few algorithmic solutions are available to find game equilibria. Here, we propose a novel Gaussian-process based approach for solving games in this context. We f…
SplitWise enhances stepwise regression by adaptively encoding numeric predictors into binary features.
problem Capturing nonlinear relationships in regression models without sacrificing interpretability.
method Adaptive encoding of numeric predictors into binary features using shallow decision trees, assessed by AIC or BIC.
result Consistently produces more parsimonious and generalizable models than traditional techniques.
We consider the problem of estimating the set of all inputs that leads a system to some particular behavior. The system is modeled by an expensive-to-evaluate function, such as a computer experiment, and we are interested in its excursion set, i.e. the set of points where the function takes values above or below some p…
We consider the problem of learning the level set for which a noisy black-box function exceeds a given threshold. To efficiently reconstruct the level set, we investigate Gaussian process (GP) metamodels. Our focus is on strongly stochastic samplers, in particular with heavy-tailed simulation noise and low signal-to-no…
A novel stepwise VI method using vine copulas for complex latent dependence.
problem Modeling complex latent dependence structures in probabilistic models.
method Stepwise estimation of vine copula parameters using Rényi divergence and a stopping criterion.
result Our method outperforms mean-field VI and is more parsimonious in complex applications.
The paper extends consistency results for sequential design strategies to vector-valued Gaussian processes.
problem Estimating excursion sets of vector-valued Gaussian processes.
method Clarifying the connection between continuous Gaussian processes and Gaussian measures in Banach spaces, extending concepts and properties from scalar-valued settings to vector-valued settings.
result Consistency results for sequential design strategies can be applied to vector-valued Gaussian processes.
Improved equation learning accuracy via comprehensive R²-elimination and Bayesian model selection.
problem Challenges in exhaustive equation learning due to multicollinearity and greedy steps.
method Combines R2 and Bayesian model evidence for a comprehensive yet efficient search. result Our approach surpasses all other methods in identification accuracy, especially in exact equation recovery.
Proposes a new algorithm for Sparse Bayesian Learning connected to Stepwise Regression.
problem Sparse Bayesian Learning for probabilistic models.
method Coordinate ascent algorithm (RMP) for SBL, showing connection to Stepwise Regression.
result RMP's noise variance parameter limit connects to Stepwise Regression, with derived guarantees.
We consider the problem of maximizing a real-valued continuous function f using a Bayesian approach. Since the early work of Jonas Mockus and Antanas Žilinskas in the 70's, the problem of optimization is usually formulated by considering the loss function maxf−Mn (where Mn denotes the best function value ob…
In this article, we advocate the ensemble approach for variable selection. We point out that the stochastic mechanism used to generate the variable-selection ensemble (VSE) must be picked with care. We construct a VSE using a stochastic stepwise algorithm, and compare its performance with numerous state-of-the-art algo…
I consider unsupervised extensions of the fast stepwise linear regression algorithm \cite{efroymson1960multiple}. These extensions allow one to efficiently identify highly-representative feature variable subsets within a given set of jointly distributed variables. This in turn allows for the efficient dimensional reduc…
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Develops a more powerful selective inference method for stepwise feature selection.
problem Loss of power in existing conditional SI methods due to over-conditioning.
method Uses homotopy continuation approach to overcome over-conditioning.
result Shows improved power and efficiency in selective inference for feature selection.
Proposes a novel BO algorithm for stepwise model selection in sequence prediction.
problem Optimal model selection in sequential settings where model performance varies over time.
method Bayesian optimization with deep kernel learning to handle multiple black-box functions.
result Outperforms standard and multi-objective BO algorithms on sequence prediction tasks.
Grid-scale batteries' bid patterns in price uncertainty markets
problem Interpreting bids from grid-scale batteries in wholesale electricity markets under price uncertainty
method Developing an asset-level model of a price-taking battery
result Empirical results deliver insights into withholding behavior, uncertainty effects, and risk management reshaping bid curves
In this era of big data, feature selection techniques, which have long been proven to simplify the model, makes the model more comprehensible, speed up the process of learning, have become more and more important. Among many developed methods, forward and stepwise feature selection regression remained widely used due t…
A method constructs a stochastic surrogate from dimensionality reduction results for high-dimensional uncertainty quantification.
problem High-dimensional uncertainty quantification with physics-based models.
method Constructs a stochastic surrogate model from dimensionality reduction results.
result Preserves convenience of sequential dimensionality reduction and Gaussian process regression while overcoming limitations.
Unified framework explains few-shot multimodal medical imaging performance.
problem Limited labeled data in rare diseases and low-resource settings.
method PAC learning, VC theory, PAC Bayesian analysis, information gain, Chain of Thought reasoning.
result Unified theoretical framework for few-shot multimodal medical imaging.
Sequence generative adversarial networks (SeqGAN) have been used to improve conditional sequence generation tasks, for example, chit-chat dialogue generation. To stabilize the training of SeqGAN, Monte Carlo tree search (MCTS) or reward at every generation step (REGS) is used to evaluate the goodness of a generated sub…
A new random forest algorithm uncovers feature interdependencies better than traditional methods.
problem Tackles the sub-optimality of greedy decision tree implementations in random forests.
method Presented a 'stepwise lookahead' variation of random forests that considers multiple split nodes simultaneously.
result Significantly outperforms greedy random forests in uncovering feature interdependencies, especially in high-noise environments.
A new method for online prediction uncertainty quantification in non-exchangeable panel data.
problem Challenges in quantifying predictive uncertainty for non-exchangeable panel data.
method Online conformal prediction framework for non-exchangeable panel data, using similarity weights and adaptive miscoverage levels.
result Improves coverage on worst-covered target units through adaptive interval-width allocation.
Bayesian method corrects for model selection multiplicity in regression.
problem Model selection multiplicity in regression analysis.
method Developed a Bayesian prior distribution based on Holm procedure analogy.
result Adequate multiplicity correction requires sparsity not provided by recommended priors.
Spiking neural networks perform similarly to deep networks on occluded images.
problem Robust object recognition in partially occluded images.
method Developed a two-layer spiking neural network trained on natural scenes with a biologically plausible learning rule, compared to deep convolutional networks.
result Spiking neural networks achieve good accuracy and robustness on stepwise pixel erasement tasks.
A new method for reducing model complexity using neural active manifolds.
problem Uncertainty quantification in computationally expensive models.
method Autoencoders and surrogate models to discover a neural active manifold.
result Neural active manifolds reduce model variance in multifidelity sampling.
RMFGP combines multi-fidelity models for efficient uncertainty quantification.
problem Efficiently infer quantities of interest with limited high-fidelity data.
method Rotated multi-fidelity Gaussian process with dimension reduction and Bayesian active learning.
result RMFGP model improves accuracy and efficiency in high-dimensional problems.
A new method reduces complexity and uncertainty in neural networks.
problem Uncertainty quantification in complex neural networks.
method Condensed Stein Variational Gradient Descent (cSVGD) method.
result Condensed SVGD provides uncertainty quantification on parameters.
Select-DC reduces GFLOPS for uncertainty estimation in neural networks.
problem Computational inefficiency in estimating model uncertainty for low-latency applications.
method Select-DC uses a subset of layers to model epistemic uncertainty with MCDC, reducing GFLOPS.
result Significant reduction in GFLOPS required for uncertainty estimation with marginal performance loss.
Bayesian Neural Networks improve geophysical model ensembles with reduced uncertainty.
problem Improving geophysical model projections and uncertainty quantification.
method Developed a Bayesian Neural Network ensemble strategy for geophysical models.
result Bayesian Neural Network ensemble outperforms existing methods in ozone prediction.
Bayesian model fuses multiple classifiers with explicit correlation modeling.
problem Combining outputs of multiple classifiers with explicit correlation.
method Hierarchical Bayesian model with correlated Dirichlet distribution.
result Fused classifier performance can be Bayes optimal even for highly correlated base classifiers.
We present a technique to perform dimensionality reduction on data that is subject to uncertainty. Our method is a generalization of traditional principal component analysis (PCA) to multivariate probability distributions. In comparison to non-linear methods, linear dimensionality reduction techniques have the advantag…
PCENet reduces uncertainty in high-dimensional data efficiently.
problem Uncertainty quantification in high-dimensional data is computationally expensive.
method Two-stage learning process: variational autoencoder for low-dimensional representation, polynomial chaos expansion for mapping.
result Model captures system dynamics, learns under uncertainty, estimates high-dimensional data uncertainty, matches output distribution moments.
Nonparametric adaptive robust control tackles model uncertainty in stochastic processes.
problem Model uncertainty in stochastic processes.
method Adaptive robust control methodology using online learning and uncertainty reduction, empirical distribution, and Lagrangian duality.
result Nonparametric adaptive robust control approach is preferable to traditional robust frameworks.
How can we design safe reinforcement learning agents that avoid unnecessary disruptions to their environment? We show that current approaches to penalizing side effects can introduce bad incentives, e.g. to prevent any irreversible changes in the environment, including the actions of other agents. To isolate the source…
Bayesian neural networks improve uncertainty quantification in non-linear dimensionality reduction.
problem Current neural network models lack adequate uncertainty quantification.
method Deploy Markov chain Monte Carlo sampling algorithms for Bayesian inference in ANN models with latent variables.
result New research directions are needed due to fundamental challenges in neural networks with latent variables.
New framework improves classification accuracy using Pillai's trace and ULDA.
problem Traditional LDA's limitations in noise sensitivity and non-invertible matrices.
method Integrates Pillai's trace with ULDA for a unified classifier.
result Effective control of Type I error rates and improved classification accuracy.
We present an adaptive approach for valuing the European call option on assets with stochastic volatility. The essential feature of the method is a reduction of uncertainty in latent volatility due to a Bayesian learning procedure. Starting from a discrete-time stochastic volatility model, we derive a recurrence equati…
Paper reduces uncertainty in predictive models using neural networks and Gaussian processes.
problem High epistemic uncertainty in predictive models.
method Adaptive sampling approach with prediction interval-generation neural networks and Gaussian processes.
result Method consistently converges faster to minimum epistemic uncertainty levels.
New method uses sparse random features for crashworthiness analysis.
problem Efficient surrogate modelling for uncertainty quantification.
method Sparse Random Features combined with self-supervised dimensionality reduction.
result Superiority over state-of-the-art techniques in crashworthiness analysis.
Assistive robots can potentially improve the quality of life and personal independence of elderly people by supporting everyday life activities. To guarantee a safe and intuitive interaction between human and robot, human intentions need to be recognized automatically. As humans communicate their intentions multimodall…
Semi-supervised clustering seeks to augment traditional clustering methods by incorporating side information provided via human expertise in order to increase the semantic meaningfulness of the resulting clusters. However, most current methods are \emph{passive} in the sense that the side information is provided before…
Beam search improves UQ in LLMs by reducing duplicates and variance.
problem Peaked distributions in multinomial sampling lead to duplicates and high variance in uncertainty estimates.
method Employ beam search to generate candidates for consistency-based UQ, providing a theoretical lower bound and empirical evaluation.
result Beam search achieves smaller error than multinomial sampling, leading to state-of-the-art UQ performance.