Global sensitivity analysis improves BNN hyperparameter selection for accurate uncertainty quantification.
problem Difficulties in obtaining accurate uncertainty quantification with Bayesian Neural Networks (BNNs).
method Global sensitivity analysis of BNN performance under varying hyperparameter settings.
result Many hyperparameters interact to affect both predictive accuracy and uncertainty quantification.
Framework for uncertainty estimation in training parameters.
problem Estimating uncertainty in training parameters.
method Marginalizing hyperparameters as random variables, investigating various forms of marginalisation.
result Some marginalisations can reliably estimate uncertainty without extensive tuning.
A cubing strategy identifies stable hyperparameter regions for uncertainty quantification in spatial deep learning.
problem Uncertainty quantification in spatial deep learning models.
method Cubing-based diagnostic framework to recursively partition hyperparameter space and evaluate regions using scoring rules.
result Our approach produces competitive or superior predictive intervals compared to a statistical baseline model.
New algorithm improves Gaussian process hyperparameter tuning for large datasets.
problem Scalable hyperparameter tuning for Gaussian processes on large datasets.
method Estimates smoothness and length-scale parameters in Matern kernel using novel loss functions.
result Improved uncertainty quantification over traditional methods.
Bayesian optimization reduces hyperparameter tuning cost for stochastic models.
problem Hyperparameter tuning under uncertainty in noisy function evaluations.
method Bayesian optimization framework for scale parameter in stochastic models, using statistical surrogate and closed-form optimizer.
result Significant reduction in computational cost (40 times fewer data points, 40-fold reduction in cost).
Sparse Gaussian process hyperparameters optimized using MCMC.
problem Hyperparameter uncertainty leads to biased estimates and underestimation of predictive uncertainty.
method Proposes an MCMC algorithm to sample from the hyperparameter posterior in sparse Gaussian process regression.
result Significantly improves sampling efficiency in the Gaussian likelihood case.
New hyperparameter ensembles boost neural network performance and uncertainty.
problem Improving neural network robustness and uncertainty quantification.
method Designing ensembles over both weights and hyperparameters, stratified across random initializations.
result Hyper-deep and hyper-batch ensembles outperform deep and batch ensembles on various architectures.
We introduce a Bayesian framework for inference with a supervised version of the Gaussian process latent variable model. The framework overcomes the high correlations between latent variables and hyperparameters by using an unbiased pseudo estimate for the marginal likelihood that approximately integrates over the late…
Bayesian optimisation improves with fully-Bayesian treatment of hyperparameters.
problem Overconfident model predictions in BO due to ignoring hyperparameter uncertainty.
method Investigate FBBO using three approximate inference schemes compared to maximum likelihood approach.
result FBBO using EI with an ARD kernel leads to best performance in noise-free setting.
New method improves uncertainty quantification in latent variable models.
problem Uncertainty quantification in latent variable models with SGLD-Gibbs.
method Statistical scaling limit theory for SGLD-Gibbs, proposing hyperparameter tuning.
result Explicit guidance on hyperparameter tuning for SGLD-Gibbs ensures meaningful uncertainty quantification.
This paper explores BDL hyperparameters for robust polynomial mapping with noise.
problem Designing BDL hyperparameters for robust function mapping with uncertainty quantification.
method Mapping Bayesian connectionist representations to polynomials of varying orders and noise types.
result Optimal network depth and ensemble size for prediction and uncertainty quantification.
This paper uses Nested Sampling to improve Gaussian Process uncertainty quantification.
problem Underestimating predictive uncertainty and overfitting in Gaussian Process models.
method Marginalises hyperparameters using Nested Sampling for spectral mixture kernels.
result Improves predictive performance and uncertainty quantification across various data sets.
BITS for GAPS uses Bayesian methods to improve surrogate model accuracy in complex systems.
problem Improving surrogate model accuracy in complex physical systems with uncertainty.
method Bayesian Information-Theoretic Sampling for hierarchical Gaussian Process Surrogates.
result Increased expected information gain and predictive accuracy by targeting high-uncertainty regions.
Unified Bayesian-AI framework improves epidemiological risk prediction and uncertainty quantification.
problem Lack of calibrated uncertainty in machine learning models for epidemiology.
method Combines Bayesian prediction with Bayesian hyperparameter optimization using logistic regression and Gaussian-process Bayesian optimization.
result Unified Bayesian-AI framework provides reliable coverage and improved calibration, enhancing epidemiological decision making.
We use PDPs with confidence bands to explain HPO results.
problem Lack of explainable insights into HPO results.
method Use IML techniques, specifically PDPs, with estimated confidence bands.
result Increased quality of PDPs in relevant sub-regions.
Post-hoc uncertainty quantification improves on pre-trained neural networks without underfitting.
problem Uncertainty quantification in neural networks is underfitting or computationally demanding.
method Gaussian Process Activation function (GAPA) for neuron-level uncertainty, with two methods: GAPA-Free and GAPA-Variational.
result GAPA-Variational outperforms Laplace approximation on most datasets in uncertainty quantification metrics.
A new method combines Gaussian Processes to optimize under uncertainty.
problem Bayesian Optimization's weakness in fitting Gaussian Processes.
method Wasserstein Barycenter Gaussian Process (WBGP) approach.
result WBGP-BO converges to the optimum, improving on vanilla BO.
Hyperparameter tuning is an omnipresent problem in machine learning as it is an integral aspect of obtaining the state-of-the-art performance for any model. Most often, hyperparameters are optimized just by training a model on a grid of possible hyperparameter values and taking the one that performs best on a validatio…
A new method for optimizing hyperparameters using conformalized quantile regression.
problem Optimizing hyperparameters with strong assumptions about noise.
method Conformalized quantile regression for more realistic modeling.
result Quicker convergence on empirical benchmarks.
A two-stage GPR framework with automatic kernel search and subsampling improves prediction accuracy and efficiency.
problem Inaccurate predictions due to misspecified mean and kernel functions in Gaussian Process Regression.
method Two-stage GPR, automatic kernel search, subsampling for hyperparameter initialization.
result Competitive or better performance compared to full dataset training, robust on real-world datasets.
OptFormer learns universal HPO from diverse datasets.
problem Learning HPO from experiments with different hyperparameters.
method Text-based Transformer framework for joint policy and function prediction.
result OptFormer can imitate multiple HPO algorithms and improve predictions.
Automated Budget Constrained Training optimizes model training under time constraints.
problem Balancing model quality and computational cost in constrained time.
method Developed a hyperparameter optimisation algorithm that learns the relationship between hyperparameters, model quality, and computational cost.
result The algorithm optimally decides whether to terminate or continue training, and what hyperparameters to use.
The standard taxonomy of predictive uncertainty is inconsistent with standard measures.
problem Uncertainty taxonomy and measure inconsistency
method Proof of inconsistency
result Uncertainty is not reducible to data collection
Develops fully Bayesian LVGP for better uncertainty quantification.
problem Uncertainty in qualitative inputs for GP models.
method Maps qualitative inputs to latent variables, uses standard GP over LVs, estimates LVs through ML, develops fully Bayesian approach.
result Significant improvements in prediction accuracy and uncertainty quantification over plug-in approach.
We propose a novel method for closed-form predictive distribution modeling with neural nets. In quantifying prediction uncertainty, we build on Evidential Deep Learning, which has been impactful as being both simple to implement and giving closed-form access to predictive uncertainty. We employ it to model aleatoric un…
New method extracts aleatoric and epistemic uncertainties from regression-based neural networks.
problem Need for principled uncertainty reasoning in machine learning systems.
method Learning evidential distributions for aleatoric and epistemic uncertainties.
result Allows for the simultaneous extraction of both uncertainties without sampling or out-of-distribution data.
Online Passive-Aggressive (PA) learning is a class of online margin-based algorithms suitable for a wide range of real-time prediction tasks, including classification and regression. PA algorithms are formulated in terms of deterministic point-estimation problems governed by a set of user-defined hyperparameters: the a…
Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for …
Method estimates uncertainty in CT reconstructions.
problem Lack of accurate uncertainty estimates in deep-learning CT reconstructions.
method Linearised deep image prior with conjugate Gaussian-linear model error bars and Gaussian surrogate for TV regularisation.
result Method provides superior calibration of uncertainty estimates.
Soil moisture is an important variable that determines floods, vegetation health, agriculture productivity, and land surface feedbacks to the atmosphere, etc. Accurately modeling soil moisture has important implications in both weather and climate models. The recently available satellite-based observations give us a un…
New score helps choose PIML model parameters, reducing ambiguity in model quality.
problem Ambiguity in measuring model quality in PIML due to multi-objective fitting.
method Introduces Physics-Informed Log Evidence (PILE) score in Gaussian process framework.
result PILE minimizes ambiguity in model selection, improving hyperparameter choices.
A new method reduces Volterra kernel complexity and uncertainty quantification.
problem Challenges in modeling nonlinear systems with Volterra series due to high model order.
method Bayesian Tensor Network Volterra kernel machines (BTN-V) using canonical polyadic decomposition.
result Competitive accuracy, enhanced uncertainty quantification, and reduced computational cost.
Bayesian TNKMs automatically infer model complexity and feature relevance.
problem Manual tuning of TN rank and feature dimensions is error-prone and computationally expensive.
method Bayesian approach with hierarchical priors on TN factors for automatic rank and feature selection.
result Superior performance in prediction accuracy, uncertainty quantification, interpretability, and scalability.
We propose a fast inference method for Bayesian nonlinear support vector machines that leverages stochastic variational inference and inducing points. Our experiments show that the proposed method is faster than competing Bayesian approaches and scales easily to millions of data points. It provides additional features …
BSG learns dynamic network spillovers and uncertainty quantification.
problem Identifying indirect spillovers and systemic risk in dynamic networks.
method Bayesian Spillover Graphs using FEVD and Bayesian time series models.
result Significant performance gains over baselines in identifying source and sink nodes.
A new method uses Gaussian Processes to solve power flow problems with uncertain renewable and load inputs.
problem Solving power flow problems with uncertain renewable and load inputs.
method Non-parametric Bayesian inference-based uncertainty propagation using Gaussian Processes.
result The method provides reasonably accurate solutions with fewer samples and time compared to Monte-Carlo simulations.
Develops scalable Bayesian inference methods for neural networks.
problem Lack of model uncertainty in deep learning leading to overconfident predictions.
method Linearised Laplace approximation, conjugate Gaussian-linear models, stochastic gradient descent, sample-based EM algorithm.
result Equips neural networks with model uncertainty using scalable methods.
An exciting branch of machine learning research focuses on methods for learning, optimizing, and integrating unknown functions that are difficult or costly to evaluate. A popular Bayesian approach to this problem uses a Gaussian process (GP) to construct a posterior distribution over the function of interest given a se…
Deterministic neural networks (NNs) are increasingly being deployed in safety critical domains, where calibrated, robust, and efficient measures of uncertainty are crucial. In this paper, we propose a novel method for training non-Bayesian NNs to estimate a continuous target as well as its associated evidence in order …
We design a new algorithm for batch active learning with deep neural network models. Our algorithm, Batch Active learning by Diverse Gradient Embeddings (BADGE), samples groups of points that are disparate and high-magnitude when represented in a hallucinated gradient space, a strategy designed to incorporate both pred…
UncertaintyPlayground simplifies uncertainty estimation in Python.
problem Uncertainty estimation in supervised learning tasks.
method Sparse and Variational Gaussian Process Regressions for normally distributed outcomes, Mixed Density Networks for mixed distributions.
result Fast and simplified uncertainty estimation through Python library.
A new method for effective VAE training using calibrated decoders.
problem Training VAEs requires hyperparameter tuning, leading to inefficiency.
method Calibrated decoders that learn uncertainty and automatically determine information retention.
result Calibrated decoders can simplify VAE training without heuristic modifications.
The paper improves Bayesian optimization by calibrating uncertainty estimates.
problem Improper uncertainty estimates in Bayesian optimization when data is non-stationary.
method Proposes online learning algorithms to maintain calibration on non-i.i.d. data and integrates them into Bayesian optimization.
result Calibrated Bayesian optimization converges to better optima in fewer steps.
Gaussian Processes improve data interpolation from diverse experiments.
problem Interpolation of sparse and inconsistent datasets from various experiments.
method Used Gaussian Processes (GP) for data interpolation, including uncertainty quantification.
result GPs successfully interpolate data and quantify uncertainties, demonstrating consistency across different sources.
A new stochastic method tackles bi-level optimization problems in deep learning.
problem Bi-level optimization problems in deep learning, including hyperparameter optimization and meta learning.
method Turning a BLO problem into a stochastic optimization, using SGLD MCMC and a recurrent algorithm to compute MC-estimated hypergradient.
result Our method is more robust to suboptimal inner optimization and non-unique inner minima, leading to more reliable solutions.
Generative approach speeds hyperparameter tuning for machine learning models.
problem Computational infeasibility of cross-validation and difficulty of fully Bayesian hyper-parameter learning.
method Combines optimization-based approximations and amortization techniques.
result Rapid evaluation of hyper-parameters over grids or ranges, supporting predictive tuning and uncertainty quantification.
Fast algorithm solves BVPs in linear time with probabilistic uncertainty.
problem Solving boundary value problems efficiently and accurately.
method Gauss--Markov prior tailored to BVPs, linear-time computation.
result Probabilistic solution with linear time complexity and comparable quality.
Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inferenc…