Enhances Gaussian process models for handling variable error variances and multiple responses.
problem Limited ability of Gaussian process models to capture abrupt changes and heteroscedastic errors.
method Introduces a novel heteroscedastic Gaussian process (HeGP) framework coupled with variational inference and EM algorithm.
result Effective modeling of multivariate responses with varying error variances.
Model separates overall uncertainty into aleatoric and epistemic components for active learning.
problem Active learning with uncertainty quantification.
method Non-stationary Heteroscedastic Gaussian process model.
result Model separates overall uncertainty into aleatoric and epistemic components.
GGMPs improve non-Gaussian conditional density estimation.
problem Multimodality, heteroscedasticity, and strong non-Gaussianity in conditional density estimation.
method GGMP combines local Gaussian mixture fitting, cross-input component alignment, and per-component heteroscedastic GP training.
result GGMPs improve distributional approximation on synthetic and real-world datasets.
Flexible GP model improves wind power prediction accuracy.
problem Accurate probabilistic prediction of wind power for grid stability.
method Heteroscedastic non-stationary Gaussian process with generalised spectral mixture kernel.
result The proposed model outperforms conventional GP models in wind power prediction.
To restore the historical sea surface temperatures (SSTs) better, it is important to construct a good calibration model for the associated proxies. In this paper, we introduce a new model for alkenone (U37K′) based on the heteroscedastic Gaussian process (GP) regression method. Our nonparametric app…
Generalized autoregressive conditional heteroscedasticity (GARCH) models have long been considered as one of the most successful families of approaches for volatility modeling in financial return series. In this paper, we propose an alternative approach based on methodologies widely used in the field of statistical mac…
Gaussian Process (GP) regression models typically assume that residuals are Gaussian and have the same variance for all observations. However, applications with input-dependent noise (heteroscedastic residuals) frequently arise in practice, as do applications in which the residuals do not have a Gaussian distribution. …
This paper presents a novel approach for approximate integration over the uncertainty of noise and signal variances in Gaussian process (GP) regression. Our efficient and straightforward approach can also be applied to integration over input dependent noise variance (heteroscedasticity) and input dependent signal varia…
Heteroscedastic regression considering the varying noises among observations has many applications in the fields like machine learning and statistics. Here we focus on the heteroscedastic Gaussian process (HGP) regression which integrates the latent function and the noise function together in a unified non-parametric B…
Bayesian optimisation is a sample-efficient search methodology that holds great promise for accelerating drug and materials discovery programs. A frequently-overlooked modelling consideration in Bayesian optimisation strategies however, is the representation of heteroscedastic aleatoric uncertainty. In many practical a…
This paper improves GP for learning complex data distributions.
problem Vanilla Gaussian processes struggle with complex data distributions.
method Introduces scalable GP paradigms with latent variables and variational inference.
result Scalable modulated GPs, especially latent GPs, learn diverse data distributions better.
Optimising black-box functions is important in many disciplines, such as tuning machine learning models, robotics, finance and mining exploration. Bayesian optimisation is a state-of-the-art technique for the global optimisation of black-box functions which are expensive to evaluate. At the core of this approach is a G…
Paper proposes GPM for heteroscedastic PCA estimation.
problem Estimating ground truth from heterogeneous data.
method Generalized power method (GPM) for HQPOC.
result GPM achieves geometrically decreasing distances to ground truth.
Accurately modeling traffic speeds is a fundamental part of efficient intelligent transportation systems. Nowadays, with the widespread deployment of GPS-enabled devices, it has become possible to crowdsource the collection of speed information to road users (e.g. through mobile applications or dedicated in-vehicle dev…
We propose a robust method to estimate heteroscedastic noise models using Student's t-distribution.
problem Identifying cause and effect from bivariate observational data with non-Gaussian noise.
method We propose a novel approach using Student's t-distribution to estimate heteroscedastic noise models, which is more robust and achieves better performance.
result Our estimators are more robust and achieve better overall performance across synthetic and real benchmarks.
This work improves Gaussian process inference using mixtures of experts and nested SMC samplers.
problem High computational and memory costs of Gaussian processes.
method Mixtures of Gaussian process experts with nested SMC samplers.
result Significantly improved inference compared to importance sampling.
Novel framework for contextual anomaly detection models uncertainty.
problem Identifying anomalies in target variables influenced by contextual variables.
method Normalcy score (NS) framework using heteroscedastic Gaussian process regression.
result NS outperforms state-of-the-art methods in detection accuracy and interpretability.
Study on signal detection in heteroscedastic Gaussian sequences with sparse alternatives.
problem Signal detection in heterogeneous Gaussian sequences with unknown means and known covariance.
method Characterization of minimax separation radius and derivation of matching upper and lower bounds.
result Matching minimax upper and lower bounds for signal detection in heteroscedastic Gaussian sequences.
A simple method treats heteroscedastic variance variatively, improving model calibration and sample quality.
problem Brittle optimization impacts model likelihoods for mean and variance estimation.
method Proposes a variational approach to heteroscedastic variance, improving predictive mean and variance calibration.
result The proposed method significantly improves parameter calibration and sample quality for regression and VAEs.
Improved heteroscedastic regression using neural networks with provably accurate mean estimates and calibrated variance.
problem Optimizing neural network parameters for heteroscedastic regression leads to suboptimal mean and variance estimates.
method Two simple modifications to optimization to retain accuracy of mean-only models and offer best-in-class variance calibration.
result Mean estimates from the proposed method are provably as accurate as those from a homoscedastic model.
Bayesian optimisation (BO) is widely used to optimise stochastic black box functions. While most BO approaches focus on optimising conditional expectations, many applications require risk-averse strategies and alternative criteria accounting for the distribution tails need to be considered. In this paper, we propose ne…
HET-XL improves heteroscedastic classifiers for large-scale image classification.
problem Scaling heteroscedastic classifiers to handle large numbers of classes and tuning the temperature hyperparameter.
method HET-XL, a heteroscedastic classifier with independent parameter count from the number of classes, learns the temperature hyperparameter directly from training data.
result HET-XL requires 14X fewer additional parameters and performs better than baseline heteroscedastic classifiers on large image classification datasets.
SVGP KAN integrates uncertainty quantification into Kolmogorov-Arnold networks.
problem Uncertainty quantification in scientific machine learning models.
method Sparse variational Gaussian process inference with Kolmogorov-Arnold topology.
result Demonstrated ability to distinguish aleatoric and epistemic uncertainty in various scientific applications.
This paper presents a data-driven approach to model planar pushing interaction to predict both the most likely outcome of a push and its expected variability. The learned models rely on a variation of Gaussian processes with input-dependent noise called Variational Heteroscedastic Gaussian processes (VHGP) that capture…
This paper introduces a multi-output Gaussian process for censored data.
problem Modeling bias in censored data using correlations between multiple outputs.
method Heteroscedastic multi-output Gaussian process with input-dependent noise and variational inference.
result The model better estimates the true process under complex censoring dynamics.
Algorithm identifies and corrects noisy labels using Gaussian process regression.
problem Detecting and correcting real-valued noisy labels from mixed data.
method Gaussian process regression with heteroscedastic noise model and leave-one-out cross-validation.
result The method can pinpoint corrupted sample points and improve regression models.
New method learns DAGs from noisy data without identifiability assumptions.
problem Learning DAGs from non-identifiable Gaussian models with heteroscedastic noise.
method Mixed-integer programming framework for medium-sized problems.
result Asymptotically optimal solution with early stopping criterion.
We improve optimization for data with varying variance.
problem Optimizing data with varying variance.
method Generalized learning and optimization frameworks for data-driven optimization.
result Asymptotic and finite sample guarantees for stochastic programs.
This paper introduces a novel clustering algorithm for heteroscedastic Gaussian data without needing to know the number of clusters.
problem Clustering heteroscedastic Gaussian data without prior knowledge of the number of clusters.
method Introduces a novel cost function and fixed-point analysis to estimate centroids, introduces Wald kernel for measurement plausibility, and derives CENTRE-X algorithm.
result CENTRE-X algorithm can estimate centroids without prior knowledge of the number of clusters and performs comparably to standard algorithms K-means and Mean-Shift.
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly …
We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of…
Simple method improves deep classifier accuracy under noisy labels.
problem Training deep classifiers with noisy labels.
method Probabilistic approach using temperature parameterized softmax.
result Improves accuracy, log-likelihood and calibration on noisy datasets.
Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.
problem Minimizing regret in Gaussian process bandits with unknown reward functions.
method New upper bound on maximum posterior variance, refined MVR and PE algorithms.
result Optimal regret bounds for noiseless, varying noise, and RKHS norms.
The log-likelihood loss in heteroscedastic neural networks can lead to poor parameter estimates.
problem Capturing aleatoric uncertainty in deep learning models.
method Examine the log-likelihood loss in conjunction with gradient-based optimizers and propose an alternative formulation, β-NLL. result Using an appropriate β largely mitigates the issue of poor parameter estimates. The role of uncertainty quantification (UQ) in deep learning has become crucial with growing use of predictive models in high-risk applications. Though a large class of methods exists for measuring deep uncertainties, in practice, the resulting estimates are found to be poorly calibrated, thus making it challenging to …
Improves deep learning performance on noisy datasets using inverse-variance weighting.
problem Heteroscedastic regression with varying noise levels.
method Batch Inverse-Variance (BIV) loss function for neural networks.
result Significantly improves network performance on noisy datasets compared to other methods.
ALPCAH improves PCA for noisy data by estimating sample-wise noise variances.
problem Noisy data with varying noise levels in different samples.
method Sample-wise heteroscedastic PCA with tail singular value regularization.
result Improves subspace basis estimation for low-rank data.
Bayesian optimisation tackles stochastic MPC hyper-parameter tuning.
problem Fine-tuning hyper-parameters in stochastic MPC models.
method Heteroscedastic Bayesian optimisation framework.
result Framework effectively tunes hyper-parameters in control problems.
Study on online regression with noise, achieving near-optimal regret bounds.
problem Online generalized linear regression with stochastic noise.
method Sharp analysis of FTRL algorithm for stochastic label noise.
result Achieved near-optimal regret bounds for O(σ2dlogT)+o(logT). SA-REMBO adapts to nonstationary high-dimensional optimization.
problem Bayesian Optimization in high-dimensional spaces is limited by the curse of dimensionality and rigidity of global assumptions.
method SA-REMBO uses multiple random Gaussian embeddings and an index variable to adaptively select the best embedding for the optimization problem.
result SA-REMBO outperforms traditional REMBO and other low-rank BO methods across synthetic and real-world benchmarks.
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
QS-BO optimizes functions using only rank-based feedback.
problem Optimizing expensive functions with unreliable or unavailable metric values.
method Quantile-scaling pipeline to convert ranks into Gaussian targets.
result QS-BO consistently achieves lower objective values and is statistically significant.
Optimizes cryptocurrency portfolios using MNTS GARCH model.
problem Optimizing cryptocurrency portfolios with non-Gaussian return dynamics.
method Multivariate normal tempered stable (MNTS) GARCH model for non-Gaussian returns, Foster-Hart risk optimization.
result Foster-Hart optimization yields a more profitable portfolio with better risk-return balance.
Sparse Gaussian Processes simplify GP inference for large datasets.
problem Efficiently handling large datasets in Gaussian Process models.
method Sparse Gaussian Processes combined with variational inference.
result Sparse GPs enable approximate inference with reduced memory and computational requirements.
PNNs model aleatoric uncertainty in scientific machine learning with high accuracy.
problem Aleatoric uncertainty in scientific systems with unequal variance.
method Developed a probabilistic distance metric to optimize PNN architecture and used it in material science applications.
result PNNs yield remarkably accurate output mean estimates and high correlation in predicted intervals.
ALPCAHUS clusters data from multiple subspaces with varying noise.
problem Heteroscedastic data with varying noise levels.
method Develops a heteroscedastic PCA method for subspace clustering.
result Improves subspace clustering by accounting for sample-wise noise variances.
We introduce a generalisation of the well-known ARCH process, widely used for generating uncorrelated stochastic time series with long-term non-Gaussian distributions and long-lasting correlations in the (instantaneous) standard deviation exhibiting a clustering profile. Specifically, inspired by the fact that in a var…
No-regret optimization for time-varying functions using uncertainty injection.
problem Optimizing time-varying functions with no-regret in bandit feedback.
method W-SparQ-GP-UCB, incorporating uncertainty injection and additional queries.
result Achieves no-regret with a vanishing number of additional queries per iteration.