Proposes a new method to find overconfident predictions in classifiers.
problem Challenges in assessing classifier accuracy without labeled test data.
method Facility locations utility model and greedy query algorithm.
result Greedy query algorithm finds overconfident unknown unknowns more effectively.
New method improves deep learning models' uncertainty estimates.
problem Overconfidence in deep learning predictions.
method Develops a novel training algorithm using conformal inference.
result Produces more reliable uncertainty estimates without sacrificing accuracy.
Bayesian framework improves deep classifier reliability.
problem Overconfident models under dataset shift.
method Bayesian inference with out-of-distribution data augmentation.
result Reliable uncertainty estimates for deep classifiers.
In this article, we analyze the application of options contract in special commodity supply chain such as fresh agricultural products. This problem is discussed in the point of the retailer. When spot market and future market are both available, we discuss how the retailer chooses the optimal production. Furthermore, o…
The thesis tackles overconfident approximations in simulation-based inference.
problem Overconfident conclusions from machine learning approximations in statistical analyses.
method Introduces balancing and Bayesian neural networks to reduce overconfidence.
result Balancing and Bayesian neural networks lead to less overconfident approximations.
Bayesian ReLU nets fix asymptotic overconfidence with infinite features.
problem Bayesian ReLU nets can be asymptotically overconfident far from training data.
method Extend finite ReLU BNNs with infinite ReLU features via a Gaussian process.
result The resulting model is asymptotically maximally uncertain far from the data.
Deep neural networks (DNNs) have excellent representative power and are state of the art classifiers on many tasks. However, they often do not capture their own uncertainties well making them less robust in the real world as they overconfidently extrapolate and do not notice domain shift. Gaussian processes (GPs) with …
Regularized mixtures improve inflation and interest rate forecasts, especially correcting overconfidence.
problem Improving density forecasts of Eurozone inflation and real interest rates.
method Construct regularized mixtures of density forecasts with various objectives and penalties.
result Regularized mixtures outperform individual forecasters, especially correcting overconfidence.
Improved conformal prediction for better conditional coverage of classifier predictions.
problem Achieving exact conditional coverage in finite samples for prediction sets.
method Developed a variant of conformal prediction targeting coverage conditional on confidence and trust score.
result Empirically improved conditional coverage properties compared to standard conformal prediction.
Bayesian approach fixes overconfidence in ReLU networks, even slightly.
problem Overconfidence in ReLU networks far from training data.
method Theoretical analysis of approximate Gaussian distributions on ReLU weights, and empirical validation.
result Even a simplistic Bayesian approximation fixes overconfidence issues.
Generates confident out-of-distribution samples to improve classifier robustness.
problem Overconfidence in deep learning models on out-of-distribution inputs.
method Uses a GAN to generate out-of-distribution samples that the classifier is confident on, maximizing entropy.
result Shows effectiveness on handwritten characters and natural images datasets.
The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their…
Paper proposes a method to improve deep neural networks' confidence estimates.
problem Overconfident predictions limit practical use of deep neural networks in safety-critical applications.
method Proposes a novel loss function, Correctness Ranking Loss, to regularize class probabilities.
result The method produces well-ranked confidence estimates and is effective for out-of-distribution detection and active learning.
New CNN approach reduces overconfidence in object classification predictions.
problem Overconfident predictions from deep models, especially SoftMax layer.
method Introduces CNN probabilistic approach using Logit layer for Bayesian inference.
result Proposed approach shows promising performance compared to SoftMax.
DRO-NPE improves neural posterior estimation by reducing overconfidence and overfitting.
problem Overconfident and unreliable posteriors in simulation-based inference with limited simulation budgets.
method Distributionally robust approach using Wasserstein ambiguity set and KL-based metrics.
result Consistently improves coverage and calibration across benchmark tasks.
Most classifiers operate by selecting the maximum of an estimate of the conditional distribution p(y∣x) where x stands for the features of the instance to be classified and y denotes its label. This often results in a {\em hubristic bias}: overconfidence in the assignment of a definite label. Usually, the observa…
New method improves variational inference for dynamical systems without extra computational cost.
problem Inexact variational inference leading to overconfident posterior and overestimation of process noise.
method Proposes a non-factorised posterior distribution for Gaussian process transition functions.
result Improves accuracy of posterior over transition function and process noise estimation.
Novel neural network models quantify uncertainty for deep classifiers.
problem Deep networks' overconfidence and ignorance about uncertainty.
method Variational autoencoders and GANs generate out-of-distribution samples.
result Better uncertainty estimates for in- and out-of-distribution samples.
Bayesian neural networks struggle with uncertainty estimates between regions.
problem Limited expressiveness of predictive uncertainty estimates in between regions.
method Compared mean-field variational inference (MFVI) with linearised Laplace approximation.
result Linearised Laplace approximation handles 'in-between' uncertainty better.
Detecting test samples drawn sufficiently far away from the training distribution statistically or adversarially is a fundamental requirement for deploying a good classifier in many real-world machine learning applications. However, deep neural networks with the softmax classifier are known to produce highly overconfid…
This work improves neural network trustworthiness through uncertainty estimation.
problem Overconfident neural networks lead to poor performance under distribution shifts.
method Develops a general uncertainty framework for neural networks, including classification with rejection.
result Improves model trustworthiness and robustness in decision-making tasks.
New BNN method reduces training time and model size.
problem Overconfident predictions in deep learning models.
method Designing STF-BNN for efficient scaling of BNNs.
result Significantly reduces training time and model size compared to vanilla BNNs.
The paper proposes a method to estimate predictive uncertainty in neural networks using gradient uncertainty.
problem Overconfidence in neural network predictions, especially in safety-critical applications.
method Incorporates gradient uncertainty into posterior sampling for efficient predictive uncertainty estimation.
result The proposed method effectively estimates predictive uncertainty on MNIST and notMNIST datasets.
Adapts GAN-based robustness training to live traffic data.
problem Improving classifier robustness to out-of-distribution samples in real-world traffic.
method Adaptive regularization technique based on maximum predictive probability score.
result Significantly improved detection of out-of-distribution samples without degrading in-distribution performance.
A technique called 'prior laundering' uses legacy reconstructions to create uncertainty in Bayesian inverse problems.
problem Uncertainty in Bayesian inverse problems when data is uninformative.
method Using an archive of legacy reconstructions to create uncertainty in the posterior distribution, averaging the legacy posterior over measurements.
result The uncertainty reported in the posterior is inherited from the legacy reconstructions, not from the data itself.
The paper addresses poor calibration in fine-tuned LLMs after preference alignment.
problem Poor calibration in fine-tuned Large Language Models (LLMs) after preference alignment.
method Proposes a calibration-aware fine-tuning approach to restore calibration without compromising model performance.
result Demonstrates the effectiveness of the proposed methods through extensive experiments.
New method improves OOD detection by integrating diffusion models into discriminator models.
problem Overconfidence in discriminator models leads to poor OOD detection.
method Integrates diffusion models into discriminator and generation models to mitigate overconfidence.
result Demonstrates significant improvement in AUROC scores for challenging datasets.
New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.
problem Lack of metrics capturing model reliability under confidence thresholds.
method Introducing CWSA and CWSA+ metrics that reward confident accuracy and penalize overconfident mistakes.
result CWSA and CWSA+ outperform classical metrics in trust-sensitive tests.
Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.
problem Bayesian deep learning struggles with model-specific weight-space priors that are hard to interpret and specify.
method Apply a Dirichlet prior in predictive space and perform approximate function-space variational inference.
result The approach improves uncertainty quantification, scalability, and adversarial robustness in large-scale image classification.
BLoB fine-tunes LLMs with Bayesian methods to improve uncertainty estimation.
problem Overconfidence in LLMs during inference for domain-specific tasks.
method Continuous Bayesian low-rank adaptation during fine-tuning.
result BLoB improves generalization and uncertainty estimation in LLMs.
Combines Laplace approximations of deep networks for better uncertainty quantification.
problem Overconfident predictions on outliers in deep learning models.
method Gaussian mixture model posterior using weighted sum of Laplace approximations of pre-trained deep networks.
result Mitigates overconfidence 'far away' from training data.
Bayesian models for networks are often misspecified, leading to overconfident inference.
problem Real-world networks violate assumptions of geometry and link function in latent space models.
method Proposes a generalized posterior framework for random geometric graphs, using Link-Sequential R-SafeBayes to adaptively tune posterior regularization.
result Improved calibration and better link prediction performance demonstrated on synthetic and real-world networks.
Time series foundation models are well-calibrated, improving over baseline models.
problem Calibration of time series foundation models for practical applications.
method Systematic evaluations of five time series foundation models and two baselines, assessing calibration, prediction heads, and long-term forecasting.
result Time series foundation models are consistently better calibrated than baseline models and do not show over- or under-confidence.
New research challenges the independence assumption in neurosymbolic learning, leading to overconfident predictions and unrepresentable uncertainty.
problem The independence assumption in neurosymbolic learning systems can lead to overconfident predictions and hinder uncertainty quantification.
method The study proves the limitations of the independence assumption and introduces new loss functions that are non-convex and difficult to optimise.
result Neurosymbolic learning systems using the independence assumption are prone to overconfidence and cannot represent uncertainty over multiple valid options.
Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.
problem Estimating noise transition matrix from noisy data.
method Total variation regularization to encourage distinguishable predicted probabilities.
result Consistent estimator of the noise transition matrix under mild assumptions.
OrthoGrad improves neural calibration by constraining gradient updates orthogonally.
problem Overconfidence in neural networks, leading to poor uncertainty estimates.
method Orthogonal gradient updates to optimize for decision boundaries and reduce overconfidence.
result Significant improvements in test loss, predictive entropy, and confidence measures.
A novel post-hoc calibration method reduces neural network calibration errors.
problem Neural networks produce poorly calibrated probabilities, leading to underconfidence and overconfidence.
method Probability bounding (PB) via box-constrained softmax (BCSoftmax) function.
result Consistently reduces calibration errors on four real-world datasets.
QLA improves Bayesian uncertainty estimation for DNNs without increasing computational cost.
problem Overconfident out-of-distribution predictions from DNNs.
method Proposes Quadratic Laplace Approximation (QLA) to improve Bayesian uncertainty quantification.
result QLA yields modest yet consistent uncertainty estimation improvements over Linearized Laplace Approximation (LLA) on five regression datasets.
Bayesian optimization improves efficiency with semi-supervised learning.
problem Efficiently find global optima of expensive functions.
method Density ratio estimation combined with semi-supervised learning.
result Improved accuracy in identifying global optima with unlabeled data.
HyperGAN generates diverse neural network parameters for improved performance and uncertainty.
problem Overconfidence of neural networks in out-of-distribution data.
method Generative model using a novel mixer to learn a distribution of neural network parameters.
result HyperGAN can generate parameters that perform competitively with fully supervised learning and provide better uncertainty estimates.
Proposes a new calibration error estimator for deep neural networks.
problem Improves calibration of deep neural networks, especially for canonical calibration.
method Uses a Dirichlet kernel density estimate to create a low-bias, trainable calibration error estimator.
result Asymptotically converges to true Lp calibration error, enabling efficient estimation and mini-batch updates. The main aim of this work is to incorporate selected findings from behavioural finance into a Heterogeneous Agent Model using the Brock and Hommes (1998) framework. Behavioural patterns are injected into an asset pricing framework through the so-called `Break Point Date', which allows us to examine their direct impact.…
Bayes-TrEx finds in-distribution examples for model inspection.
problem Challenges in interpreting neural networks, especially high-confidence failures and ambiguous classifications.
method Bayesian sampling approach to find in-distribution examples with specified prediction confidence.
result Bayes-TrEx enables more flexible holistic model analysis than just inspecting the test set.
This work approximates neural networks with Gaussian processes for more efficient active learning.
problem Efficiently updating uncertainty estimates for neural networks without retraining.
method Approximating Bayesian neural networks with Gaussian processes.
result The proposed approach outperforms state-of-the-art methods in experiments.
Flexible evidential deep learning improves uncertainty quantification in machine learning.
problem Overconfident predictions in machine learning models can lead to serious consequences.
method Proposes flexible evidential deep learning (F-EDL) to model uncertainty over class probabilities using a flexible Dirichlet distribution.
result Empirically demonstrates state-of-the-art uncertainty quantification performance across diverse scenarios.
Improved Variational Autoencoder for better out-of-distribution detection.
problem Overconfident uncertainty estimates on out-of-distribution data in Variational Autoencoders.
method Employing negative samples in an adversarial training scheme.
result Reduced overconfident likelihood estimates of out-of-distribution inputs.
Automates zero-shot classification by scoring and weighting prompts.
problem Improving zero-shot accuracy through prompt ensembling.
method Automatic prompt scoring and weighting method.
result Method outperforms existing techniques on various benchmarks.
New method calibrates neural SBI to avoid overconfident posteriors.
problem Overconfident posteriors in SBI due to inaccurate uncertainty quantification.
method Introduces a calibration term into neural model training objective, enabling end-to-end backpropagation.
result Achieves competitive or better coverage and posterior density than existing methods.