arXiv research

Develops a method to find costly high-confidence errors in black box models.

Generative models need per-sample confidence scores to improve quality and stability.

The paper proposes using entropy to assess model confidence.

Safety-critical applications require machine learning models that output accurate and calibrated probabilities. While uncalibrated deep networks are known to make over-confident predictions, it is unclear how model confidence is impacted by the variations in the data, such as label noise or class size. In this paper, w…

The paper shows over-confidence in models isn't just due to over-parametrization.

Confidence-based filtering reveals latent structure in diffusion models.

Positive-confidence (Pconf) classification [Ishida et al., 2018] is a promising weakly-supervised learning method which trains a binary classifier only from positive data equipped with confidence. However, in practice, the confidence may be skewed by bias arising in an annotation process. The Pconf classifier cannot be…

Method constructs confidence regions for linear models with arbitrary predictors.

New method constructs confidence sets for GLMs via game theory.

Framework for confidence estimation in deep CT reconstructions.

In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model. A natural measure of confidence is $\|F({\bf x})\|_\infty$ (i.e. how confident $F$ is about its prediction?). We start by analyzing an adversarial training…

We propose a novel confidence scoring mechanism for deep neural networks based on a two-model paradigm involving a base model and a meta-model. The confidence score is learned by the meta-model observing the base model succeeding/failing at its task. As features to the meta-model, we investigate linear classifier probe…

New methods improve confidence set calibration in complex models.

New attacks can infer model training membership using only label predictions, not confidence.

Develops confidence intervals for ECE, a measure of model calibration.

Introduces CCR for constructing confidence regions from conformal predictions.

Paper introduces a novel method for estimating model confidence in deep neural classifiers.

Adversarial training yields robust models against a specific threat model, e.g., $L_\infty$ adversarial examples. Typically robustness does not generalize to previously unseen threat models, e.g., other $L_p$ norms, or larger perturbations. Our confidence-calibrated adversarial training (CCAT) tackles this problem by b…

New method calibrates confidence for object detection and segmentation models.

We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to cons…

CONTINA provides adaptive confidence intervals for traffic demand prediction.

New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.

Paper presents robust confidence sequences for means with known moment bounds and arbitrary corruption.

Confidence-based deferral works well in many scenarios but fails in specific cases.

Random forest training yields confidence intervals for generalization error.

CCAC calibrates DNN classifiers on OOD datasets by separating mis-classified samples.

Paper presents methods to create stock price confidence intervals using LSTM models.

We present a new method for computing ASR word confidences that effectively mitigates the effect of ASR errors for diverse downstream applications, improves the word error rate of the 1-best result, and allows better comparison of scores across different models. We propose 1) a new method for modeling word confidence u…

In classification applications, we often want probabilistic predictions to reflect confidence or uncertainty. Dropout, a commonly used training technique, has recently been linked to Bayesian inference, yielding an efficient way to quantify uncertainty in neural network models. However, as previously demonstrated, conf…

Confidence intervals and joint confidence sets are constructed for the nonparametric calibration of exponential Lévy models based on prices of European options. To this end, we show joint asymptotic normality in the spectral calibration method for the estimators of the volatility, the drift, the jump intensity and the …

Paper defines and solves a problem in representation learning to ensure fairness with high confidence.

Develops methods for estimating and providing confidence bands in sparse high-dimensional additive models.

This paper improves confidence measurement in deep metric learning models.

Novel framework for unbiased confidence estimates in object detection.

A new method for predicting with confidence for complex models.

Improved confidence bounds for linear logistic model with applications to bandits.

The paper provides a method to find optimal machine learning model parameters with confidence.

We develop a novel procedure for constructing confidence bands for components of a sparse additive model. Our procedure is based on a new kernel-sieve hybrid estimator that combines two most popular nonparametric estimation methods in the literature, the kernel regression and the spline method, and is of interest in it…

New method improves auto-labeling accuracy by optimizing confidence functions.

Improved AI lung ultrasound segmentation using expert confidence values.

This paper rethinks confidence calibration under covariate shifts.

General predictive models do not provide a measure of confidence in predictions without Bayesian assumptions. A way to circumvent potential restrictions is to use conformal methods for constructing non-parametric confidence regions, that offer guarantees regarding validity. In this paper we provide a detailed descripti…

Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase the trust, interpretability and usefulness of virtual screening models in drug discovery, techniques t…

Calibrates network confidence for unsupervised domain adaptation.

Entropy-based decoding improves DLM sampling efficiency.

Pathology reports contain useful information such as the main involved organ, diagnosis, etc. These information can be identified from the free text reports and used for large-scale statistical analysis or serve as annotation for other modalities such as pathology slides images. However, manual classification for a hug…

CCE improves anomaly detection metrics by measuring both confidence and consistency.

Improved regret bounds for logistic bandits via novel confidence set construction.