Algorithm constructs confidence sets for deep neural networks with PAC guarantees.
problem Ensuring reliable predictions for deep neural networks with high confidence.
method Combines calibrated prediction and learning theory bounds.
result Constructs PAC confidence sets for various deep models.
Softmax confidence misrepresents uncertainty in neural networks.
problem Neural networks fail to increase uncertainty on out-of-distribution data.
method Investigates two implicit biases in softmax confidence.
result Softmax confidence correlates with epistemic uncertainty due to decision boundary structure and deep network filtering.
Deep Confidence provides efficient error estimation for deep neural networks.
problem Estimating the reliability of predictions from deep learning models.
method Snapshot Ensembling and conformal prediction.
result Deep Confidence generates narrower confidence intervals than alternative methods.
Structured dropout improves confidence calibration in convolutional neural networks.
problem Poor confidence calibration in convolutional neural networks using dropout.
method Structured dropout to promote model diversity and improve calibration.
result Structured dropout reduces calibration error compared to naive dropout.
Paper proposes a method to improve deep neural networks' confidence estimates.
problem Overconfident predictions limit practical use of deep neural networks in safety-critical applications.
method Proposes a novel loss function, Correctness Ranking Loss, to regularize class probabilities.
result The method produces well-ranked confidence estimates and is effective for out-of-distribution detection and active learning.
ReLU networks often give high confidence far from training data, new technique mitigates this.
problem ReLU networks produce high confidence predictions far from training data, which is undesirable.
method Proposed a new robust optimization technique similar to adversarial training to enforce low confidence predictions.
result The technique reduces confidence of predictions far from training data while maintaining test error on the original task.
A simple method for neural network confidence scores.
problem Measuring confidence in neural network predictions.
method Distance-based loss or Adversarial Training for data embedding.
result Significant improvement over traditional confidence scores.
Calibrates network confidence for unsupervised domain adaptation.
problem Calibrating a model trained on a source domain to a target domain without labeled data.
method Estimates network accuracy on the target domain and calibrates prediction confidence directly in the target domain.
result Significantly outperforms existing methods across standard datasets.
Proposes a method to calibrate deep neural network predictions using stochastic inferences.
problem Improving confidence calibration in deep neural networks.
method Interprets stochastic regularization as Bayesian model, designs a variance-weighted loss function.
result Demonstrates significant improvement in confidence calibration and classification accuracy.
Paper improves confidence intervals and variance estimation for deep learning models.
problem Improving confidence intervals and variance estimation in deep learning models.
method Residual-based framework for conditional variance estimation; robust bootstrap procedure for confidence intervals.
result First non-asymptotic bounds for variance estimation using ReLU networks.
This paper improves lottery ticketing by calibrating network confidence.
problem Uncalibrated confidence in lottery tickets leads to overconfidence and poor performance.
method The paper introduces various calibration strategies and explores their impact on lottery tickets.
result Calibration mechanisms consistently improve lottery ticket performance, even under distribution shifts.
New method improves ASR word confidence for diverse applications.
problem Mitigating ASR errors and improving word error rate.
method Heterogeneous Word Confusion Network (HWCN) with score calibration.
result Word sequence with best overall confidence is more accurate than 1-best result.
The paper presents a method to compute trusted confidence bounds for LECs in CPS.
problem Non-transparent predictions of LECs make CPS safety challenging.
method Inductive Conformal Prediction (ICP) and Triplet Network architecture.
result Efficient real-time computation of trusted confidence bounds.
New method identifies diffusion sources on networks with statistical confidence.
problem Identifying sources of diffusion on networks without restrictive assumptions.
method Statistical framework and confidence set inference approach based on hypothesis testing.
result Efficiently produces a small subset of nodes covering the source node with any confidence level.
A method to reduce computation by dynamically sacrificing accuracy in deep neural networks.
problem Balancing computational effort and classification accuracy in deep neural networks.
method A cascade of deep neural networks with dynamically set confidence thresholds based on softmax outputs.
result Reduces 15%-50% in MAC operations with a 1% accuracy degradation.
Improves confidence calibration in neural networks by smoothing labels based on class similarity.
problem Improving confidence calibration in deep neural networks for safety-critical applications.
method Proposes a novel label smoothing technique where label values are based on similarities with the reference class, using different similarity measurements.
result Consistently outperforms state-of-the-art calibration techniques on various datasets and network architectures.
The paper proposes a method to improve neural network confidence for out-of-distribution detection.
problem Improving neural networks' ability to recognize when predictions are incorrect.
method A method of learning confidence estimates for neural networks that produces interpretable outputs.
result The proposed method outperforms existing techniques in out-of-distribution detection.
Paper proposes new method for time series confidence intervals using LSTM.
problem Constructing accurate confidence intervals for multivariate time series.
method Uses Long Short Term Memory Network (LSTM) and novel block bootstrap techniques.
result Demonstrates improved accuracy in constructing confidence intervals.
Survey of methods to calibrate neural network predictions.
problem Ensuring neural networks provide accurate confidence levels.
method Empirical comparison of calibration methods.
result Various techniques for calibrating neural networks.
A new confidence scoring method using whitebox models and linear probes.
problem Improving confidence scores for deep neural networks.
method Two-model paradigm with a base model and a meta-model using linear classifier probes.
result The method outperforms baselines in filtering tasks.
Paper proposes a method to predict deep neural network confidences with guarantees.
problem Quantifying uncertainty in deep neural networks for safety-critical applications.
method Uses Clopper-Pearson confidence intervals and histogram binning for calibrated prediction.
result Demonstrates the effectiveness of predicted confidences in improving DNN performance and safety.
New method HNCI for evaluating treatment effects in network interference.
problem Evaluating the effectiveness of treatments or policies under network interference.
method High-dimensional network causal inference (HNCI) using linear regression with latent homogeneity.
result Valid confidence intervals and sets for average direct treatment effect and neighborhood size.
New method estimates grouping loss in neural networks to improve confidence scores.
problem Improving confidence scores in neural networks to reflect true posterior probabilities.
method Proposed an estimator to approximate the grouping loss.
result Modern neural networks exhibit grouping loss, especially in distribution shifts.
The paper investigates how dataset quality and heterogeneity affect model confidence in machine learning.
problem Understanding how dataset quality and heterogeneity impact model confidence in machine learning.
method The study uses theoretical explanations and experimental demonstrations to investigate the effects of dataset size, label noise, and class heterogeneity on model confidence.
result Label noise reduces model confidence, while reduced dataset size increases it, and class heterogeneity leads to inconsistent confidence across classes.
CCAC calibrates DNN classifiers on OOD datasets by separating mis-classified samples.
problem Calibrating DNN classifiers on out-of-distribution datasets is challenging.
method CCAC introduces an auxiliary class to map DNN output to calibrated confidence, separating mis-classified from correctly classified samples.
result CCAC consistently outperforms prior methods on various DNN models, datasets, and applications.
Novel framework for unbiased confidence estimates in object detection.
problem Unbiased confidence estimates for safety-critical object detection.
method Combines regression output with additional information for calibration.
result Calibrated confidence estimates for image location and scale.
Improved drug-protein interaction prediction using FTL method.
problem Predicting drug-protein interactions from noisy data with uncertain labels.
method Filtered Transfer Learning (FTL) method that fine-tunes a deep neural network across multiple tiers of data confidence.
result FTL method outperforms deep neural networks trained on single confidence ranges.
DeepLR constructs confidence intervals for neural networks with asymmetric expansions.
problem Uncertainty estimation for neural network predictions.
method Likelihood-ratio-based approach for constructing asymmetric confidence intervals.
result DeepLR offers asymmetric intervals expanding in regions with limited data.
ConfGCN estimates labels and confidences in graph-based semi-supervised learning.
problem Predicting node properties in graphs with limited labeled data.
method ConfGCN uses graph convolutional networks to estimate labels and confidences jointly, improving upon anisotropic neighborhood aggregation.
result ConfGCN outperforms state-of-the-art baselines on standard benchmarks.
Mutual teaching improves graph models with less labeled data.
problem Training graph models with limited labeled data.
method Dual model training with mutual teaching strategy.
result Significant performance improvement with less labeled data.
New priors improve Bayesian neural networks without cooling.
problem Bayesian neural networks underfit on clean datasets.
method Introduce DirClip and confidence priors to replace cooling.
result DirClip and confidence priors outperform cold posterior.
Binary classifier learned from only positive data with confidence.
problem Learning binary classifier from only positive data.
method Simple empirical risk minimization framework, model-independent and optimization-independent.
result Consistency and estimation error bound established for Pconf classification.
New approach ensures neural networks know when they don't know.
problem Neural networks over-confident far from training data in safety-critical applications.
method Proposes a new approach to out-of-distribution detection (OOD) with provable guarantees.
result First certificates for low confidence predictions in a neighborhood of an out-distribution point.
New method calibrates confidence for object detection and segmentation models.
problem Intrinsically miscalibrated confidence estimates in object detection and segmentation models.
method Introduces multivariate confidence calibration for object detection and segmentation, extending ECE.
result Improves calibration, positively impacts segmentation quality.
New method for accurate uncertainty estimation in deep learning predictions.
problem Insufficient methods for assessing prediction uncertainty in deep learning.
method Valid non-parametric bootstrap method for deep neural networks.
result Accurate confidence intervals and simultaneous confidence bands for survival data.
Study finds root vertex in large networks with high probability.
problem Finding the root vertex in large growing networks.
method Constructs confidence sets for the root vertex in various random network models.
result Confidence sets of size independent of the number of vertices contain the root vertex with high probability.
EC method calibrates neural networks by matching average confidence to correct label proportion.
problem Overoptimism in neural network prediction confidence.
method Expectation consistency (EC) post-training rescaling of weights.
result EC achieves similar calibration performance to temperature scaling (TS) but is based on a principled Bayesian principle.
Proposes a new confidence criterion for deep neural networks to predict failures.
problem Predicting failures in deep neural networks.
method Introduces True Class Probability (TCP) as a new confidence criterion and proposes a learning scheme to estimate it.
result The proposed approach consistently outperforms existing methods in failure prediction.
DBLE improves confidence calibration of DNNs by learning distances in representation space.
problem Poor confidence calibration of deep neural networks (DNNs).
method DBLE trains a confidence model jointly with the classification model, using distances in the representation space.
result DBLE outperforms alternative single-model confidence calibration approaches and ensemble methods.
signADAM++ uses confidence functions to train deep neural networks faster.
problem Training deep neural networks efficiently and effectively.
method Integrates sign operation into ADAM and introduces a confidence function.
result signADAM++ generates more sparse gradients and improves performance.
Improves binary classification from positive data with skewed confidence.
problem Skewed confidence in positive data affects the performance of Pconf classifiers.
method Parameterized model of skewed confidence and hyperparameter selection.
result Proposed method effectively cancels out the negative impact of skewed confidence.
Model shows how confidence feedback can lead to different crisis outcomes.
problem Characterizing the impact of economic recessions on different social strata.
method A self-reflexive DSGE model with heterogeneous households, varying parameters to analyze crisis typologies.
result Crisis propagation can be confined to high or low income households, depending on social network structure and income inequality.
The paper improves NN predictions with reliable confidence intervals.
problem Creating accurate prediction intervals for neural network outputs.
method Introduces Conformal Prediction (CP) for NNs, ensuring reliable confidence measures.
result The proposed method produces well-calibrated and tight prediction intervals.
Modified AUC improves CNN training by considering model confidence.
problem Improving binary classifier performance metrics.
method Proposes a modified AUC metric that incorporates model confidence into BCE loss for CNN training.
result Demonstrates improved performance on three datasets: MNIST, prostate MRI, and brain MRI.
New CMCL method reduces errors in deep neural network tasks.
problem Overconfidence in ensemble methods for deep neural networks.
method Confident Multiple Choice Learning (CMCL) with new loss, architecture, and training method.
result CMCL reduces top-1 error rates by 14.05% on CIFAR and 6.60% on SVHN.
A new method improves simple models by transferring information from deep networks.
problem Improving simple models with limited resources and interpretability.
method ProfWeight method using confidence scores from deep neural network intermediate layers to train simpler models.
result Significant improvement in test accuracy of simple models (up to 13%) on a real manufacturing problem.
Paper presents methods to create stock price confidence intervals using LSTM models.
problem Creating accurate confidence intervals for LSTM-estimated stock prices.
method Three bootstrap methods for dependent data, optimal block length selection, and benchmark comparison.
result Illustrated through stock price data, different bootstrap strategies provide varying confidence intervals.
New methods improve uncertainty in machine learning predictions for asset returns.
problem Uncertainty in machine learning predictions for asset returns.
method Developed new methods to construct forecast confidence intervals for expected returns from neural networks.
result Neural network forecasts of expected returns have the same asymptotic distribution as classic nonparametric methods, enabling standard error calculation.