DeepLR constructs confidence intervals for neural networks with asymmetric expansions.
problem Uncertainty estimation for neural network predictions.
method Likelihood-ratio-based approach for constructing asymmetric confidence intervals.
result DeepLR offers asymmetric intervals expanding in regions with limited data.
New methods improve uncertainty in machine learning predictions for asset returns.
problem Uncertainty in machine learning predictions for asset returns.
method Developed new methods to construct forecast confidence intervals for expected returns from neural networks.
result Neural network forecasts of expected returns have the same asymptotic distribution as classic nonparametric methods, enabling standard error calculation.
RFpredInterval package builds prediction intervals for random forests and boosted forests.
problem Quantifying uncertainty in random forest and boosted forest point predictions.
method 16 methods to build prediction intervals with random forests and boosted forests.
result The proposed method outperforms existing methods in building prediction intervals.
Proposes methods to aggregate prediction intervals for domain shift uncertainty.
problem Uncertainty quantification in distribution shifts.
method Aggregates prediction intervals for minimal width and adequate coverage.
result Effective methodologies for unsupervised domain shift under labeled source and unlabeled target.
Enhances polynomial chaos models with uncertainty intervals.
problem Uncertainty quantification in surrogate models.
method Jackknife-based conformal prediction integrated into polynomial chaos expansions.
result Produces accurate predictive intervals for low-accuracy models.
New metrics quantify implementation risk in portfolio backtesting, revealing systematic differences in engine implementations.
problem Systematic divergence in backtested portfolio metrics due to differences in engine implementations.
method Formalized implementation risk, proposed four metrics, executed 15 strategies through five engines, analyzed source-code defects.
result Implementation risk introduces measurable ambiguity in performance attribution, but does not alter investment decisions.
CREDO combines credal and conformal methods to create interpretable prediction intervals.
problem Overconfident prediction intervals in regions of model extrapolation.
method CREDO uses a credal envelope to widen intervals in weak evidence regions and then applies conformal calibration.
result CREDO prediction intervals are interpretable and maintain target coverage.
UTOPIA aggregates multiple prediction intervals efficiently.
problem Constructing optimal prediction intervals for various real-world data problems.
method UTOPIA is a universally trainable strategy using linear or convex programming.
result UTOPIA constructs prediction intervals with small average width and high coverage probability.
Paper proposes a method to create more reliable confidence intervals for off-policy evaluations.
problem Creating reliable confidence intervals for off-policy evaluations.
method Proposes a deeply-debiasing procedure to construct efficient, robust, and flexible confidence intervals.
result Validated by theoretical results and numerical experiments, the method improves the reliability of off-policy evaluations.
Efficient method for high confidence level inference using parallel stochastic optimization.
problem Uncertainty quantification for online estimation.
method Small number of independent multi-runs to construct t-based confidence intervals.
result Rigorous theoretical guarantee for exact coverage of confidence intervals.
This paper tackles uncertainty in deep learning for construction of prediction intervals.
problem Deep learning models lack the ability to provide reliable prediction intervals for high-risk tasks.
method The authors design a special loss function to learn both aleatory and epistemic uncertainties without requiring uncertainty labels.
result The method constructs prediction intervals that are competitive with state-of-the-art methods on publicly available datasets.
Discriminative jackknife estimates deep learning uncertainty.
problem Quantifying uncertainty in deep learning models.
method Discriminative jackknife using influence functions of loss.
result DJ satisfies frequentist coverage and discriminative accuracy.
INNs produce interval-valued uncertainty scores for DNNs.
problem Uncertainty quantification in deep neural networks.
method Data-driven interval propagating network using interval arithmetic.
result INNs produce sensible lower and upper bounds for prediction error.
In this article the package High-dimensional Metrics (\texttt{hdm}) is introduced. It is a collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dim…
UnKGCP generates prediction intervals for uncertain knowledge graphs with statistical guarantees.
problem Lack of quantified predictive uncertainty in existing UnKGE methods.
method Proposes extsc{UnKGCP} framework using conformal prediction with a novel nonconformity measure.
result Sharp prediction intervals effectively capture predictive uncertainty in diverse UnKGE methods.
New method assesses prediction intervals across different operating points.
problem Difficulty in comparing prediction intervals across studies.
method Operating characteristics curves and gain over a simple reference.
result A novel operating point agnostic assessment methodology for prediction intervals.
This work provides uncertainty intervals for semantic latent variables in disentangled latent spaces.
problem Challenges in providing meaningful uncertainty quantification for semantic information in disentangled latent spaces.
method Uses quantile regression to output heuristic uncertainty intervals, calibrates these intervals to contain true latent values, and propagates them through the generator.
result Reliably communicates semantically meaningful, principled, and instance-adaptive uncertainty in image super-resolution and image completion.
A new method combines synthetic data analysis and DP generation to produce accurate uncertainty estimates.
problem Invalid inferences from DP synthetic data analysis.
method Combining synthetic data analysis techniques from MI and NA Bayesian modeling with a novel noise-aware synthetic data generation algorithm.
result Accurate confidence intervals from DP synthetic data are produced, wider with tighter privacy.
New method creates adaptive prediction intervals for regression models.
problem Need to quantify uncertainty in regression model predictions.
method Regression trees and Random Forests trained on conformity scores.
result Superior scalability and performance compared to baselines.
The paper monitors model deterioration using uncertainty estimation.
problem Challenges in monitoring deployed machine learning models.
method Non-parametric bootstrapped uncertainty estimates and SHAP values.
result The approach detects model deterioration and uncertainty effectively.
MAPIE provides uncertainty quantification for ML models.
problem Estimating uncertainties in ML model predictions.
method Conformal prediction methods for single-output regression and multi-class classification.
result Strong theoretical guarantees on marginal coverages.
A new method for estimating uncertainty intervals in regression.
problem Lack of effective methods to estimate uncertainty intervals in regression.
method Collaborating Networks (CN) approach using two neural networks with distinct loss functions.
result CN method improves performance on various real-world datasets, including forecasting A1c values in diabetic patients.
A framework for uncertainty-aware multimodal learning using conformal Shapley intervals.
problem Uncertainty and modality level importance in multimodal learning.
method Introduces conformal Shapley intervals to quantify modality level importance and uncertainty.
result Demonstrates meaningful uncertainty quantification and strong predictive performance.
A cubing strategy identifies stable hyperparameter regions for uncertainty quantification in spatial deep learning.
problem Uncertainty quantification in spatial deep learning models.
method Cubing-based diagnostic framework to recursively partition hyperparameter space and evaluate regions using scoring rules.
result Our approach produces competitive or superior predictive intervals compared to a statistical baseline model.
Framework improves PV forecasting by accounting for missing data uncertainty.
problem Uncertainty from missing data in PV power data.
method Combines stochastic multiple imputation with Rubin's rule.
result Improves prediction interval calibration without sacrificing point prediction accuracy.
Hierarchical framework for model evaluation on leaderboards
problem Uncertainty and variability in model performance across tasks
method Hierarchical framework with task-level and leaderboard-level rank prediction intervals
result Statistically valid and informative model rank intervals
New method for accurate uncertainty estimation in deep learning predictions.
problem Insufficient methods for assessing prediction uncertainty in deep learning.
method Valid non-parametric bootstrap method for deep neural networks.
result Accurate confidence intervals and simultaneous confidence bands for survival data.
CLAPS improves conformal regression by adaptively scaling interval widths based on last-layer Laplace uncertainty.
problem Lack of adaptive interval width scaling in conformal regression for heterogeneous inputs.
method CLAPS uses heteroscedastic last-layer Laplace uncertainty to adaptively scale interval widths, combining aleatoric and epistemic uncertainties.
result CLAPS provides competitive interval efficiency with nominal-level coverage, reducing to aleatoric scaling as epistemic uncertainty decreases.
New method uses interval-based metric to validate prediction uncertainty in machine learning.
problem Validation of prediction uncertainty in machine learning regression tasks is unreliable due to heavy-tailed distributions.
method Shift from variance-based metrics to interval-based Prediction Interval Coverage Probability (PICP).
result PICP method more quickly and reliably tests prediction intervals than variance-based metrics.
Unified framework for estimating random forest prediction errors.
problem Estimating prediction errors for random forests.
method Novel estimator of conditional prediction error distribution function.
result Proposed estimators enable competitive prediction intervals.
The package High-dimensional Metrics (\Rpackage{hdm}) is an evolving collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dimensional subcomponents…
Two-step conformal prediction method for adaptive bounding box uncertainties in multi-object detection.
problem Quantifying predictive uncertainty for multi-object detection in safety-critical applications.
method Developed a two-step conformal prediction approach to propagate uncertainty in predicted class labels into bounding box uncertainties, ensuring coverage for incorrectly classified objects.
result Desired coverage levels are satisfied with practically tight predictive uncertainty intervals on real-world datasets.
Interval bankruptcy problems arise in situations where an estate has to be liquidated among a fixed number of creditors and uncertainty about the amounts of the claims is modeled by intervals. We extend in the interval setting the classical results by Curiel, Maschler and Tijs (1987) that characterize division rules wh…
SEMF predicts prediction intervals for ML models using latent variables.
problem Uncertainty quantification in ML models, especially for diverse data distributions.
method Supervised Expectation-Maximization Framework (SEMF) extending EM algorithm for latent variable modeling.
result SEMF produces narrower prediction intervals with desired coverage probability.
ConfEviSurrogate improves surrogate model accuracy and uncertainty quantification.
problem Uncertainty in surrogate models hinders reliable analysis.
method Introduces ConfEviSurrogate, a novel model that learns evidential distributions, separates uncertainty sources, and provides reliable prediction intervals.
result Demonstrates accurate predictions and robust uncertainty estimates in various simulations.
New methods needed to evaluate uncertainty estimates in neural networks.
problem Evaluating uncertainty estimates in neural networks is flawed and inconsistent.
method Proposes a simulation-based testing approach to address flaws in current methods.
result Current methods for evaluating uncertainty estimates have significant flaws and cannot accurately compare different methods.
With rapid adoption of deep learning in critical applications, the question of when and how much to trust these models often arises, which drives the need to quantify the inherent uncertainties. While identifying all sources that account for the stochasticity of models is challenging, it is common to augment prediction…
Enhances XGBoost for better uncertainty quantification in ML predictions.
problem Uncertainty in ML predictions, especially for XGBoost.
method Quantile Extreme Gradient Boosting (QXGBoost) using Huber norm in quantile regression.
result QXGBoost produces more accurate 90% prediction intervals.
Paper develops a method to estimate value of a policy in confounded MDPs.
problem Estimating value of a policy in the presence of unmeasured confounders.
method Uses auxiliary variables to identify target policy's value in a confounded MDP.
result Develops an off-policy value estimator robust to model misspecification.
UACQR improves CQR by separating aleatoric and epistemic uncertainties.
problem Ineffective CQR for problems with varying quantile regressor performance.
method Integrates aleatoric and epistemic uncertainties in CQR.
result UACQR provides stronger conditional coverage in simulated and real-world data.
Develops a framework to control risk in online learning models.
problem Rigorous uncertainty quantification for online learning models.
method A framework for constructing uncertainty sets that provably control risk.
result Guarantees risk control at any user-specified level even with distribution shifts.
TDistNNs improve prediction intervals for neural networks by using t-distributions.
problem Traditional neural networks provide only point estimates, lacking predictive uncertainty.
method TDistNNs generate t-distributed outputs with adjustable degrees of freedom, enhancing robustness to non-Gaussian data.
result TDistNNs produce narrower prediction intervals with proper coverage compared to Gaussian-based PNNs.
Ongoing developments in neural network models are continually advancing the state of the art in terms of system accuracy. However, the predicted labels should not be regarded as the only core output; also important is a well-calibrated estimate of the prediction uncertainty. Such estimates and their calibration are cri…
CLEAR calibrates both aleatoric and epistemic uncertainties for better predictive intervals.
problem Balanced uncertainty quantification for reliable predictive modeling.
method CLEAR uses two parameters, γ1 and γ2, to combine aleatoric and epistemic uncertainties.
result Clear achieves significant improvements in interval width and coverage.
Proposes a new network for accurate predictions and uncertainty estimation.
problem Uncertainty estimation in regression predictions without sacrificing accuracy.
method Decoupled two-stage training process with custom loss function.
result Reduces prediction error by 23-34% while maintaining 95% PICP.
Proposes a method to estimate drug sensitivity uncertainty using deep regression forests.
problem Lack of confidence intervals in deep learning models for critical tasks.
method Uses Deep Regression Forests to estimate variance and uncertainty for drug sensitivity prediction.
result Improves efficiency and coverage of uncertainty estimates for drug sensitivity predictions.
The paper presents a method for generating well-calibrated prediction intervals using quality-driven deep ensembles.
problem Generating reliable prediction intervals for regression analysis.
method A multi-objective loss function combining quality measures for prediction intervals and point estimates, with a penalty function to ensure semantic integrity and stability.
result The method produces well-calibrated prediction intervals and point estimates, capturing both aleatoric and epistemic uncertainty.
This paper proposes methods to compute differentially private confidence intervals for the median.
problem Ensuring privacy in statistical inference for the median.
method Directly estimating interval bounds for the median under differential privacy constraints.
result The proposed methods provide valid differentially private confidence intervals for the median.