CoinDICE estimates confidence intervals for unknown behavior policies in reinforcement learning.
problem Estimating value of a target policy using only behavior policy data.
method Function space embedding, generalized empirical likelihood method, Lagrangian optimization.
result Valid confidence intervals with tighter and more accurate estimates than existing methods.
Novel framework for unbiased confidence estimates in object detection.
problem Unbiased confidence estimates for safety-critical object detection.
method Combines regression output with additional information for calibration.
result Calibrated confidence estimates for image location and scale.
Paper proposes a method to estimate confidence bands for survival random forests.
problem No statistically valid and computationally feasible approach for estimating confidence bands for survival random forests.
method Extending recent developments in infinite-order incomplete U-statistics, the paper proposes an unbiased confidence band estimation.
result The proposed method accurately estimates the confidence band and achieves desired coverage rate.
New method reduces confidence interval sizes for causal inference.
problem Inaccurate propensity scores and extreme scores cause large confidence intervals.
method Data-dependent Coarse IPW (CIPW) estimators.
result Robust CIPW estimators reduce confidence interval sizes to ε+1/√n.
The estimation of probabilities of default (PDs) for low default portfolios by means of upper confidence bounds is a well established procedure in many financial institutions. However, there are often discussions within the institutions or between institutions and supervisors about which confidence level to use for the…
Paper improves confidence intervals and variance estimation for deep learning models.
problem Improving confidence intervals and variance estimation in deep learning models.
method Residual-based framework for conditional variance estimation; robust bootstrap procedure for confidence intervals.
result First non-asymptotic bounds for variance estimation using ReLU networks.
Framework for confidence estimation in deep CT reconstructions.
problem Uncertainty in deep learning-based CT reconstructions.
method Sequential likelihood mixing framework with log-linear forward model.
result Deep models yield tighter confidence regions than classical methods.
Develops methods for estimating and providing confidence bands in sparse high-dimensional additive models.
problem Estimating and providing reliable confidence bands for nonparametric components in high-dimensional additive models.
method Integrates sieve estimation into a high-dimensional Z-estimation framework, employing a multiplier bootstrap procedure.
result Constructs uniformly valid confidence bands for the target component f1 in sparse high-dimensional additive models. Paper proposes a method to improve deep neural networks' confidence estimates.
problem Overconfident predictions limit practical use of deep neural networks in safety-critical applications.
method Proposes a novel loss function, Correctness Ranking Loss, to regularize class probabilities.
result The method produces well-ranked confidence estimates and is effective for out-of-distribution detection and active learning.
We provide a pointwise confidence bound for non-linear least-squares with fixed design.
problem Confidence estimation in non-linear ℓ2-regularized least squares. method Pointwise confidence bound for local minimizers, using weighted norm involving inverse-Hessian.
result The proposed confidence bound scales with the test input's similarity to the training data.
Paper introduces a novel method for estimating model confidence in deep neural classifiers.
problem Reliable confidence estimation for deep neural classifiers in safety-critical applications.
method Proposes a novel target criterion (true class probability) and learns it from data with an auxiliary model.
result The proposed method outperforms strong baselines in various tasks and network architectures.
Responds to critiques on tests for causal parameter confidence intervals.
problem Testing nominal confidence interval coverage for causal parameters estimated by machine learning.
method Rejoinder to critiques on nearly assumption-free tests.
result Clarifies and supports the original research's approach.
Efficient method for high confidence level inference using parallel stochastic optimization.
problem Uncertainty quantification for online estimation.
method Small number of independent multi-runs to construct t-based confidence intervals.
result Rigorous theoretical guarantee for exact coverage of confidence intervals.
Develops a simple method for creating private confidence intervals.
problem Creating private confidence intervals for parametric estimation.
method Parametric bootstrap approach to construct confidence intervals.
result The parametric bootstrap provides consistent and effective confidence intervals.
Calibrates network confidence for unsupervised domain adaptation.
problem Calibrating a model trained on a source domain to a target domain without labeled data.
method Estimates network accuracy on the target domain and calibrates prediction confidence directly in the target domain.
result Significantly outperforms existing methods across standard datasets.
We develop a novel procedure for constructing confidence bands for components of a sparse additive model. Our procedure is based on a new kernel-sieve hybrid estimator that combines two most popular nonparametric estimation methods in the literature, the kernel regression and the spline method, and is of interest in it…
Confidence measures for the generalization error are crucial when small training samples are used to construct classifiers. A common approach is to estimate the generalization error by resampling and then assume the resampled estimator follows a known distribution to form a confidence set [Kohavi 1995, Martin 1996,Yang…
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood techniques to formulate our estimator and confidence interval as simple convex optimization problems. Using the lower bound of our confiden…
Paper tackles weakly supervised learning from similarity-confidence data.
problem Learning binary classifier from unlabeled data pairs with confidence of similarity.
method Proposes an unbiased estimator of classification risk from Sconf data and risk correction scheme.
result Demonstrates effectiveness of proposed methods through experiments.
Waldo method constructs valid confidence regions for simulator-based inference.
problem Constructing valid confidence regions for simulator-based inference with high-dimensional data.
method Reframes Wald test statistic and uses regression-based machinery for Neyman inversion.
result Waldo method produces conditionally valid and precise confidence regions.
Confidence bands for tuning curves improve hyperparameter comparison in NLP.
problem Ambiguity in comparing hyperparameter tuning methods.
method Constructs exact, simultaneous, and distribution-free confidence bands for tuning curves.
result Confidence bands provide a robust basis for comparing methods rigorously.
New method estimates grouping loss in neural networks to improve confidence scores.
problem Improving confidence scores in neural networks to reflect true posterior probabilities.
method Proposed an estimator to approximate the grouping loss.
result Modern neural networks exhibit grouping loss, especially in distribution shifts.
TeLeS improves ASR confidence estimation by considering temporal alignment and lexical errors.
problem Inaccurate confidence scores from E2E ASR models, especially for overconfident predictions.
method Proposes TeLeS, a novel confidence score that considers temporal alignment and lexical errors, and uses shrinkage loss to handle data imbalance.
result TeLeS generalizes well across different languages and ASR models, leading to significant WER reduction.
WS-KDE provides robust confidence bounds for stochastic functions.
problem Optimizing time-consuming black-box functions with stochastic outputs.
method Wilson Score Kernel Density Estimation (WS-KDE) for Bayesian optimization.
result WS-KDE provides reliable confidence bounds for any stochastic function.
Paper shows how online betting algorithms' regret can be used to create tight confidence sequences.
problem Estimating the expectation of random variables from samples and creating time-uniform confidence sequences.
method Converts the regret guarantee of universal portfolio algorithms into time-uniform concentration inequalities and confidence sequences.
result Numerically obtained confidence sequences are never vacuous and satisfy the law of iterated logarithm.
This work uses statistical bootstrapping to provide accurate confidence intervals for policy value in reinforcement learning.
problem Bias in estimating policy value using empirical transitions and rewards.
method Statistical bootstrapping to produce calibrated confidence intervals for the true policy value.
result Statistical bootstrapping can yield correct confidence intervals under certain conditions, and mechanisms are proposed to mitigate these conditions.
Estimates manifold dimension from random samples.
problem Estimating the dimension of a manifold from random samples.
method Explicit theoretical and heuristic bounds for data set size.
result Data set needs to be sufficiently large for accurate dimension estimation.
A universal framework for constructing confidence sets using sequential likelihood mixing.
problem Constructing reliable confidence sets for realizable likelihood functions.
method Sequential likelihood mixing, integrating Bayesian inference and regret inequalities.
result Establishes fundamental connections and provable coverage guarantees for various inference techniques.
The paper improves the empirical bootstrap method for non-normal estimators.
problem Theoretical properties of empirical bootstrap for non-asymptotically normal estimators.
method Establishing limiting distribution, deriving consistency conditions, proposing alternative methods.
result The empirical bootstrap method can be asymptotically consistent under stability conditions.
DRF improves confidence and uncertainty assessment for multivariate conditional distributions.
problem Estimating multivariate conditional distributions with confidence and uncertainty.
method Developed a bootstrap approximation of the asymptotic distribution of DRF to derive inferential tools.
result Asymptotic coverage guarantees for confidence regions and hypothesis testing.
Reduces change detection to estimation using confidence sequences.
problem Detecting changes in data streams with minimal delay and false alarms.
method Reduction from sequential change detection to sequential estimation using confidence sequences.
result Change detection scheme with minimal structural assumptions and strong guarantees.
This paper improves uncertainty quantification in ELM models.
problem Uncertainty in ELM predictions due to data assumptions and randomness.
method Analytical derivations and variance estimates under various conditions.
result Improved understanding and estimation of ELM variability.
The study examines methods to correct measurement error in nutritional epidemiology studies.
problem Measurement error in nutritional studies leads to biased and underconfident estimates.
method The article reviews various bias-correction models for exposure variables in nutritional epidemiology.
result Bias-correction methods are essential for accurate inference in nutritional studies.
Efficiently estimates Cox model coefficients without sharing data.
problem Privacy and ownership concerns in multi-center biomedical studies.
method Communication-efficient iterative distributed algorithms for estimation and inference.
result Achieves convergence rate of full-sample estimator with minimal iterations.
New method constructs confidence sets for GLMs via game theory.
problem Developing reliable confidence intervals for GLM parameters.
method Reduction to sequential prediction games with low regret.
result Online-to-confidence-set conversions provide new types of intervals.
Perry uses auxiliary data to estimate RL policy values with confidence intervals.
problem Leveraging auxiliary datasets for off-policy evaluation with uncertainty quantification.
method Two methods for constructing valid confidence intervals for OPE with data augmentation.
result Methods consistently produce confidence intervals covering ground truth policy values.
Paper proposes a new UCB approach for estimating maximum mean.
problem Estimating the maximum mean in various applications.
method Upper Confidence Bound (UCB) approach with adaptive sampling.
result LSA estimator shows faster bias decay compared to GA.
Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessar…
Improved confidence interval estimation with control variates.
problem Estimating confidence intervals with minimal samples.
method Designing an estimation algorithm using control variates and leveraging order statistics.
result Improved asymptotic efficiency compared to existing algorithms.
Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…
CCE improves anomaly detection metrics by measuring both confidence and consistency.
problem Existing anomaly detection metrics lack discriminative power, hyperparameter dependency, and robustness to perturbations.
method CCE uses Bayesian estimation to quantify uncertainty and constructs global and event-level confidence and consistency scores.
result CCE demonstrates strict boundedness, robustness, and linear time complexity.
This paper presents approximate confidence intervals for each function of parameters in a Banach space based on a bootstrap algorithm. We apply kernel density approach to estimate the persistence landscape. In addition, we evaluate the quality distribution function estimator of random variables using integrated mean sq…
New method for accurate uncertainty estimation in deep learning predictions.
problem Insufficient methods for assessing prediction uncertainty in deep learning.
method Valid non-parametric bootstrap method for deep neural networks.
result Accurate confidence intervals and simultaneous confidence bands for survival data.
Private statistical inference methods improve confidence interval lengths.
problem Constructing private confidence intervals with differential privacy.
method Proposed two private variants of non-parametric bootstrap.
result Achieve similar coverage accuracy to non-private methods with shorter intervals.
We develop time-uniform confidence spheres for estimating means of random vectors.
problem Sequential mean estimation in high-dimensional spaces.
method Derive time-uniform confidence sphere sequences (CSSs) for various types of random vectors.
result Optimal CSSs for log-concave, sub-Gaussian, and sub-ψ random vectors. Confidence intervals and joint confidence sets are constructed for the nonparametric calibration of exponential Lévy models based on prices of European options. To this end, we show joint asymptotic normality in the spectral calibration method for the estimators of the volatility, the drift, the jump intensity and the …
New method for constructing confidence intervals for time series data.
problem Constructing confidence intervals for statistical functionals from time series data.
method Proposes a general purpose confidence interval procedure based on overlapping batches of time series data.
result Large overlapping batches yield confidence intervals of higher quality than generic methods.
DeepLR constructs confidence intervals for neural networks with asymmetric expansions.
problem Uncertainty estimation for neural network predictions.
method Likelihood-ratio-based approach for constructing asymmetric confidence intervals.
result DeepLR offers asymmetric intervals expanding in regions with limited data.