Proposes sparsified intervals for high-dimensional regression coefficients.
problem Challenges of high-dimensional regression coefficient inference.
method Sparsified simultaneous confidence intervals.
result Intervals can shrink some coefficients to zero, indicating unimportance.
New method for accurate uncertainty estimation in deep learning predictions.
problem Insufficient methods for assessing prediction uncertainty in deep learning.
method Valid non-parametric bootstrap method for deep neural networks.
result Accurate confidence intervals and simultaneous confidence bands for survival data.
Confidence intervals improve evaluation of binary prediction rules in data mining.
problem Uncertainty in performance measures estimation from finite datasets.
method Asymptotic normal approximations for confidence intervals, with a blurring correction.
result Improved finite sample coverage probabilities and general performance measures inference.
Practical or scientific considerations often lead to selecting a subset of parameters as ``important.'' Inferences about those parameters often are based on the same data used to select them in the first place. That can make the reported uncertainties deceptively optimistic: confidence intervals that ignore selection g…
Data-driven method for error estimation without needing class complexity.
problem Constructing confidence intervals for a class of estimates.
method Data-driven approach to derive high-probability upper bounds on maximum error.
result Method naturally adapts to unknown correlation structures and works for finite and infinite classes.
This article studies local and global inference for smoothing spline estimation in a unified asymptotic framework. We first introduce a new technical tool called functional Bahadur representation, which significantly generalizes the traditional Bahadur representation in parametric models, that is, Bahadur [Ann. Inst. S…
The paper develops methods to accurately locate change points in high-dimensional mean shift models.
problem Locating change points in high-dimensional mean shift models.
method Locally refitted least squares estimator, component-wise and simultaneous rates of estimation.
result Asymptotic validity of component-wise and simultaneous confidence intervals for change point parameters.
This paper proposes methods to compute differentially private confidence intervals for the median.
problem Ensuring privacy in statistical inference for the median.
method Directly estimating interval bounds for the median under differential privacy constraints.
result The proposed methods provide valid differentially private confidence intervals for the median.
The paper introduces a method to assess machine translation quality with confidence intervals.
problem Evaluating the uncertainty and quality of machine translation.
method Utilizes conformal predictive distributions to produce prediction intervals with guaranteed coverage.
result The method outperforms a baseline on six language pairs in terms of coverage and sharpness.
The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.
problem Constructing reliable confidence regions for band-limited functions from noisy data.
method Improved norm bounds using Hoeffding's inequality and empirical Bernstein bound, majority voting to aggregate intervals.
result Confidence intervals retain their simultaneous coverage guarantee even when aggregated from random subsamples.
In this article the package High-dimensional Metrics (\texttt{hdm}) is introduced. It is a collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dim…
A framework for quantifying uncertainty in feature importance values.
problem Stable interpretation of feature importance values in machine learning models.
method A novel method based on pairwise comparisons of feature importance values to produce confidence intervals for feature ranks.
result The method produces simultaneous confidence intervals for feature ranks, enabling selection of top-k important features.
New confidence intervals improve treatment effect estimation in randomized experiments.
problem Improving confidence intervals for treatment effects in randomized experiments.
method Systematic exploitation of negative dependence or variance adaptivity.
result Achieved nonasymptotic confidence intervals with the same effective sample size as asymptotic ones.
The package High-dimensional Metrics (\Rpackage{hdm}) is an evolving collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dimensional subcomponents…
CoinDICE estimates confidence intervals for unknown behavior policies in reinforcement learning.
problem Estimating value of a target policy using only behavior policy data.
method Function space embedding, generalized empirical likelihood method, Lagrangian optimization.
result Valid confidence intervals with tighter and more accurate estimates than existing methods.
CONTINA provides adaptive confidence intervals for traffic demand prediction.
problem Uncertainty in future traffic demand predictions and the need for valid confidence intervals.
method Adaptive confidence interval method that adjusts based on deployment errors.
result Valid confidence intervals with shorter lengths and theoretical coverage guarantee.
Confidence intervals are a popular way to visualize and analyze data distributions. Unlike p-values, they can convey information both about statistical significance as well as effect size. However, very little work exists on applying confidence intervals to multivariate data. In this paper we define confidence interval…
Responds to critiques on tests for causal parameter confidence intervals.
problem Testing nominal confidence interval coverage for causal parameters estimated by machine learning.
method Rejoinder to critiques on nearly assumption-free tests.
result Clarifies and supports the original research's approach.
New method for estimating spatial associations with discrete data, even under model misspecification.
problem Estimating associations between covariates and discrete responses with spatial variability and nonrandom sampling.
method Proposes a novel approach to handle spatially varying noise, provides a proof of consistency, and uses a delta method argument.
result Empirically shows reliable confidence intervals compared to standard methods, even with model misspecification.
Clarifies the confidence interval approach for bioequivalence testing.
problem Ensuring the reliability of bioequivalence testing methods.
method Clarifies the conditions under which a 100(1-2α)% confidence interval yields a size-α test.
result A 100(1-2α)% confidence interval approach for bioequivalence testing yields a size-α test only when the two one-sided tests are 'equal-tailed'.
Paper proposes new method for time series confidence intervals using LSTM.
problem Constructing accurate confidence intervals for multivariate time series.
method Uses Long Short Term Memory Network (LSTM) and novel block bootstrap techniques.
result Demonstrates improved accuracy in constructing confidence intervals.
Random forest training yields confidence intervals for generalization error.
problem Computing accurate confidence intervals for random forest generalization error.
method Directly computes confidence intervals from training data without data splitting.
result Confidence intervals have good coverage and appropriate width.
Research aims to improve confidence intervals for RKHS elements in online learning.
problem Improper confidence intervals lead to suboptimal regret bounds in kernel-based bandit and reinforcement learning.
method Formalizes the open problem of online confidence intervals in RKHS and reviews existing results.
result Identifies the online nature of observation points as the main challenge for tight confidence intervals.
Paper presents methods to create stock price confidence intervals using LSTM models.
problem Creating accurate confidence intervals for LSTM-estimated stock prices.
method Three bootstrap methods for dependent data, optimal block length selection, and benchmark comparison.
result Illustrated through stock price data, different bootstrap strategies provide varying confidence intervals.
Develops confidence intervals for ECE, a measure of model calibration.
problem Ensuring the calibration of probabilistic predictions in machine learning models.
method Develops confidence intervals for the ℓ2 Expected Calibration Error (ECE), considering top-1-to-k calibration. result Shows asymptotic normality and different convergence rates for calibrated and miscalibrated models, developing methods to construct valid confidence intervals.
Develops a simple method for creating private confidence intervals.
problem Creating private confidence intervals for parametric estimation.
method Parametric bootstrap approach to construct confidence intervals.
result The parametric bootstrap provides consistent and effective confidence intervals.
The age of big data has produced data sets that are computationally expensive to analyze and store. Algorithmic leveraging proposes that we sample observations from the original data set to generate a representative data set and then perform analysis on the representative data set. In this paper, we present efficient a…
New statistical methods improve TD learning for policy evaluation.
problem Improving statistical inference for reinforcement learning.
method Polyak-Ruppert averaging, refined high-dimensional Berry-Esseen bounds, online plug-in estimator, asymptotic covariance matrix.
result Guaranteed finite-sample coverage of confidence regions and simultaneous confidence intervals.
We provide the asymptotic distribution of the major indexes used in the statistical literature to quantify disparate treatment in machine learning. We aim at promoting the use of confidence intervals when testing the so-called group disparate impact. We illustrate on some examples the importance of using confidence int…
Paper improves confidence intervals and variance estimation for deep learning models.
problem Improving confidence intervals and variance estimation in deep learning models.
method Residual-based framework for conditional variance estimation; robust bootstrap procedure for confidence intervals.
result First non-asymptotic bounds for variance estimation using ReLU networks.
The paper improves confidence intervals for test error using cross-validation.
problem Improving confidence intervals for test error in machine learning.
method Develops central limit theorems and consistent estimators for cross-validation.
result Provides asymptotically-exact confidence intervals and hypothesis tests.
The paper finds a fundamental trade-off between confidence and efficiency in transductive conformal prediction.
problem The challenge is to balance confidence and efficiency in predicting multiple data points.
method The authors derive a strict finite-sample bound and introduce a practical algorithm to approach this bound.
result Any non-trivial confidence level leads to exponential growth in prediction set size, with a linear scaling in the number of samples.
New method reduces confidence interval sizes for causal inference.
problem Inaccurate propensity scores and extreme scores cause large confidence intervals.
method Data-dependent Coarse IPW (CIPW) estimators.
result Robust CIPW estimators reduce confidence interval sizes to ε+1/√n.
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood techniques to formulate our estimator and confidence interval as simple convex optimization problems. Using the lower bound of our confiden…
Private statistical inference methods improve confidence interval lengths.
problem Constructing private confidence intervals with differential privacy.
method Proposed two private variants of non-parametric bootstrap.
result Achieve similar coverage accuracy to non-private methods with shorter intervals.
We derive formulas for F measures' standard error and confidence intervals.
problem Estimating F measures' accuracy with confidence.
method Analytic formulas based on asymptotic normality.
result Valid formulas for sample size planning.
Due to the increasing availability of high-dimensional empirical applications in many research disciplines, valid simultaneous inference becomes more and more important. For instance, high-dimensional settings might arise in economic studies due to very rich data sets with many potential covariates or in the analysis o…
Efficient method for high confidence level inference using parallel stochastic optimization.
problem Uncertainty quantification for online estimation.
method Small number of independent multi-runs to construct t-based confidence intervals.
result Rigorous theoretical guarantee for exact coverage of confidence intervals.
The paper introduces methods to quantify uncertainty in sampling without replacement.
problem Accurately estimating parameters from finite populations sampled without replacement.
method Develops confidence sequences using Bayesian and empirical methods.
result Improved confidence intervals and sequences for sampling without replacement.
We propose confidence sequences -- sequences of confidence intervals which are valid uniformly over time -- for quantiles of any distribution over a complete, fully-ordered set, based on a stream of i.i.d. observations. We give methods both for tracking a fixed quantile and for tracking all quantiles simultaneously. Sp…
DeepLR constructs confidence intervals for neural networks with asymmetric expansions.
problem Uncertainty estimation for neural network predictions.
method Likelihood-ratio-based approach for constructing asymmetric confidence intervals.
result DeepLR offers asymmetric intervals expanding in regions with limited data.
This paper reviews methods for constructing confidence intervals for error rates in 1:1 matching tasks.
problem Challenges in assessing uncertainty of error rates in matching algorithms, especially when data are dependent and error rates are low.
method Derives and examines statistical properties of methods for constructing confidence intervals for error rates in 1:1 matching tasks.
result Coverage and interval width vary with sample size, error rates, and data dependence.
Study compares imputation methods' effects on IML confidence intervals.
problem Missing data impacts IML interpretation and confidence intervals.
method Compared single vs multiple imputation methods on IML confidence intervals.
result Multiple imputation provides closer coverage to nominal than single imputation.
The paper develops approximations for Pearson's chi-square statistic and applies them to confidence intervals.
problem Finding confidence intervals for strictly convex functions of discrete distribution weights.
method Non-asymptotic local normal approximation for multinomial probabilities, deriving bounds and coupling inequalities.
result Developed methods to find confidence intervals for negative entropy of discrete distributions.
This paper introduces time-uniform CLT-based confidence intervals for statistical inference.
problem Developing valid statistical inference methods for sequential data.
method Time-uniform central limit theory and strong invariance principles.
result Asymptotic confidence sequences (CSs) that are uniformly valid over time.
Optimal learning via moderate deviations theory improves statistical accuracy.
problem Statistical estimation of expected loss in various models.
method Develops confidence intervals using moderate deviation principle.
result Proposed confidence intervals are statistically optimal.
Paper introduces methods for more reliable probabilistic predictions with confidence intervals.
problem Inaccurate labeling of datasets due to unreliable probabilistic predictions from weak labeling functions.
method Proposes a methodology to provide confidence intervals for label probabilities using uncertainty sets of distributions.
result Improves reliability of probabilistic predictions and provides confidence intervals for label probabilities.
Near-optimal confidence intervals for bounded data.
problem Online inference for sequential decision problems like A/B testing.
method Utilizing Bentkus' concentration results to improve on existing methods.
result Near-optimal confidence intervals confirmed favorable in synthetic and practical applications.