Responds to critiques on tests for causal parameter confidence intervals.
problem Testing nominal confidence interval coverage for causal parameters estimated by machine learning.
method Rejoinder to critiques on nearly assumption-free tests.
result Clarifies and supports the original research's approach.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
Method constructs confidence regions for linear models with arbitrary predictors.
problem Constructing confidence regions for linear models with non-linear predictors.
method Mixed Integer Linear Programming for constraints.
result Empty confidence regions for hypothesis testing.
The paper provides a method to find optimal machine learning model parameters with confidence.
problem Finding optimal machine learning model parameters that generalize well to the entire population.
method Constructs valid confidence sets for the optimal parameter using only training data.
result Valid confidence sets for optimal machine learning model parameters can be generated using bootstrapping techniques.
ACORE improves hypothesis testing and confidence sets in likelihood-free inference.
problem Constructing hypothesis tests and confidence sets in likelihood-free inference settings.
method Formulates classical LRT as a classification problem, uses machine learning to improve estimates.
result Demonstrates improved accuracy in hypothesis testing and confidence sets.
This paper studies the geometry of minimum-volume confidence sets for multinomial parameters.
problem Determining if minimum-volume confidence sets for multinomial outcomes are disjoint.
method Enumerating and covering the continuous regions of the exact p-value function to study the geometry of minimum-volume confidence sets.
result The geometry of minimum-volume confidence sets for multinomial parameters is studied, providing insights into their structure and properties.
Introduces CCR for constructing confidence regions from conformal predictions.
problem Challenges in constructing confidence regions for model parameters.
method Combines conformal prediction intervals for model outputs to establish confidence regions for parameters under minimal assumptions.
result Valid coverage guarantees for finite sample regime, applicable to various model types.
The paper develops optimal confidence regions for categorical data.
problem Constructing tight confidence regions for categorical data.
method Develops new theory for minimum average volume confidence regions.
result Shows optimality of the regions for categorical data and its implications for machine learning.
Waldo method constructs valid confidence regions for simulator-based inference.
problem Constructing valid confidence regions for simulator-based inference with high-dimensional data.
method Reframes Wald test statistic and uses regression-based machinery for Neyman inversion.
result Waldo method produces conditionally valid and precise confidence regions.
Practical or scientific considerations often lead to selecting a subset of parameters as ``important.'' Inferences about those parameters often are based on the same data used to select them in the first place. That can make the reported uncertainties deceptively optimistic: confidence intervals that ignore selection g…
New method constructs confidence sets for GLMs via game theory.
problem Developing reliable confidence intervals for GLM parameters.
method Reduction to sequential prediction games with low regret.
result Online-to-confidence-set conversions provide new types of intervals.
A new method uses SVM classification to efficiently compute confidence sets.
problem Computing confidence sets for moment inequalities is computationally intensive.
method Converts confidence set construction into a classification problem using SVM.
result Asymptotically reproduces the test in the confidence set using SVM classification.
Typical dimensionality reduction (DR) methods are often data-oriented, focusing on directly reducing the number of random variables (features) while retaining the maximal variations in the high-dimensional data. In unsupervised situations, one of the main limitations of these methods lies in their dependency on the sca…
We consider the setting of linear regression in high dimension. We focus on the problem of constructing adaptive and honest confidence sets for the sparse parameter θ, i.e. we want to construct a confidence set for theta that contains theta with high probability, and that is as small as possible. The l_2 diameter of a …
The unified approach of Feldman and Cousins allows for exact statistical inference of small signals that commonly arise in high energy physics. It has gained widespread use, for instance, in measurements of neutrino oscillation parameters in long-baseline experiments. However, the approach relies on the Neyman construc…
Typical dimensionality reduction methods focus on directly reducing the number of random variables while retaining maximal variations in the data. In this paper, we consider the dimensionality reduction in parameter spaces of binary multivariate distributions. We propose a general Confident-Information-First (CIF) prin…
This paper extends the existing literature on empirical estimation of the confidence intervals associated to the Detrended Fluctuation Analysis (DFA). We used Montecarlo simulation to evaluate the confidence intervals. Varying the parameters in DFA technique, we point out the relationship between those and the standard…
The SPS method constructs confidence regions for true parameters with optimal sample complexity.
problem Constructing exact, non-asymptotic confidence regions for true system parameters.
method Sign-Perturbed Sums (SPS) method, generalized to various types of problems.
result High probability upper bounds for SPS confidence regions show optimal shrinkage rate.
The study examines methods to correct measurement error in nutritional epidemiology studies.
problem Measurement error in nutritional studies leads to biased and underconfident estimates.
method The article reviews various bias-correction models for exposure variables in nutritional epidemiology.
result Bias-correction methods are essential for accurate inference in nutritional studies.
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
Novel confidence sets improve linear bandit performance by adapting to unknown noise levels.
problem Adapting to unknown noise levels in sequential decision-making.
method Proposed semi-adaptive and variance-adaptive confidence sets.
result Improved regret bounds and better performance in Bayesian optimization tasks.
EB-TCε identifies the best arm with ε confidence in stochastic bandits.
problem Identifying the best arm in stochastic bandits with a fixed level of confidence.
method EB-TCε is a novel sampling rule for ε-best arm identification in stochastic bandits.
result EB-TCε is the first anytime algorithm for fixed confidence or fixed budget identification.
A method to approximate instance-dependent label noise using instance-confidence embedding.
problem Real-world label noise that depends on individual instances.
method Variational approximation with instance embedding to capture instance-specific label corruption.
result ICE method effectively approximates instance-dependent noise and detects ambiguous instances.
A new framework bridges classical and machine learning methods for reliable inference from complex models.
problem Intractable likelihood functions in complex systems make classical statistics ineffective for likelihood-free inference.
method Likelihood-Free Frequentist Inference (LF2I) framework that combines classical statistics and machine learning.
result Valid confidence sets with near finite-sample validity can be constructed for any parameter value.
In this paper, we study a simple algorithm to construct asymptotically valid confidence regions for model parameters using the batch means method. The main idea is to cancel out the covariance matrix which is hard/costly to estimate. In the process of developing the algorithm, we establish process-level functional cent…
Hypothesis testing in the linear regression model is a fundamental statistical problem. We consider linear regression in the high-dimensional regime where the number of parameters exceeds the number of samples (p>n). In order to make informative inference, we assume that the model is approximately sparse, that is th…
In this article the package High-dimensional Metrics (\texttt{hdm}) is introduced. It is a collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dim…
The paper shows over-confidence in models isn't just due to over-parametrization.
problem Over-confidence in machine learning models, especially in binary classification.
method Theoretical analysis of logistic regression and other binary classification problems.
result Logistic regression is inherently over-confident in certain settings, but over-confidence is not always the case.
Paper certifies intersection of minimum-volume confidence sets for multinomial outcomes.
problem Certifying intersection of minimum-volume confidence sets for multinomial outcomes.
method Exploits likelihood ordering to induce halfspace constraints, enabling adaptive geometric partitioning and computable bounds on p-values.
result Efficient and provably sound algorithm for certifying intersection, disjointness, or indeterminate result.
Efficient estimators for smooth Hilbert-valued parameters with theoretical guarantees.
problem Estimating smooth Hilbert-valued parameters with theoretical guarantees.
method Pathwise differentiable Hilbert-valued parameters, efficient influence functions, regularized one-step estimators.
result Theoretical guarantees for efficient estimators even when nuisance functions are arbitrary.
Private statistical inference methods improve confidence interval lengths.
problem Constructing private confidence intervals with differential privacy.
method Proposed two private variants of non-parametric bootstrap.
result Achieve similar coverage accuracy to non-private methods with shorter intervals.
New methods improve confidence set calibration in complex models.
problem Challenges in maintaining confidence set coverage in complex models.
method TRUST and TRUST++ methods using simulated data for calibration.
result Methods achieve distribution-free conditional coverage and robust inference.
CCAC calibrates DNN classifiers on OOD datasets by separating mis-classified samples.
problem Calibrating DNN classifiers on out-of-distribution datasets is challenging.
method CCAC introduces an auxiliary class to map DNN output to calibrated confidence, separating mis-classified from correctly classified samples.
result CCAC consistently outperforms prior methods on various DNN models, datasets, and applications.
The process of data mining with differential privacy produces results that are affected by two types of noise: sampling noise due to data collection and privacy noise that is designed to prevent the reconstruction of sensitive information. In this paper, we consider the problem of designing confidence intervals for the…
The paper introduces methods to quantify uncertainty in sampling without replacement.
problem Accurately estimating parameters from finite populations sampled without replacement.
method Develops confidence sequences using Bayesian and empirical methods.
result Improved confidence intervals and sequences for sampling without replacement.
Can we learn a binary classifier from only positive data, without any negative data or unlabeled data? We show that if one can equip positive data with confidence (positive-confidence), one can successfully learn a binary classifier, which we name positive-confidence (Pconf) classification. Our work is related to one-c…
Improved confidence bounds for linear logistic model with applications to bandits.
problem Improving confidence bounds for linear logistic model.
method Self-concordant analysis of the logistic loss to avoid dependence on worst-case variance.
result Significant improvement in confidence bounds, avoiding dependence on 1/κ. The package High-dimensional Metrics (\Rpackage{hdm}) is an evolving collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dimensional subcomponents…
This paper presents approximate confidence intervals for each function of parameters in a Banach space based on a bootstrap algorithm. We apply kernel density approach to estimate the persistence landscape. In addition, we evaluate the quality distribution function estimator of random variables using integrated mean sq…
FreB protocol uses AI to infer hidden parameters with valid confidence regions.
problem Generating biased or overconfident conclusions from AI-generated posterior distributions.
method Frequentist-Bayes (FreB) protocol reshapes AI-generated posterior distributions into valid confidence regions.
result FreB provides valid confidence regions that consistently include true parameters with expected probability.
In this article we consider the parameter risk in the context of internal modelling of the reserve risk under Solvency II. We discuss two opposed perspectives on parameter uncertainty and point out that standard methods of classical reserving focusing on the estimation error of claims reserves are in general not approp…
Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase the trust, interpretability and usefulness of virtual screening models in drug discovery, techniques t…
StratPPI improves prediction-powered inference with stratified sampling.
problem Improving statistical estimates with limited human-labeled data.
method Combining small human-labeled data with large automatic-labeled data, stratifying data for tighter confidence intervals.
result StratPPI provides substantially tighter confidence intervals than unstratified approaches.
In engineering applications almost all processes are described with the help of models. Especially forming machines heavily rely on mathematical models for control and condition monitoring. Inaccuracies during the modeling, manufacturing and assembly of these machines induce model uncertainty which impairs the controll…
FP-UCB algorithm achieves bounded regret for finitely parameterized multi-armed bandits.
problem Finitely parameterized multi-armed bandits with unknown but known parameter set.
method FP-UCB algorithm using structural information about the parameter set.
result FP-UCB achieves bounded regret under structural condition, logarithmic otherwise.
Reduces change detection to estimation using confidence sequences.
problem Detecting changes in data streams with minimal delay and false alarms.
method Reduction from sequential change detection to sequential estimation using confidence sequences.
result Change detection scheme with minimal structural assumptions and strong guarantees.
Deep neural networks (DNNs) are poorly calibrated when trained in conventional ways. To improve confidence calibration of DNNs, we propose a novel training method, distance-based learning from errors (DBLE). DBLE bases its confidence estimation on distances in the representation space. In DBLE, we first adapt prototypi…
Paper improves confidence intervals for LSA with multiplier bootstrap.
problem Improving confidence intervals for parameter estimation in LSA.
method Berry-Esseen bound for multivariate normal approximation and multiplier bootstrap.
result Valid confidence intervals for parameter estimation in LSA.