arXiv research

New methods improve confidence set calibration in complex models.

New method constructs confidence sets for GLMs via game theory.

We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to cons…

Improved regret bounds for logistic bandits via novel confidence set construction.

Confidence intervals and joint confidence sets are constructed for the nonparametric calibration of exponential Lévy models based on prices of European options. To this end, we show joint asymptotic normality in the spectral calibration method for the estimators of the volatility, the drift, the jump intensity and the …

The paper provides a method to find optimal machine learning model parameters with confidence.

This paper studies the geometry of minimum-volume confidence sets for multinomial parameters.

The paper addresses the gap between theoretical and practical confidence set widths in universal inference.

A new method for predicting with confidence for complex models.

Confidence-based deferral works well in many scenarios but fails in specific cases.

A universal framework for constructing confidence sets using sequential likelihood mixing.

Paper presents methods to create stock price confidence intervals using LSTM models.

Introduces CCR for constructing confidence regions from conformal predictions.

Develops spatial uncertainty guarantees for image segmentation models.

We consider the setting of linear regression in high dimension. We focus on the problem of constructing adaptive and honest confidence sets for the sparse parameter θ, i.e. we want to construct a confidence set for theta that contains theta with high probability, and that is as small as possible. The l_2 diameter of a …

Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase the trust, interpretability and usefulness of virtual screening models in drug discovery, techniques t…

Develops methods for valid and validated confidence sets in multiclass and multilabel prediction.

This paper introduces a new method for uncertainty quantification in prediction models.

New algorithm improves best arm identification in Bayesian settings.

Confidence measures for the generalization error are crucial when small training samples are used to construct classifiers. A common approach is to estimate the generalization error by resampling and then assume the resampled estimator follows a known distribution to form a confidence set [Kohavi 1995, Martin 1996,Yang…

Spatially weighted conformal prediction improves uncertainty quantification in house price models.

Method constructs confidence regions for linear models with arbitrary predictors.

New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.

Parameter estimation, statistical tests and confidence sets are the cornerstones of classical statistics that allow scientists to make inferences about the underlying process that generated the observed data. A key question is whether one can still construct hypothesis tests and confidence sets with proper coverage and…

Develops statistical confidence sets for multidimensional scaling.

The paper shows over-confidence in models isn't just due to over-parametrization.

Adversarial training yields robust models against a specific threat model, e.g., $L_\infty$ adversarial examples. Typically robustness does not generalize to previously unseen threat models, e.g., other $L_p$ norms, or larger perturbations. Our confidence-calibrated adversarial training (CCAT) tackles this problem by b…

New method uses weak labels to create valid confidence sets for predictions.

New method optimizes offline linear bandits using different confidence sets.

Novel confidence sets improve linear bandit performance by adapting to unknown noise levels.

The age of big data has produced data sets that are computationally expensive to analyze and store. Algorithmic leveraging proposes that we sample observations from the original data set to generate a representative data set and then perform analysis on the representative data set. In this paper, we present efficient a…

Proposes a method to compute valid lower confidence bounds for multiple models selected based on their performance.

New method HNCI for evaluating treatment effects in network interference.

General predictive models do not provide a measure of confidence in predictions without Bayesian assumptions. A way to circumvent potential restrictions is to use conformal methods for constructing non-parametric confidence regions, that offer guarantees regarding validity. In this paper we provide a detailed descripti…

Paper presents robust confidence sequences for means with known moment bounds and arbitrary corruption.

A new method uses SVM classification to efficiently compute confidence sets.

Paper improves confidence intervals and variance estimation for deep learning models.

We shrink confidence sets for equivalent discrete distributions using permutation equivalence.

Develops methods for estimating and providing confidence bands in sparse high-dimensional additive models.

New method for linear bandits with unknown sparsity, improving sparse regret bounds.

This paper improves deep learning classifiers by integrating conformal prediction during training.

The paper proposes a method to construct confidence sets using likelihood ratios for sequential decision-making.

New attacks can infer model training membership using only label predictions, not confidence.

Robust MDPs (RMDPs) can be used to compute policies with provable worst-case guarantees in reinforcement learning. The quality and robustness of an RMDP solution are determined by the ambiguity set---the set of plausible transition probabilities---which is usually constructed as a multi-dimensional confidence region. E…

This paper presents the R package MCS which implements the Model Confidence Set (MCS) procedure recently developed by Hansen et al. (2011). The Hansen's procedure consists on a sequence of tests which permits to construct a set of 'superior' models, where the null hypothesis of Equal Predictive Ability (EPA) is not rej…

FROB model improves robustness and reliable confidence for few-shot OoD detection.

New method estimates grouping loss in neural networks to improve confidence scores.

New method identifies diffusion sources on networks with statistical confidence.