arXiv research

CCAC calibrates DNN classifiers on OOD datasets by separating mis-classified samples.

The contour map of estimation error of Expected Shortfall (ES) is constructed. It allows one to quantitatively determine the sample size (the length of the time series) required by the optimization under ES of large institutional portfolios for a given size of the portfolio, at a given confidence level and a given esti…

The contour maps of the error of historical resp. parametric estimates for large random portfolios optimized under the risk measure Expected Shortfall (ES) are constructed. Similar maps for the sensitivity of the portfolio weights to small changes in the returns as well as the VaR of the ES-optimized portfolio are also…

A method to approximate instance-dependent label noise using instance-confidence embedding.

Approximating non-linear kernels using feature maps has gained a lot of interest in recent years due to applications in reducing training and testing times of SVM classifiers and other kernel based learning algorithms. We extend this line of work and present low distortion embeddings for dot product kernels into linear…

No policy can simultaneously be fully autonomous, optimally calibrated, and helpful, proving a trilemma.

Deep learning models are now used in many different industries, while in certain domains safety is not a critical issue in the medical field it is a huge concern. Not only, we want the models to generalize well but we also want to know the models confidence respect to its decision and which features matter the most. Ou…

Forest tree species mapped with high accuracy using satellite data.

TASFAR adapts regression models without labeled source data.

New method uses path signatures for causal discovery in time series data.

Structural variants compose the majority of human genetic variation, but are difficult to assess using current genomic sequencing technologies. Optical mapping technologies, which measure the size of chromosomal fragments between labeled markers, offer an alternative approach. As these technologies mature towards becom…

The recent empirical success of unsupervised cross-domain mapping algorithms, between two domains that share common characteristics, is not well-supported by theoretical justifications. This lacuna is especially troubling, given the clear ambiguity in such mappings. We work with adversarial training methods based on IP…

New flexible confidence sequences for robust statistical inference.

New tighter confidence bounds for sequential kernel regression.

The paper extends confidence sequences for infinite variance data.

Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single corrective multiplicative factor for inputs to the last softmax layer. On non-neural models the exist…

Improved algorithms for stochastic linear bandits using tighter confidence sequences.

Confidence intervals are a popular way to visualize and analyze data distributions. Unlike p-values, they can convey information both about statistical significance as well as effect size. However, very little work exists on applying confidence intervals to multivariate data. In this paper we define confidence interval…

Paper presents robust confidence sequences for means with known moment bounds and arbitrary corruption.

This paper studies the geometry of minimum-volume confidence sets for multinomial parameters.

CoinDICE estimates confidence intervals for unknown behavior policies in reinforcement learning.

The paper investigates how dataset quality and heterogeneity affect model confidence in machine learning.

A new concept of confidence in learning is defined and analyzed.

The paper shows over-confidence in models isn't just due to over-parametrization.

Weak lensing maps contain information beyond two-point statistics on small scales. Much recent work has tried to extract this information through a range of different observables or via nonlinear transformations of the lensing field. Here we train and apply a 2D convolutional neural network to simulated noiseless lensi…

New confidence intervals improve treatment effect estimation in randomized experiments.

Develops a method to find costly high-confidence errors in black box models.

Positive-confidence (Pconf) classification [Ishida et al., 2018] is a promising weakly-supervised learning method which trains a binary classifier only from positive data equipped with confidence. However, in practice, the confidence may be skewed by bias arising in an annotation process. The Pconf classifier cannot be…

The paper introduces neural INGARCH models for time series of counts.

Develops confidence bounds for off-policy evaluation in contextual bandits.

Generative models need per-sample confidence scores to improve quality and stability.

Confidence-based filtering reveals latent structure in diffusion models.

We consider the setting of linear regression in high dimension. We focus on the problem of constructing adaptive and honest confidence sets for the sparse parameter θ, i.e. we want to construct a confidence set for theta that contains theta with high probability, and that is as small as possible. The l_2 diameter of a …

Softmax confidence misrepresents uncertainty in neural networks.

This paper rethinks confidence calibration under covariate shifts.

Method constructs confidence regions for linear models with arbitrary predictors.

Novel framework for unbiased confidence estimates in object detection.

The paper addresses the difficulty of decision makers trusting AI-assisted predictions and proposes a method to improve confidence values.

New method constructs confidence sets for GLMs via game theory.

Construction of tight confidence regions and intervals is central to statistical inference and decision making. This paper develops new theory showing minimum average volume confidence regions for categorical data. More precisely, consider an empirical distribution $\widehat{\boldsymbol{p}}$ generated from $n$ iid real…

This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.

The paper proposes using entropy to assess model confidence.

Annealing Double-Head calibrates deep neural networks during training.

Paper shows how online betting algorithms' regret can be used to create tight confidence sequences.

In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model. A natural measure of confidence is $\|F({\bf x})\|_\infty$ (i.e. how confident $F$ is about its prediction?). We start by analyzing an adversarial training…

A new differentiable UCB algorithm for linear bandits learns adaptive confidence bounds.

Confidence bands for tuning curves improve hyperparameter comparison in NLP.

Responds to critiques on tests for causal parameter confidence intervals.