Discriminatively trained neural classifiers can be trusted, only when the input data comes from the training distribution (in-distribution). Therefore, detecting out-of-distribution (OOD) samples is very important to avoid classification errors. In the context of OOD detection for image classification, one of the recen…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “confidence distributions”
Softmax confidence misrepresents uncertainty in neural networks.
problem Neural networks fail to increase uncertainty on out-of-distribution data.
method Investigates two implicit biases in softmax confidence.
result Softmax confidence correlates with epistemic uncertainty due to decision boundary structure and deep network filtering.
Construction of tight confidence regions and intervals is central to statistical inference and decision making. This paper develops new theory showing minimum average volume confidence regions for categorical data. More precisely, consider an empirical distribution generated from iid real…
A new framework tightens risk measure confidence bounds.
problem Improving confidence bounds for various risk measures.
method Distribution optimization framework with two estimation schemes based on concentration bounds.
result Consistently tighter confidence bounds compared to previous methods.
The paper extends confidence sequences for infinite variance data.
problem Addressing confidence sequences for distributions with infinite variance.
method Establishing lower bounds and deriving tight confidence sequences for relaxed bounded -moment distributions.
result Derived confidence sequences are tighter than those using Dubins-Savage inequality.
Distribution-free binary classification: prediction sets, confidence intervals and calibrationstat.ML
Study three types of uncertainty quantification for binary classification without distributional assumptions.
problem Uncertainty quantification for binary classification in a distribution-free setting.
method Established theorems connecting calibration, confidence intervals, and prediction sets for score-based classifiers.
result Distribution-free calibration is only possible using scoring functions that partition feature space into countably many sets.
DRF improves confidence and uncertainty assessment for multivariate conditional distributions.
problem Estimating multivariate conditional distributions with confidence and uncertainty.
method Developed a bootstrap approximation of the asymptotic distribution of DRF to derive inferential tools.
result Asymptotic coverage guarantees for confidence regions and hypothesis testing.
The paper develops approximations for Pearson's chi-square statistic and applies them to confidence intervals.
problem Finding confidence intervals for strictly convex functions of discrete distribution weights.
method Non-asymptotic local normal approximation for multinomial probabilities, deriving bounds and coupling inequalities.
result Developed methods to find confidence intervals for negative entropy of discrete distributions.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
Paper presents robust confidence sequences for means with known moment bounds and arbitrary corruption.
problem Tackles robustness to outliers and adversarial corruptions in mean estimation.
method Designs new robust exponential supermartingales to create confidence sequences.
result Achieves optimal width and shows smaller margin of error compared to fixed-time robust methods.
We shrink confidence sets for equivalent discrete distributions using permutation equivalence.
problem Building high-probability confidence sets for equivalent discrete distributions.
method Exploiting permutation-equivalence to refine confidence sets.
result Confidence sets shrink at asymptotic rates of and .
CCAC calibrates DNN classifiers on OOD datasets by separating mis-classified samples.
problem Calibrating DNN classifiers on out-of-distribution datasets is challenging.
method CCAC introduces an auxiliary class to map DNN output to calibrated confidence, separating mis-classified from correctly classified samples.
result CCAC consistently outperforms prior methods on various DNN models, datasets, and applications.
Bayes-assisted confidence sequences improve efficiency for bounded means.
problem Efficient uncertainty quantification for bounded IID means without parametric assumptions.
method Bayesian working predictive model selects adaptive martingale updates maximizing predictive log-growth.
result Asymptotically log-optimal performance with informative priors reducing width and sampling effort.
This paper improves confidence measurement in deep metric learning models.
problem Measuring confidence in deep metric learning models is challenging.
method Approximates class distributions using Gaussian kernel smoothing and calibrates the confidence metric.
result Improves generalization and robustness of deep metric learning models.
New algorithm for precise changepoint localization without assumptions.
problem Offline changepoint localization in arbitrary distributions.
method Distribution-free algorithm CONformal CHangepoint localization (CONCH) using exchangeability arguments.
result Derives principled score functions for informative and small confidence sets with normalized length shrinking to zero.
Note improves confidence bounds for random variables.
problem Improving confidence bounds for random variables with unbounded ranges and different distributions.
method PAC-Bayes-ification of a derived confidence bound.
result Streamlined proofs for existing results.
New flexible confidence sequences for robust statistical inference.
problem Creating robust statistical inference methods that work under mild assumptions.
method Proposed a new class of asymptotic time-uniform confidence sequences.
result Sharp asymptotic time-uniform confidence sequences achieved under mild assumptions.
The paper develops a method to create non-asymptotic confidence ellipsoids for linear regression without strong noise distribution assumptions.
problem Constructing reliable confidence regions for linear regression with finite sample sizes and general noise distributions.
method The paper introduces the SPS EOA algorithm to create non-asymptotically guaranteed confidence ellipsoids for linear regression problems.
result The sizes of SPS outer ellipsoids are shown to decrease at the optimal rate for linear regression problems.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
Efficiently estimates Cox model coefficients without sharing data.
problem Privacy and ownership concerns in multi-center biomedical studies.
method Communication-efficient iterative distributed algorithms for estimation and inference.
result Achieves convergence rate of full-sample estimator with minimal iterations.
The paper finds a fundamental trade-off between confidence and efficiency in transductive conformal prediction.
problem The challenge is to balance confidence and efficiency in predicting multiple data points.
method The authors derive a strict finite-sample bound and introduce a practical algorithm to approach this bound.
result Any non-trivial confidence level leads to exponential growth in prediction set size, with a linear scaling in the number of samples.
Algorithm finds small confidence sets for arbitrary distributions.
problem Learning high-density regions in arbitrary distributions.
method Competitive with sets from a concept class with bounded VC-dimension.
result Algorithm finds a confidence set with volume competitive with optimal ball.
Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a network must determine whether or not an input is outside of the set on which it is expected to safel…
Mahalanobis distance detects anomalies well, but not for classification.
problem Detecting anomalies in neural classifier outputs.
method Analyzes Mahalanobis distance-based anomaly detection method, revealing its reliance on information not useful for classification.
result Combining Mahalanobis and ODIN methods improves anomaly detection performance and robustness.
We provide the asymptotic distribution of the major indexes used in the statistical literature to quantify disparate treatment in machine learning. We aim at promoting the use of confidence intervals when testing the so-called group disparate impact. We illustrate on some examples the importance of using confidence int…
The estimation of probabilities of default (PDs) for low default portfolios by means of upper confidence bounds is a well established procedure in many financial institutions. However, there are often discussions within the institutions or between institutions and supervisors about which confidence level to use for the…
Confidence intervals are a popular way to visualize and analyze data distributions. Unlike p-values, they can convey information both about statistical significance as well as effect size. However, very little work exists on applying confidence intervals to multivariate data. In this paper we define confidence interval…
A/B testing refers to the task of determining the best option among two alternatives that yield random outcomes. We provide distribution-dependent lower bounds for the performance of A/B testing that improve over the results currently available both in the fixed-confidence (or delta-PAC) and fixed-budget settings. When…
Improved confidence bounds for linear logistic model with applications to bandits.
problem Improving confidence bounds for linear logistic model.
method Self-concordant analysis of the logistic loss to avoid dependence on worst-case variance.
result Significant improvement in confidence bounds, avoiding dependence on .
The paper provides a method to find optimal machine learning model parameters with confidence.
problem Finding optimal machine learning model parameters that generalize well to the entire population.
method Constructs valid confidence sets for the optimal parameter using only training data.
result Valid confidence sets for optimal machine learning model parameters can be generated using bootstrapping techniques.
The paper proposes using entropy to assess model confidence.
problem Discarded information in probability distributions hinders model confidence assessment.
method Entropy methods applied to assess model confidence.
result Theoretical explanation of confidence degradation in Naive Bayes model.
New method estimates grouping loss in neural networks to improve confidence scores.
problem Improving confidence scores in neural networks to reflect true posterior probabilities.
method Proposed an estimator to approximate the grouping loss.
result Modern neural networks exhibit grouping loss, especially in distribution shifts.
Confidence measures for the generalization error are crucial when small training samples are used to construct classifiers. A common approach is to estimate the generalization error by resampling and then assume the resampled estimator follows a known distribution to form a confidence set [Kohavi 1995, Martin 1996,Yang…
ATC predicts target domain accuracy using only labeled and unlabeled data.
problem Predicting out-of-distribution performance with limited labeled data.
method Average Thresholded Confidence (ATC) method that learns a threshold on model confidence.
result ATC outperforms previous methods across various types of distribution shifts and datasets.
New method improves certified robustness for classifier confidence.
problem Certifying confidence in classifier predictions.
method Randomized smoothing with modified Neyman-Pearson lemma.
result Certified radii for prediction confidence improved.
Motivated by the growing popularity of variants of the Wasserstein distance in statistics and machine learning, we study statistical inference for the Sliced Wasserstein distance--an easily computable variant of the Wasserstein distance. Specifically, we construct confidence intervals for the Sliced Wasserstein distanc…
This paper improves offline contextual bandits using distributional robustness.
problem Improving offline contextual bandits with robustness.
method Extends Distributionally Robust Optimization (DRO) for offline contextual bandits, introducing a convex reformulation of Counterfactual Risk Minimization.
result Automatic calibration of asymptotic confidence intervals for policy optimization.
This paper offers a distribution-free method for post-detection changepoint localization.
problem Locating the exact time of a change in distribution after a sequential detection procedure.
method A distribution-free framework using conformal test martingales for sequential change detection and post-detection inference.
result Valid post-detection coverage guarantees and non-asymptotic bounds on confidence set size.
Reduces change detection to estimation using confidence sequences.
problem Detecting changes in data streams with minimal delay and false alarms.
method Reduction from sequential change detection to sequential estimation using confidence sequences.
result Change detection scheme with minimal structural assumptions and strong guarantees.
New method improves model calibration by adjusting confidence based on prediction correctness.
problem Improving model confidence alignment with true class probabilities.
method Post-hoc calibration objective using transformed samples for training.
result Competitive calibration performance on in-distribution and out-of-distribution test sets.
We develop time-uniform confidence spheres for estimating means of random vectors.
problem Sequential mean estimation in high-dimensional spaces.
method Derive time-uniform confidence sphere sequences (CSSs) for various types of random vectors.
result Optimal CSSs for log-concave, sub-Gaussian, and sub- random vectors.
Nonparametric, Nonasymptotic Confidence Bands with Paley-Wiener Kernels for Band-Limited Functionsstat.ML
The paper creates nonparametric confidence bands for band-limited functions.
problem Estimating confidence bands for band-limited functions with finite samples and unknown noise.
method Uses Paley-Wiener reproducing kernel Hilbert spaces and gradient-perturbation methods.
result Non-asymptotic guarantees for confidence regions without assuming a parametric model.
In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model. A natural measure of confidence is (i.e. how confident is about its prediction?). We start by analyzing an adversarial training…
This paper proposes a DGP approach with UCBs for point target tracking over WSNs.
problem Uncertainty quantification in distributed machine learning-based tracking over WSNs.
method Distributed Gaussian process (DGP) approach with upper confidence bounds (UCBs).
result UCBs provide 88% and 42% higher probability of encompassing true target states in X and Y coordinates, respectively.
A new OOD detector using an overlap index improves accuracy without high computational costs.
problem Effective OOD detection for machine learning models in open-world scenarios.
method Proposes an overlap index-based confidence score function for OOD detection.
result The proposed method achieves competitive accuracy with lower computational costs compared to state-of-the-art detectors.
Clarifies the confidence interval approach for bioequivalence testing.
problem Ensuring the reliability of bioequivalence testing methods.
method Clarifies the conditions under which a 100(1-2α)% confidence interval yields a size-α test.
result A 100(1-2α)% confidence interval approach for bioequivalence testing yields a size-α test only when the two one-sided tests are 'equal-tailed'.
The paper improves the empirical bootstrap method for non-normal estimators.
problem Theoretical properties of empirical bootstrap for non-asymptotically normal estimators.
method Establishing limiting distribution, deriving consistency conditions, proposing alternative methods.
result The empirical bootstrap method can be asymptotically consistent under stability conditions.
Improved drug-protein interaction prediction using FTL method.
problem Predicting drug-protein interactions from noisy data with uncertain labels.
method Filtered Transfer Learning (FTL) method that fine-tunes a deep neural network across multiple tiers of data confidence.
result FTL method outperforms deep neural networks trained on single confidence ranges.