A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Calibrates network confidence for unsupervised domain adaptation.
We study the tradeoff between computational effort and classification accuracy in a cascade of deep neural networks. During inference, the user sets the acceptable accuracy degradation which then automatically determines confidence thresholds for the intermediate classifiers. As soon as the confidence threshold is met,…
We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by …
New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.
The paper shows over-confidence in models isn't just due to over-parametrization.
ATC predicts target domain accuracy using only labeled and unlabeled data.
Optimal learning via moderate deviations theory improves statistical accuracy.
Learn low-degree functions with few random queries.
Specialists outperform generalists in ensemble classification.
This paper reviews metrics to assess AI model calibration accuracy.
This paper extends the existing literature on empirical estimation of the confidence intervals associated to the Detrended Fluctuation Analysis (DFA). We used Montecarlo simulation to evaluate the confidence intervals. Varying the parameters in DFA technique, we point out the relationship between those and the standard…
In this paper, we propose a new method called ProfWeight for transferring information from a pre-trained deep neural network that has a high test accuracy to a simpler interpretable model or a very shallow network of low complexity and a priori low test accuracy. We are motivated by applications in interpretability and…
A new OOD detector using an overlap index improves accuracy without high computational costs.
A modification of the confidence screening mechanism based on adaptive weighing of every training instance at each cascade level of the Deep Forest is proposed. The idea underlying the modification is very simple and stems from the confidence screening mechanism idea proposed by Pang et al. to simplify the Deep Forest …
Safety-critical applications require machine learning models that output accurate and calibrated probabilities. While uncalibrated deep networks are known to make over-confident predictions, it is unclear how model confidence is impacted by the variations in the data, such as label noise or class size. In this paper, w…
Paper uses evidence theory to improve stock price forecasting accuracy.
Develops a method to find costly high-confidence errors in black box models.
LLMs generate answers under incomplete context, and their uncertainty should scale with missing information.
Develops confidence intervals for ECE, a measure of model calibration.
There are two major paradigms of white-box adversarial attacks that attempt to impose input perturbations. The first paradigm, called the fix-perturbation attack, crafts adversarial samples within a given perturbation level. The second paradigm, called the zero-confidence attack, finds the smallest perturbation needed …
Adversarial training yields robust models against a specific threat model, e.g., adversarial examples. Typically robustness does not generalize to previously unseen threat models, e.g., other norms, or larger perturbations. Our confidence-calibrated adversarial training (CCAT) tackles this problem by b…
A conformal procedure improves CoT reasoning by aggregating reasoning paths and calibrating abstention rules.
Private statistical inference methods improve confidence interval lengths.
Pathology reports contain useful information such as the main involved organ, diagnosis, etc. These information can be identified from the free text reports and used for large-scale statistical analysis or serve as annotation for other modalities such as pathology slides images. However, manual classification for a hug…
Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…
New framework tightens certified robustness gaps in machine learning models.
The paper introduces multicalibration to improve confidence scores in LLMs.
Develops statistical confidence sets for multidimensional scaling.
This paper presents approximate confidence intervals for each function of parameters in a Banach space based on a bootstrap algorithm. We apply kernel density approach to estimate the persistence landscape. In addition, we evaluate the quality distribution function estimator of random variables using integrated mean sq…
This paper improves confidence measurement in deep metric learning models.
Paper proposes a method to predict deep neural network confidences with guarantees.
New method improves auto-labeling accuracy by optimizing confidence functions.
Ensembling multiple predictions is a widely used technique for improving the accuracy of various machine learning tasks. One obvious drawback of ensembling is its higher execution cost during inference. In this paper, we first describe our insights on the relationship between the probability of prediction and the effec…
Deep neural networks bring in impressive accuracy in various applications, but the success often relies on the heavy network architecture. Taking well-trained heavy networks as teachers, classical teacher-student learning paradigm aims to learn a student network that is lightweight yet accurate. In this way, a portable…
New method estimates grouping loss in neural networks to improve confidence scores.
This paper proposes a DGP approach with UCBs for point target tracking over WSNs.
Confidence intervals improve evaluation of binary prediction rules in data mining.
Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessar…
It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence. In this work, we take the opposite stance: an overly confident model is more likely to be vulnerable to adversarial examples. This work is one of the most proactive approaches taken to date, as…
Assessing the predictive accuracy of black box classifiers is challenging in the absence of labeled test datasets. In these scenarios we may need to rely on a human oracle to evaluate individual predictions; presenting the challenge to create query algorithms to guide the search for points that provide the most informa…
Study improves fraud detection in e-commerce with a stacked model combining CNNs, GNNs, and confidence gating.
New method for accurate uncertainty estimation in deep learning predictions.
The unified approach of Feldman and Cousins allows for exact statistical inference of small signals that commonly arise in high energy physics. It has gained widespread use, for instance, in measurements of neutrino oscillation parameters in long-baseline experiments. However, the approach relies on the Neyman construc…
Structural equation models and Bayesian networks have been widely used to study causal relationships between continuous variables. Recently, a non-Gaussian method called LiNGAM was proposed to discover such causal models and has been extended in various directions. An important problem with LiNGAM is that the results a…
The hypothesis that sub-network initializations (lottery) exist within the initializations of over-parameterized networks, which when trained in isolation produce highly generalizable models, has led to crucial insights into network initialization and has enabled efficient inferencing. Supervised models with uncalibrat…
Improved DNN estimator with scalable subsampling for efficient inference.
Paper improves confidence set construction for SGD using multiplier bootstrap.
PaRCE estimates model confidence for CNNs across various uncertainties.