arXiv research

Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.

The study examines when to trust confidence thresholding in pseudo-labelling regression.

Improved AI lung ultrasound segmentation using expert confidence values.

New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.

Adaptive algorithm for outlier detection by balancing arm exploration and threshold estimation.

A new SSL method uses instance-dependent thresholds to improve accuracy.

ATC predicts target domain accuracy using only labeled and unlabeled data.

We study the tradeoff between computational effort and classification accuracy in a cascade of deep neural networks. During inference, the user sets the acceptable accuracy degradation which then automatically determines confidence thresholds for the intermediate classifiers. As soon as the confidence threshold is met,…

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors $k$ may depend on and diverge with sample size $n$. In addition to the case of known error variance, we define and study versions of the estimators when…

New method improves auto-labeling accuracy by optimizing confidence functions.

New framework assesses LM uncertainty without thresholding.

Method constructs confidence regions for linear models with arbitrary predictors.

Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the sho…

Paper presents a probabilistic model to improve LLM cascade performance.

Adversarial training yields robust models against a specific threat model, e.g., $L_\infty$ adversarial examples. Typically robustness does not generalize to previously unseen threat models, e.g., other $L_p$ norms, or larger perturbations. Our confidence-calibrated adversarial training (CCAT) tackles this problem by b…

Develops a method to find costly high-confidence errors in black box models.

Ensembling multiple predictions is a widely used technique for improving the accuracy of various machine learning tasks. One obvious drawback of ensembling is its higher execution cost during inference. In this paper, we first describe our insights on the relationship between the probability of prediction and the effec…

The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.

We define the information threshold in Bayesian decision-making.

Ridge regression is revisited with debiasing and thresholding, offering advantages over Lasso.

Develops spatial uncertainty guarantees for image segmentation models.

A new method optimizes material discovery by balancing exploration and exploitation.

New method improves certified robustness for classifier confidence.

We present a new algorithm based on an gradient ascent for a general Active Exploration bandit problem in the fixed confidence setting. This problem encompasses several well studied problems such that the Best Arm Identification or Thresholding Bandits. It consists of a new sampling rule based on an online lazy mirror …

CV inference can be invalid for relatively unstable model comparisons.

Entropy-based decoding improves DLM sampling efficiency.

This paper introduces a new method for uncertainty quantification in prediction models.

TBAL reduces manual annotation but requires validated data.

In this work, we propose data augmentation methods for embeddings from pre-trained deep learning models that take a weighted combination of a pair of input embeddings, as inspired by Mixup, and combine such augmentation with extra label softening. These methods are shown to significantly increase classification accurac…

An algorithm for maximizing rewards under linear cost constraints.

RegMixMatch optimizes Mixup for semi-supervised learning by integrating high- and low-confidence samples.

Assessing the predictive accuracy of black box classifiers is challenging in the absence of labeled test datasets. In these scenarios we may need to rely on a human oracle to evaluate individual predictions; presenting the challenge to create query algorithms to guide the search for points that provide the most informa…

New method corrects bias in CVaR estimation for extreme risks.

In a wide variety of sequential decision making problems, it can be important to estimate the impact of rare events in order to minimize risk exposure. A popular risk measure is the conditional value-at-risk (CVaR), which is commonly estimated by averaging observations that occur beyond a quantile at a given confidence…

Paper shows equivalence between two dividend preference models.

Deep neural networks obtain state-of-the-art performance on a series of tasks. However, they are easily fooled by adding a small adversarial perturbation to input. The perturbation is often human imperceptible on image data. We observe a significant difference in feature attributions of adversarially crafted examples f…

No policy can simultaneously be fully autonomous, optimally calibrated, and helpful, proving a trilemma.

Learning exists in the context of data, yet notions of confidence typically focus on model predictions, not label quality. Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, …

This is a report of our lessons learned building acoustic models from 1 Million hours of unlabeled speech, while labeled speech is restricted to 7,000 hours. We employ student/teacher training on unlabeled data, helping scale out target generation in comparison to confidence model based methods, which require a decoder…

To be effective, state of the art machine learning technology needs large amounts of annotated data. There are numerous compelling applications in healthcare that can benefit from high performance automated decision support systems provided by deep learning technology, but they lack the comprehensive data resources req…

The paper accelerates LLM inference by adding early exit heads trained in a self-supervised manner.

Framework for precise recall control in spatial conflation tasks.

A reject option improves partial-label learning's accuracy.

Dash selects dynamic pseudo labels from unlabeled data for semi-supervised learning.

In this paper, we develop a theory of market crashes resulting from a deleveraging shock. We consider two representative investors in a market holding different opinions about the public available information. The deleveraging shock forces the high confidence investors to liquidate their risky assets to pay back their …

With today's abundant streams of data, the only constant we can rely on is change. For stream classification algorithms, it is necessary to adapt to concept drift. This can be achieved by monitoring the model error, and triggering counter measures as changes occur. In this paper, we propose a drift detection mechanism …

EBUCB framework achieves optimal regret with bounded approximate inference error.

FROB model improves robustness and reliable confidence for few-shot OoD detection.