Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

19385776 · May 202619922001200920172026
48 results for prevalence threshold

The paper discusses thresholds and bounds for accuracy in binary classification systems.

problem The accuracy of binary classification systems and its dependence on prevalence.
method Analyzing the precision-prevalence curve and negative predictive value-prevalence curve to find thresholds and bounds.
result Thresholds (φeφ_e and φnφ_n) bound various accuracy metrics (Fβ, F1, FM, MCC) and the ratio of maximum accuracy to prevalence.

Study finds AUC is most consistent across different prevalence in binary classification.

problem Consistency of model evaluation metrics across varying prevalence in binary classification.
method Analysis of 156 data scenarios with 18 metrics, 5 models, and a random guess model.
result AUC has the smallest variance in evaluating individual models and ranking of models.

Develops RES metrics for stable rare-event forecasting evaluation.

problem Challenges in evaluating forecasts of rare events.
method Rare-event-stable (RES) metrics designed to maintain stable thresholds under extreme rarity.
result RES metrics maintain stable thresholds, consistent model rankings, and near-complete prevalence invariance.

The study examines when to trust confidence thresholding in pseudo-labelling regression.

problem Calibrated probabilities from classifiers used for pseudo-labelling need careful handling to avoid bias in downstream regression.
method Developed a diagnostic apparatus to predict and bound the bias induced by confidence thresholding, derived a closed-form expression for the attenuation bias.
result The bias can be predicted from the residual score variance VV^{*}, motivating a structural separation between classifier features and downstream controls.

This paper provides new insight into maximizing F1 scores in the context of binary classification and also in the context of multilabel classification. The harmonic mean of precision and recall, F1 score is widely used to measure the success of a binary classifier when one class is rare. Micro average, macro average, a…

2014-02-08abs ↗pdf ↗

This paper is concerned with the hard thresholding operator which sets all but the kk largest absolute elements of a vector to zero. We establish a {\em tight} bound to quantitatively characterize the deviation of the thresholded solution from a given signal. Our theoretical result is universal in the sense that it ho…

2016-05-05abs ↗pdf ↗

New conformal prediction methods for long-tailed classification problems.

problem Rare classes are systematically omitted in existing conformal prediction methods.
method Introduced a new conformal score function and a new interpolation procedure.
result Smoothly trade off set size and class-conditional coverage.

Measures policy-violating content prevalence with ML-assisted sampling and LLM labeling.

problem Accurate measurement of content violations that are often rare and costly to label.
method Design-based measurement system using ML-assisted probability sampling and LLM labeling.
result Produces unbiased prevalence estimates with confidence intervals and dashboard drilldowns.

Paper explores how unsupervised learning can be understood through linear algebra concepts.

problem Understanding unsupervised learning through linear algebra concepts.
method Introducing the concept of linearly independent populations and using them to solve for prevalence values.
result Unsupervised learning can be realized as a generalization of supervised learning.

This study connects prevalence and machine learning for diagnostic testing.

problem Uncertainty quantification in machine learning for diagnostic tests.
method Developed a numerical homotopy algorithm to estimate classification boundaries and quantify uncertainty.
result The proposed method stabilizes uncertainty quantification in machine learning for diagnostic tests.

New method reduces GP bandit complexity while maintaining good performance.

problem Computational burden in Bayesian optimization with Gaussian processes.
method Information thresholding to compress GP posterior and reduce complexity.
result Sublinear regret bounds with sublinear posterior complexity.

Bayesian methods improve group testing for identifying infected patients.

problem Identifying infected patients from group testing results with false positives.
method Bayesian inference and belief propagation algorithm, combined with expectation-maximization method.
result True-positive rate improved by considering credible intervals.

The paper proves prevalent existence and partially determines moduli space of area-minimizing surfaces with fractal singular sets.

problem Existence and moduli space of area-minimizing surfaces with fractal singular sets.
method Proof of prevalent existence, determination of moduli space, refinement of strata.
result Sharp results on moduli space and refinement of strata, showing fractal singularities do not completely dissolve under generic perturbations.

Estimates disease prevalence using non-ignorable missing data in health surveys.

problem Estimating disease prevalence in non-representative samples with non-ignorable missing data.
method Connects auxiliary proxy variable framework to label shift setting, uses high-dimensional covariates without generative models.
result Fails to account for non-ignorable missingness can lead to significant misestimations.

New model shows neural networks can use noise to improve long-tailed data classification.

problem Understanding overfitting in neural networks with long-tailed data.
method Refined feature-noise data model incorporating class-dependent heterogeneous noise.
result Neural networks can leverage data noise to learn implicit features improving long-tailed data classification.

New algorithm for reinforcement learning in uncertain environments with unknown thresholds.

problem Safety in reinforcement learning in unknown and uncertain environments.
method Growing-Window estimator sampling and Stochastic Pessimistic-Optimistic Thresholding (SPOT) algorithm.
result Achieves sublinear regret and constraint violation of ildeO(T) ilde{\mathcal{O}}(\sqrt{T}).

The estimation of class prevalence, i.e., the fraction of a population that belongs to a certain class, is a very useful tool in data analytics and learning, and finds applications in many domains such as sentiment analysis, epidemiology, etc. For example, in sentiment analysis, the objective is often not to estimate w…

2016-05-13abs ↗pdf ↗

A new SSL method uses instance-dependent thresholds to improve accuracy.

problem Improving semi-supervised learning by better selecting confident unlabeled instances.
method Proposes instance-dependent thresholds that vary based on the ambiguity and error rates of pseudo-labels for each unlabeled instance.
result Demonstrates that instance-dependent thresholds provide a probabilistic guarantee for correct pseudo-labels.

Adaptive algorithm for outlier detection by balancing arm exploration and threshold estimation.

problem Identifying outliers in a set of rewards where the threshold is a function of all rewards.
method Adaptively updated confidence interval for the threshold based on previous rounds' estimates, balancing exploration of individual arms and the outlier threshold.
result Efficient algorithm with reduced sample complexity for outlier detection.

The article examines different thresholding methods for improving PAM algorithm in cancer classification.

problem High-dimensional classification with too many features selected by PAM.
method Extends PAM with hard and order thresholding methods and a deep search algorithm.
result Improved cancer status prediction accuracy and smaller number of features.

Although the threshold network is one of the most used tools to characterize the underlying structure of a stock market, the identification of the optimal threshold to construct a reliable stock network remains challenging. In this paper, the concept of dynamic consistence between the threshold network and the stock ma…

2018-03-06abs ↗pdf ↗

Proposes a conservative LR estimator for infrequent data near a frequency threshold.

problem Overestimation of likelihood ratios for infrequent data near a frequency threshold.
method Conservative likelihood ratio estimator for frequencies slightly above a threshold.
result Improves prediction accuracy in named entity context prediction.

New method trains neural networks with threshold activation functions efficiently.

problem Training neural networks with threshold activation functions is challenging due to zero gradients.
method We study weight decay regularized training problems of deep neural networks with threshold activations, showing they can be formulated as convex optimization problems.
result Regularized deep threshold network training problems can be formulated as standard convex optimization problems, paralleling the LASSO method.

Reconstruction error is a prevalent score used to identify anomalous samples when data are modeled by generative models, such as (variational) auto-encoders or generative adversarial networks. This score relies on the assumption that normal samples are located on a manifold and all anomalous samples are located outside…

2019-05-28abs ↗pdf ↗

Polynomial neural networks explore thresholds for maximum expressiveness.

problem Understanding the limits of polynomial neural networks' expressiveness.
method Introducing activation degree threshold to measure network expressiveness and proving its existence and upper bounds.
result Polynomial neural networks with equi-width architectures achieve the maximum expressiveness.

Paper introduces threshold invariant fairness to ensure equitable predictions across different groups.

problem Machine learning models can be unfair to certain groups based on sensitive attributes.
method Proposes threshold invariant fairness and uses two approximation methods to equalize risk distributions.
result Demonstrates effectiveness in alleviating threshold sensitivity in fairness models.

Study evaluates thresholds for removing noise from DNN weights using random matrix theory.

problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.

FILTER model uses fusion penalized logistic threshold regression for high-dimensional data with unknown cut points.

problem Modeling high-dimensional data with unknown cut points and binary responses.
method Fusion penalized logistic threshold regression (FILTER) model with fused lasso penalty for variable selection.
result Established non-asymptotic error bounds for coefficient estimation and model selection consistency.

The paper considers an investment timing problem appearing in real options theory. Present values from an investment project are modeled by general diffusion process. We prove necessary and sufficient conditions under which an optimal investment time is induced by threshold strategy. We study also the conditions of opt…

2015-11-02abs ↗pdf ↗

Paper proves integrability and entropy compactness for Kähler potentials with uniform log-log threshold.

problem Integrability and entropy compactness for Kähler potentials with specific density.
method Skoda-Zeriahi type integrability theorem and log-log threshold detection.
result Positivity of integrability threshold and entropy compactness for uniform log-log threshold.