Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessar…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “class prevalence”
Proposes PQ, a more precise Bayesian quantifier for prevalence estimation.
problem Uncertainty quantification in prevalence estimation.
method Bayesian quantification methods, focusing on precision and coverage.
result PQ provides more precise and well-calibrated uncertainty quantification.
Analysis of Diagnostics (Part II): Prevalence, Linear Independence, and Unsupervised Learningstat.ML
Paper explores how unsupervised learning can be understood through linear algebra concepts.
problem Understanding unsupervised learning through linear algebra concepts.
method Introducing the concept of linearly independent populations and using them to solve for prevalence values.
result Unsupervised learning can be realized as a generalization of supervised learning.
Analysis of Diagnostics (Part I): Prevalence, Uncertainty Quantification, and Machine Learningstat.ML
This study connects prevalence and machine learning for diagnostic testing.
problem Uncertainty quantification in machine learning for diagnostic tests.
method Developed a numerical homotopy algorithm to estimate classification boundaries and quantify uncertainty.
result The proposed method stabilizes uncertainty quantification in machine learning for diagnostic tests.
Develops RES metrics for stable rare-event forecasting evaluation.
problem Challenges in evaluating forecasts of rare events.
method Rare-event-stable (RES) metrics designed to maintain stable thresholds under extreme rarity.
result RES metrics maintain stable thresholds, consistent model rankings, and near-complete prevalence invariance.
The estimation of class prevalence, i.e., the fraction of a population that belongs to a certain class, is a very useful tool in data analytics and learning, and finds applications in many domains such as sentiment analysis, epidemiology, etc. For example, in sentiment analysis, the objective is often not to estimate w…
New conformal prediction methods for long-tailed classification problems.
problem Rare classes are systematically omitted in existing conformal prediction methods.
method Introduced a new conformal score function and a new interpolation procedure.
result Smoothly trade off set size and class-conditional coverage.
Novel unsupervised scheme for highly imbalanced and overlapping datasets.
problem Highly imbalanced and overlapping classes in medical datasets.
method Unsupervised domain adaptation scheme based on Quantification.
result High quality results for Quantification and Domain Adaptation.
HistNetQ improves quantification tasks by optimizing loss functions and eliminating label requirements.
problem Quantification of class prevalence in bags of examples.
method Permutation-invariant Histograms and deep neural networks.
result HistNetQ outperforms other quantification methods and optimizes custom loss functions.
The paper discusses thresholds and bounds for accuracy in binary classification systems.
problem The accuracy of binary classification systems and its dependence on prevalence.
method Analyzing the precision-prevalence curve and negative predictive value-prevalence curve to find thresholds and bounds.
result Thresholds ( and ) bound various accuracy metrics (Fβ, F1, FM, MCC) and the ratio of maximum accuracy to prevalence.
Measuring the Prevalence of Policy Violating Content with ML Assisted Sampling and LLM Labelingcs.LG
Measures policy-violating content prevalence with ML-assisted sampling and LLM labeling.
problem Accurate measurement of content violations that are often rare and costly to label.
method Design-based measurement system using ML-assisted probability sampling and LLM labeling.
result Produces unbiased prevalence estimates with confidence intervals and dashboard drilldowns.
Classification is the task of predicting the class labels of objects based on the observation of their features. In contrast, quantification has been defined as the task of determining the prevalences of the different sorts of class labels in a target dataset. The simplest approach to quantification is Classify & Count…
Continuous Sweep improves binary quantifier performance.
problem Estimating class prevalence in datasets.
method Parametric binary quantifier inspired by Median Sweep, using parametric class distributions and mean of Adjusted Count estimates.
result Continuous Sweep outperforms other quantifiers in simulations and empirical data analysis.
Bayesian method corrects bias in imbalanced datasets.
problem Prevalence bias in machine learning datasets.
method Bayesian risk minimization framework, bias-corrected loss function.
result Corrected loss function improves model performance.
We propose a discrete surface theory in that unites the most prevalent versions of discrete special parametrizations. This theory encapsulates a large class of discrete surfaces given by a Lax representation and, in particular, the one-parameter associated families of constant curvature surfaces. The theo…
Geometry-aware KDE model improves multiclass quantification.
problem Accurately estimating class prevalence for label shift adaptation.
method Log-ratio representations and Aitchison geometry for compositional data, shrinkage regularization.
result Competitive with state-of-the-art quantifiers, often improving over standard KDE-based baselines.
Study finds AUC is most consistent across different prevalence in binary classification.
problem Consistency of model evaluation metrics across varying prevalence in binary classification.
method Analysis of 156 data scenarios with 18 metrics, 5 models, and a random guess model.
result AUC has the smallest variance in evaluating individual models and ranking of models.
Bayesian methods improve group testing for identifying infected patients.
problem Identifying infected patients from group testing results with false positives.
method Bayesian inference and belief propagation algorithm, combined with expectation-maximization method.
result True-positive rate improved by considering credible intervals.
Optimizes classification algorithms with bounds on error rates.
problem Bounding uncertainties in classifier outputs for diagnostic testing.
method Set-theoretic and probabilistic arguments to derive uniform error bounds.
result Optimal partition minimizes the largest Gershgorin radius of the confusion matrix.
New proposed models are often compared to state-of-the-art using statistical significance testing. Literature is scarce for classifier comparison using metrics other than accuracy. We present a survey of statistical methods that can be used for classifier comparison using precision, accounting for inter-precision corre…
New proof shows rapid mixing for random walks on nilmanifolds.
problem Proving rapid mixing for random walks on nilmanifolds.
method Proved rapid mixing for almost all random walks generated by m translations on nilmanifolds under mild assumptions.
result For several classical classes of nilmanifolds, m=2 suffices for rapid mixing.
The paper proves prevalent existence and partially determines moduli space of area-minimizing surfaces with fractal singular sets.
problem Existence and moduli space of area-minimizing surfaces with fractal singular sets.
method Proof of prevalent existence, determination of moduli space, refinement of strata.
result Sharp results on moduli space and refinement of strata, showing fractal singularities do not completely dissolve under generic perturbations.
One-hot encoding is a labelling system that embeds classes as standard basis vectors in a label space. Despite seeing near-universal use in supervised categorical classification tasks, the scheme is problematic in its geometric implication that, as all classes are equally distant, all classes are equally different. Thi…
The quantification problem consists of determining the prevalence of a given label in a target population. However, one often has access to the labels in a sample from the training population but not in the target population. A common assumption in this situation is that of prior probability shift, that is, once the la…
Model compression has been widely adopted to obtain light-weighted deep neural networks. Most prevalent methods, however, require fine-tuning with sufficient training data to ensure accuracy, which could be challenged by privacy and security issues. As a compromise between privacy and performance, in this paper we inve…
We propose a framework that learns a representation transferable across different domains and tasks in a label efficient manner. Our approach battles domain shift with a domain adversarial loss, and generalizes the embedding to novel task using a metric learning-based approach. Our model is simultaneously optimized on …
Class imbalance problems manifest in domains such as financial fraud detection or network intrusion analysis, where the prevalence of one class is much higher than another. Typically, practitioners are more interested in predicting the minority class than the majority class as the minority class may carry a higher misc…
Improves SSL with doubly robust estimation of unlabeled class distribution.
problem Limited labeled data and long-tailed class distributions in unlabeled data.
method Explicitly estimate unlabeled class distribution using doubly robust estimator.
result Improves performance of SSL methods on unlabeled data.
New method adapts to structural shifts in graph data for better label prevalence estimation.
problem Structural shifts in graph data affect label prevalence estimation.
method Importance sampling variant of KDEy quantification approach.
result Adapts to structural shifts and outperforms standard approaches.
Quantification is a supervised learning task that consists in predicting, given a set of classes C and a set D of unlabelled items, the prevalence (or relative frequency) p(c|D) of each class c in C. Quantification can in principle be solved by classifying all the unlabelled items and counting how many of them have bee…
Estimates disease prevalence using non-ignorable missing data in health surveys.
problem Estimating disease prevalence in non-representative samples with non-ignorable missing data.
method Connects auxiliary proxy variable framework to label shift setting, uses high-dimensional covariates without generative models.
result Fails to account for non-ignorable missingness can lead to significant misestimations.
This chapter tackles class imbalance in datasets to promote data democracy.
problem Class imbalance in datasets leading to biased decisions and policies.
method Statistical measures and data-level methods (oversampling, undersampling, etc.) applied to a real dataset.
result Popular data-level methods improve performance in handling class imbalance.
Credit risk prediction is an effective way of evaluating whether a potential borrower will repay a loan, particularly in peer-to-peer lending where class imbalance problems are prevalent. However, few credit risk prediction models for social lending consider imbalanced data and, further, the best resampling technique t…
The Centers for Disease Control and Prevention (CDC) coordinates a labor-intensive process to measure the prevalence of autism spectrum disorder (ASD) among children in the United States. Random forests methods have shown promise in speeding up this process, but they lag behind human classification accuracy by about 5%…
Study reveals class-dependent effects in perturbation-based feature attribution metrics for time series classification.
problem Varying effectiveness of perturbation-based metrics across different classes in time series models.
method Systematic empirical analysis across multiple datasets, model architectures, and perturbation strategies.
result Perturbation-based metrics show varying effectiveness across classes, with some metrics performing better for certain classes.
Self-augmentation improves deep networks for few-shot learning with minimal training data.
problem Improving deep networks' generalization to unseen classes with limited training examples.
method Self-augmentation using self-mix and self-distillation techniques, combined with regional dropout and local representation learning.
result The method outperforms state-of-the-art few-shot learning methods on prevalent benchmarks.
Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
problem Understanding and optimizing deep learning training phases.
method Direct measurements on three deepnet architectures across seven datasets.
result Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
The paper provides a new uniform tail bound for empirical processes.
problem Developing a uniform tail bound for empirical processes indexed by a class of functions.
method Introducing a deflation step to the standard generic chaining argument, and using a natural seminorm based on Cramér functions.
result Established a new uniform tail bound for empirical processes.
Machine learning classifiers often stumble over imbalanced datasets where classes are not equally represented. This inherent bias towards the majority class may result in low accuracy in labeling minority class. Imbalanced learning is prevalent in many real-world applications, such as medical research, network intrusio…
Reconstruction error is a prevalent score used to identify anomalous samples when data are modeled by generative models, such as (variational) auto-encoders or generative adversarial networks. This score relies on the assumption that normal samples are located on a manifold and all anomalous samples are located outside…
Evaluation often aims to reduce the correctness or error characteristics of a system down to a single number, but that always involves trade-offs. Another way of dealing with this is to quote two numbers, such as Recall and Precision, or Sensitivity and Specificity. But it can also be useful to see more than this, and …
Study on top- classification with new loss functions and algorithms.
problem Improving multi-class classification accuracy and cardinality trade-off.
method Introducing cardinality-aware loss functions and deriving their consistency bounds.
result New cardinality-aware algorithms for top- classification.
In this paper, we present a new approach to time series forecasting. Time series data are prevalent in many scientific and engineering disciplines. Time series forecasting is a crucial task in modeling time series data, and is an important area of machine learning. In this work we developed a novel method that employs …
Several applications of Reinforcement Learning suffer from instability due to high variance. This is especially prevalent in high dimensional domains. Regularization is a commonly used technique in machine learning to reduce variance, at the cost of introducing some bias. Most existing regularization techniques focus o…
Reduces quantifier variance with accuracy optimization of base classifier.
problem Minimizing quantifier variance under prior probability shift.
method Optimizes the Brier score of a base classifier for training data.
result Optimizing Brier score on training data reduces quantifier variance on test data.
CFA improves model's ability to generalize across unseen domain-class combinations.
problem Challenges in real-world machine learning applications due to data distribution shifts and limited training data.
method Developed Compositional Feature Alignment (CFA) technique to improve CG ability of pretrained models.
result CFA outperforms common finetuning techniques in compositional generalization.
Dynamic topic models (DTMs) model the evolution of prevalent themes in literature, online media, and other forms of text over time. DTMs assume that word co-occurrence statistics change continuously and therefore impose continuous stochastic process priors on their model parameters. These dynamical priors make inferenc…
Fisheye cameras are commonly employed for obtaining a large field of view in surveillance, augmented reality and in particular automotive applications. In spite of their prevalence, there are few public datasets for detailed evaluation of computer vision algorithms on fisheye images. We release the first extensive fish…