Unified approach to estimating class-prior and classifier from PU data.
problem Learning binary classifier from positive and unlabeled data with class-prior estimation.
method Alternately estimating class-prior and training classifier.
result Unified approach improves classifier performance by accounting for class-prior estimation error.
New method needed for class prior estimation when covariates are reduced.
problem Class prior estimation fails under covariate shift when covariates are reduced.
method Propose a probing algorithm for class prior estimation.
result Provable transformations preserving covariate shift are necessary for class prior estimation.
Paper generalizes PU classification for class prior shift and asymmetric error scenarios.
problem Bottlenecks in binary classification from PU data due to test marginal distribution and equal error penalties.
method Analysis of Bayes optimal classifier, risk minimization framework, and density ratio estimation framework.
result PU classification under class prior shift is equivalent to PU classification with asymmetric error.
We consider the problem of estimating the class prior in an unlabeled dataset. Under the assumption that an additional labeled dataset is available, the class prior can be estimated by fitting a mixture of class-wise data distributions to the unlabeled data distribution. However, in practice, such an additional labeled…
Recent advances in weakly supervised classification allow us to train a classifier only from positive and unlabeled (PU) data. However, existing PU classification methods typically require an accurate estimate of the class-prior probability, which is a critical bottleneck particularly for high-dimensional data. This pr…
Estimates class prior for unlabeled data using kernel embedding.
problem Estimating class prior in PU learning scenario where only positive and full population samples are available.
method Direct estimator based on distribution matching and kernel embedding in Reproducing Kernel Hilbert Space.
result Asymptotic consistency and explicit deviation bound for the estimator.
The problem of developing binary classifiers from positive and unlabeled data is often encountered in machine learning. A common requirement in this setting is to approximate posterior probabilities of positive and negative classes for a previously unseen data point. This problem can be decomposed into two steps: (i) t…
A new method ReCPE removes the need for a distributional assumption in PU learning.
problem Training binary classifiers with only positive and unlabeled data without negative data.
method Regrouping CPE (ReCPE) that constructs an auxiliary distribution to ensure positive data support is never in negative data support.
result ReCPE improves all state-of-the-art CPE methods on various datasets, indicating the need for the distributional assumption.
Work in the classification literature has shown that in computing a classification function, one need not know the class membership of all observations in the training set; the unlabeled observations still provide information on the marginal distribution of the feature set, and can thus contribute to increased classifi…
We develop a classification algorithm for estimating posterior distributions from positive-unlabeled data, that is robust to noise in the positive labels and effective for high-dimensional data. In recent years, several algorithms have been proposed to learn from positive-unlabeled data; however, many of these contribu…
SJS model predicts label shifts in multinomial datasets.
problem Predicting label shifts in multinomial datasets.
method Sparse joint shift model for dataset shift.
result Valid predictions and class prior probabilities estimates.
A common approach in positive-unlabeled learning is to train a classification model between labeled and unlabeled data. This strategy is in fact known to give an optimal classifier under mild conditions; however, it results in biased empirical estimates of the classifier performance. In this work, we show that the typi…
A simple regularization method improves model generalization.
problem Overfitting due to lack of labeled data in machine learning models.
method Density-fixing regularization method based on class prior distribution.
result Improves model generalization performance by approximating class prior distribution.
Detects changes in classifier scores to identify shifts in class priors.
problem Label shift changes in classification data.
method Sequential changepoint detection of classifier scores.
result Outperforms other detection procedures in label shift settings.
Paper tackles training binary classifiers from unlabeled data with minimal supervision.
problem Training arbitrary binary classifiers from only unlabeled data is impossible without supervision.
method Proposes an ERM-based learning method from two sets of unlabeled data with different class priors.
result The proposed method is consistent and outperforms state-of-the-art methods.
Bayesian framework estimates label shift for improved classifier performance.
problem Label shift in supervised learning leading to degraded classifier performance.
method Bayesian framework with dynamic Dirichlet priors and online EM algorithms.
result Significant improvements in classifier accuracy over state-of-the-art methods.
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…
New priors can update posteriors without re-estimating likelihoods.
problem Degradation of classification approaches when class priors change.
method Recompute posteriors using recovered likelihoods from original posteriors and new priors.
result Dynamic update of original posteriors is possible without re-estimating likelihoods.
In real-world classification problems, the class balance in the training dataset does not necessarily reflect that of the test dataset, which can cause significant estimation bias. If the class ratio of the test dataset is known, instance re-weighting or resampling allows systematical bias correction. However, learning…
The paper proposes a method to calibrate metrics for better interpretability and fairness in machine learning models.
problem Machine learning metrics are biased by class priors and vary across subpopulations.
method Calibration of precision-based metrics (F1-score, AUC-PR) to make them invariant to class priors.
result Calibrated metrics improve interpretability and provide better control over model performance.
Bayesian inference for deep neural networks using trace-class priors and MLMC.
problem Efficient Bayesian inference for deep neural networks.
method Trace-class neural network priors and Multilevel Monte Carlo method.
result Optimal computational complexity for Bayesian inference of TNN models.
GS-B3SE improves label shift estimation by smoothing priors on a graph.
problem Label shift adaptation when source and target distributions share conditional but not marginal probabilities.
method Graph-Smoothed Bayesian Black-Box Shift Estimator (GS-B3SE) places Laplacian-Gaussian priors on log-priors and confusion-matrix columns tied by a label-similarity graph. result GS-B3SE produces a tractable posterior with HMC or Newton-CG schemes, proving identifiability, contraction, and robustness. Novel semi-supervised method for X-ray classification with minimal labels.
problem Classifying X-ray data with limited labeled data.
method Graph-based semi-supervised learning with carefully selected class priors.
result Competitive results on ChestX-ray14 data set with reduced need for annotated data.
TPM improves medical image segmentation by separating foreground and background.
problem Few-shot medical image segmentation challenges due to background variability.
method Tied Prototype Model (TPM) focusing on foreground, adapting thresholds, and using class priors.
result TPM leads to improved segmentation accuracy compared to ADNet.
Mitigates overfitting in UU classification from two unlabeled datasets.
problem Overfitting in the UU classification method.
method Wrapping negative empirical risk terms with correction functions and proving consistency.
result Successfully mitigates overfitting and improves classification accuracy.
In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., ordinary supervised learning). Hitherto, neither theoretical nor experimental analysis has been given to explain this phenomenon. In …
Extends FJS analysis to general label spaces, including classification and regression.
problem Distribution shift in general label spaces, including covariate and label shifts.
method Proposes a framework for analyzing FJS in general label spaces and generalizes existing results.
result Generalizes FJS analysis to general label spaces, including classification and regression.
This paper tackles domain generalization by learning invariant class conditional distributions.
problem Learning invariant representations across different domains with varying distributions.
method Proposes a conditional invariant representation to ensure invariance of class conditional distributions.
result Guarantees invariance of the joint distribution P(h(X),Y) if class prior P(Y) remains invariant. While crowdsourcing has become an important means to label data, there is great interest in estimating the ground truth from unreliable labels produced by crowdworkers. The Dawid and Skene (DS) model is one of the most well-known models in the study of crowdsourcing. Despite its practical popularity, theoretical error …
Study investigates how neural networks perform with mislabeled data.
problem Understanding and mitigating the effects of mislabeled training data on neural network performance.
method Analysis of model equations and use of Maximum Likelihood (ML) estimate to infer clean model parameters.
result ML estimate of noisy model parameters determines clean model parameters, leading to a classifier adjustment method.
Paper proposes a reinforcement learning approach to learn classifiers from positive and unlabeled data.
problem Learning binary classifiers from positive and unlabeled data without prior class prior or label noise estimation.
method Alternately trains a policy network and a classifier using reinforcement learning, adapting label assumptions for unlabeled data.
result Significant improvement in classification performance compared to existing methods.
This work bounds classification error in machine learning for low Bayes error conditions.
problem Understanding the error mismatch between Bayes error and model-based classification error.
method Applying classification error bounds to study the relationship with Kullback-Leibler divergence and proposing a linear approximation for low Bayes error conditions.
result A linear approximation of the classification error bound for low Bayes error conditions is proposed.
This paper improves multi-class calibration methods using mutual information maximization-based binning.
problem Calibration of deep neural network predictions, especially for small prior classes.
method I-Max concept for binning, shared class-wise calibration strategy.
result Improves multi-class ranking and calibration performance using a small calibration set.
Paper introduces a variational approach for PU learning with improved performance and stability.
problem Learning binary classifiers from only positive and unlabeled data.
method Variational principle for PU learning, quantitatively evaluating modeling error, efficient loss function, margin maximizing loss function.
result Improved discriminative performance and numerical stability of the variational PU learning method.
New BNN architectures reduce computational cost for uncertainty quantification.
problem High computational cost in Bayesian neural networks.
method Partial trace-class Bayesian neural networks (PaTraC BNNs).
result Comparable uncertainty quantification with fewer parameters.
Language recognition system is typically trained directly to optimize classification error on the target language labels, without using the external, or meta-information in the estimation of the model parameters. However labels are not independent of each other, there is a dependency enforced by, for example, the langu…
GC-Flow uses graph flows for better clustering than traditional GCNs.
problem Traditional GCNs miss useful clustering information.
method Designing normalizing flows to replace GCN layers, creating a generative model.
result GC-Flow produces well-separated clusters while maintaining predictive power.
In-context learning solves PU classification without iterative optimization.
problem Binary classification with only labeled positives and unlabeled samples.
method Pretrained transformer (PUICL) that learns from synthetic PU datasets.
result Outperforms four standard PU learning baselines on 20 benchmarks.
Proposes a method to classify binary data from multiple unlabeled datasets.
problem High annotation costs in training classifiers from weakly supervised data.
method Introduces surrogate set classification (SSC) to predict data origin from multiple unlabeled datasets, then uses this to train a binary classifier.
result Demonstrates superior performance compared to existing methods.
Paper analyzes multiclass classification with high-dimensional data.
problem Understanding statistical properties and behavior of multiclass classification algorithms.
method Asymptotic analysis of linear multiclass classification.
result Test error varies significantly across different training algorithms and data distributions.
A framework for evaluating and benchmarking concept drift detection methods
problem Data stream mining challenged by concept drift
method A novel benchmarking framework
result Fair comparisons of drift detection methods
A new algorithm improves efficiency for semi-supervised AUC optimization.
problem Optimizing AUC in semi-supervised learning with limited labeled data.
method Quadruply Stochastic Gradients (QSG-S2AUC) for nonlinear classification problems.
result QSG-S2AUC converges to optimal solution in O(1/t) iterations.
We develop and demonstrate a probabilistic method for classifying rare objects in surveys with the particular goal of building very pure samples. It works by modifying the output probabilities from a classifier so as to accommodate our expectation (priors) concerning the relative frequencies of different classes of obj…
New method classifies mixtures without labels, recovering latent classes.
problem Classification without reliable instance-level labels.
method Posterior simplex geometry for multiclass learning.
result Classifier trained on mixture identities recovers latent classes and their proportions.
New algorithm learns efficiently with a simple 'yes/no' oracle.
problem Can efficient learning be achieved with a simpler oracle than ERM?
method Developed an oracle that returns 'yes' or 'no' for realizable datasets.
result Learnability is possible with a polynomial price in VC dimension.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator