A new method for adapting to label shifts using class probability matching.
problem Adapting to label shifts where class probabilities differ between source and target domains.
method Class Probability Matching using Kernel Methods (CPMKM) framework.
result CPMKM outperforms existing methods on real datasets.
Estimates class posterior probabilities without using scores from classifiers.
problem Estimating class posterior probabilities for new points in classification tasks.
method Varying prior probabilities to derive the ratio of pdf's at point x, directly determining class posterior probabilities.
result A method to estimate posterior probabilities without relying on classification scores.
The law of total probability may be deployed in binary classification exercises to estimate the unconditional class probabilities if the class proportions in the training set are not representative of the population class proportions. We argue that this is not a conceptually sound approach and suggest an alternative ba…
Study on estimating class probabilities using empirical risk minimization.
problem Estimating class probabilities within binary classification.
method Empirical risk minimization (ERM) for class probability estimation.
result The estimator converges to true class probabilities under certain conditions.
Focal loss improves classification but not class-posterior probability estimation.
problem Improving class-posterior probability estimation from focal loss.
method Proved classification-calibration and derived a transformation to recover true class-posterior probabilities.
result A transformation of the confidence score from focal loss minimization allows recovery of true class-posterior probabilities.
Tutorial on estimating SVM class probabilities.
problem Estimating class probabilities for SVM models.
method Compute implied posterior probabilities via isotonic regression.
result Calibrated implied posterior probabilities for SVMs.
Fisher consistency improves class probability estimation under dataset shift.
problem Lack of Fisher consistency can lead to unreliable class probability estimates.
method Introduced Fisher consistency as a desirable property for class prior probability estimators.
result CDE-Iterate is not Fisher consistent and cannot be trusted for reliable estimates.
Improves probability estimates for small datasets in multi-class problems.
problem Inaccurate probability estimates in classification tasks, especially on small datasets.
method Introduced Data Generation and Grouping algorithm to improve calibration on small datasets, then applied to multi-class problems.
result Calibration error can be decreased using the proposed approach.
An imprecise SHAP method explains class probabilities with limited data.
problem Explaining class probabilities with limited training data.
method New approach for computing feature marginal contributions and general approach to interval-valued Shapley values.
result The imprecise SHAP method improves explanation of class probabilities.
New linear algorithms improve wSVMs for multiclass probability estimation.
problem Estimating conditional probabilities for multiclass problems.
method Proposed baseline learning and OVA learning schemes to improve wSVMs.
result Linear algorithms achieve optimal computational efficiency and good estimation accuracy.
Proposes novel wSVMs for sparse learning and accurate probability estimation.
problem Sparse features with redundant noise limit the performance of existing wSVMs.
method Develops ℓ1-norm and elastic net regularized wSVMs for automatic variable selection and probability estimation. result Elastic net regularized wSVMs achieve superior performance in variable selection and probability estimation.
Study on optimal rates for sequential probability assignment using smoothed analysis.
problem Optimal rates for sequential probability assignment under smoothed adversaries.
method General-purpose reduction from minimax rates to transductive learning, development of an efficient algorithm using MLE oracle.
result Optimal (logarithmic) fast rates for parametric and finite VC dimension classes, sublinear regret for general classes.
Friedman's method performs well for estimating class distributions.
problem Estimating prior class probabilities without label observations.
method Friedman's method and DeBias method for designing linear equation systems.
result Friedman's method performs well for binary and multi-class quantification.
We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to a robust estimator of the class probability P(y∣x). The regularization term meas…
We consider a random walk on the mapping class group of a surface of finite type. We assume that the random walk is determined by a probability measure whose support is finite and generates a non-elementary subgroup H. We further assume that H is not consisting only of lifts with respect to any one covering. Then w…
A method for classifying points with minimal queries using Hermite polynomials.
problem Classifying points from an unknown probability measure with minimal label queries.
method Convex combination of conditional probabilities, Hermite polynomial kernel for hierarchical support estimation.
result The method achieves high F-score for classification in hyper-spectral images and MNIST. Estimates growth of reciprocal classes in Hecke groups.
problem Estimating the growth of reciprocal conjugacy classes in Hecke groups.
method Using free product structure and word lengths of reciprocal elements, with tools from basic probability theory.
result Estimates the asymptotic growth of reciprocal conjugacy classes in Hecke groups.
New approach uses class domains for classification when distributions are unknown.
problem Traditional classification rules are inadequate when class distributions are ill-defined or unknown.
method Use class domains instead of class distributions for constructing a reliable decision function.
result Illustrated examples show the effectiveness of the new approach.
The paper addresses probability calibration for incomplete sequences.
problem Improving probability estimates from incomplete sequences.
method Adapting traditional calibration techniques to sequences of varying lengths.
result Proposed methods improve probability calibration for modern sequential models.
Conventional multiclass conditional probability estimation methods, such as Fisher's discriminate analysis and logistic regression, often require restrictive distributional model assumption. In this paper, a model-free estimation method is proposed to estimate multiclass conditional probability through a series of cond…
Optimizes feature shifts for tree ensemble reclassification.
problem Improving tree ensemble reclassification accuracy through feature perturbation.
method Mathematical optimization of feature shifts to maximize reclassification probability.
result Effective feature ranking and optimization for tree ensemble reclassification.
SJS model predicts label shifts in multinomial datasets.
problem Predicting label shifts in multinomial datasets.
method Sparse joint shift model for dataset shift.
result Valid predictions and class prior probabilities estimates.
New bounds on minimax regret for sequential probability assignment using logarithmic loss.
problem Minimizing regret in sequential probability assignment against arbitrary experts.
method Using self-concordance property of logarithmic loss to derive tight bounds.
result Tight bounds on minimax regret for various expert classes.
Defense against model stealing attacks by adding noise to class probabilities.
problem Protecting machine learning models from model stealing attacks.
method Adding noise to class probabilities to limit the success of attacks.
result Our defense can degrade the accuracy of the stolen model by at least 20% or require up to 64 times more queries.
This work introduces a new metric for comparing imprecise probability models.
problem Quantifying differences between imprecise probability models.
method Integral imprecise probability metric framework based on Choquet integral.
result IIPM enables comparison across different imprecise probability models and quantifies epistemic uncertainty.
CANE selects a subset of classes for large multi-class classification problems.
problem Large multi-class classification problems with high number of classes.
method CANE selects a small subset of candidate classes and samples the remaining classes.
result CANE achieves better prediction accuracy and significant speedup compared to state-of-the-art methods.
Paper introduces a novel method for estimating model confidence in deep neural classifiers.
problem Reliable confidence estimation for deep neural classifiers in safety-critical applications.
method Proposes a novel target criterion (true class probability) and learns it from data with an auxiliary model.
result The proposed method outperforms strong baselines in various tasks and network architectures.
A key prerequisite to optimal reasoning under uncertainty in intelligent systems is to start with good class probability estimates. This paper improves on the current best probability estimation trees (Bagged-PETs) and also presents a new ensemble-based algorithm (MOB-ESP). Comparisons are made using several benchmark …
Proposes a new confidence criterion for deep neural networks to predict failures.
problem Predicting failures in deep neural networks.
method Introduces True Class Probability (TCP) as a new confidence criterion and proposes a learning scheme to estimate it.
result The proposed approach consistently outperforms existing methods in failure prediction.
Subclass distillation improves small models by matching teacher's subclass probabilities.
problem Improving small models trained on limited data.
method Train a small model to match probabilities of subclasses invented by a large teacher model.
result Better small models trained on limited data.
Improves Active Learning by considering class imbalance and difficulty.
problem Active Learning's focus on individual samples ignores class distribution and difficulty.
method Proposes a method based on Bayes' rule to incorporate class imbalance, using a Variational Auto Encoder (VAE).
result Significantly outperforms state-of-the-art methods on datasets with heavy data imbalance.
Work in the classification literature has shown that in computing a classification function, one need not know the class membership of all observations in the training set; the unlabeled observations still provide information on the marginal distribution of the feature set, and can thus contribute to increased classifi…
New CPS model tackles conditional probability shift in machine learning.
problem Discrepancy between source and target distributions in machine learning.
method Conditional Probability Shift Model (CPSM) using multinomial regression and EM algorithm.
result Superior balanced classification accuracy on target data compared to existing methods.
Defines financial models without probability theory.
problem Establishing martingale theory without probability.
method Introducing supermartingales, martingales, and semimartingales in continuous price paths.
result Probability-free versions of martingale results established.
The framework of this paper is that of risk measuring under uncertainty, which is when no reference probability measure is given. To every regular convex risk measure on Cb(Ω), we associate a unique equivalence class of probability measures on Borel sets, characterizing the riskless non positive elements of $…
A Fourier-based learning algorithm for multiclass classification.
problem Highly nonlinear multiclass classification problems.
method Smoothing technique with low-pass filters to calculate probability distributions.
result Probabilistic explanation for classification without kernel functions.
ESRLCM clusters similar responses, more broadly than traditional models.
problem Clustering multivariate categorical data with common response patterns.
method Bayesian Equivalence Set Restricted Latent Class Model (ESRLCM).
result ESRLCM identifies clusters with similar item response probabilities.
A new approach to MI learning using bag-to-class divergence.
problem Sparse MI training sets and difficulty in classifying bags.
method Introducing bag-to-class divergence to MI learning, emphasizing hierarchical random vectors.
result Bag-to-class divergence is a more effective classifier for MI learning.
Hierarchical Softmax approximates class probabilities for large datasets efficiently.
problem Computational inefficiency of Softmax for large-scale classification tasks.
method Used Hierarchical Softmax to approximate class probabilities efficiently.
result Hierarchical Softmax performance degrades as the number of classes increases.
We show that the probability that a finitely supported random walk on a non-elementary subgroup of the the mapping class group gives a non-pseudo-Anosov element decays exponentially in the length of the random walk. More generally, we show that if R is a set of mapping class group elements with an upper bound on their …
Distribution networks model novel classes in open set learning.
problem Modeling novel classes in open set learning.
method Distribution networks map samples to a latent space where known and novel classes' distributions are jointly learned.
result Distribution networks accurately detect and model novel classes for subsequent classification.
A new method calculates optimal decisions from classifier outputs, improving predictions in drug discovery.
problem Finding optimal decisions from classifier outputs in fields like medicine.
method Develops a transducer that calculates probabilities from classifier outputs, enabling expected-utility maximization.
result Improves prediction accuracy in drug discovery problems, sometimes close to theoretical maximum.
New approach predicts event probabilities for better event detection.
problem Class imbalance and inaccurate event detection in time series analysis.
method Regression-based approach to predict probability densities at event locations.
result Regression-based approaches outperform segmentation-based methods.
SplineCalib calibrates probabilities using splines for better performance.
problem Calibrating probabilities for better accuracy and log-loss.
method Uses smoothing splines to determine a calibration function.
result Significant improvements to log-loss and accuracy on various problems.
Support vector classifier constructs confidence sets for binary classification.
problem Learning confidence sets with specific probability guarantees for binary classification.
method Support vector classifier to construct confidence sets.
result The proposed learner controls non-coverage rates and minimizes ambiguity with high probability.
Regular representation of measures on manifolds with boundary.
problem Representing probability densities on manifolds near boundaries.
method Sufficient conditions for Ck random maps, obstructions to representability. result Conditions for regular representation of measures near boundaries.
A new IPM uses ReLU networks to measure probability discrepancies.
problem Measuring the difference between two probability distributions in high dimensions.
method Proposes a new parametric IPM using ReLU neural networks to optimize and distinguish between distributions.
result The proposed IPM has good convergence rates and can be used as a surrogate for other IPMs.
KCal calibrates deep networks by embedding logits in a metric space.
problem Overconfident predictions from DNNs, especially in high-risk applications.
method KCal learns a metric space on the penultimate-layer latent embedding and generates predictions using kernel density estimates.
result KCal provides a provable full calibration guarantee and consistently outperforms baselines.