This work presents a new classifier that is specifically designed to be fully interpretable. This technique determines the probability of a class outcome, based directly on probability assignments measured from the training data. The accuracy of the predicted probability can be improved by measuring more probability es…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “probability classifiers”
The paper compares one-hot encoding to Naïve Bayes for categorical variables.
problem Incorrect one-hot encoding affects Naïve Bayes performance.
method Mathematical and experimental analysis of PoB vs. categorical Naïve Bayes.
result Posterior probabilities are usually greater in the PoB case, but agree on the maximum a posteriori class label.
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…
Better calibration for small datasets improves probability estimates.
problem Limited data hinders traditional calibration methods.
method Generating additional calibration data improves classifier performance.
result The proposed approach enhances calibration for small datasets.
One of the central themes in the classification task is the estimation of class posterior probability at a new point . The vast majority of classifiers output a score for , which is monotonically related to the posterior probability via an unknown relationship. There are many attempts in the literature …
Proposes a method to estimate time-dependent probability density functions using binary classifiers.
problem Estimating time-dependent probability density functions of stochastic processes.
method Trains a time-dependent binary classifier to discriminate between realizations of a stochastic process at two nearby time instants.
result Explicitly models and accurately reconstructs complex time-dependent, multi-modal, and near-degenerate densities.
Motivated by problems of anomaly detection, this paper implements the Neyman-Pearson paradigm to deal with asymmetric errors in binary classification with a convex loss. Given a finite collection of classifiers, we combine them and obtain a new classifier that satisfies simultaneously the two following properties with …
A new method detects changes in multivariate data using random forests.
problem Detecting changes in multivariate data.
method A computationally feasible search method using random forests and class probability predictions.
result Consistently locates change points in simulations.
This paper reviews metrics to assess AI model calibration accuracy.
problem AI model probabilities do not always match their true accuracy.
method Comprehensive review of 82 probability calibration metrics.
result Identified 4 classifier families and 1 object detection family of metrics.
Classification is an important statistical learning tool. In real application, besides high prediction accuracy, it is often desirable to estimate class conditional probabilities for new observations. For traditional problems where the number of observations is large, there exist many well developed approaches. Recentl…
Efficient learning of minimax risk classifiers in high dimensions.
problem Efficient learning of classifiers in high-dimensional data.
method Iterative algorithm leveraging constraint generation methods for minimax risk classifiers.
result The algorithm provides efficient learning and feature selection in high-dimensional scenarios.
A new method for measuring prediction uncertainty in classifiers.
problem Measuring uncertainty of predictions from machine learning methods.
method Density Based Calibration (DBCal) technique.
result Expected calibration error of less than 0.2% on binary classifiers and less than 3% on semantic segmentation networks.
Refinement revisited with connections to Bayes error, conditional entropy and calibrated classifiersstat.ML
The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diver…
This paper optimizes binary linear classifiers by tuning their weight vectors.
problem Optimizing the weight vector of binary linear classifiers for better performance.
method Parameterization of the discriminant through a scalar to control trade-offs between informative and noisy terms.
result Weight vector tuning compensates for non-optimal native hyperparameters, improving classification performance.
Reduces quantifier variance with accuracy optimization of base classifier.
problem Minimizing quantifier variance under prior probability shift.
method Optimizes the Brier score of a base classifier for training data.
result Optimizing Brier score on training data reduces quantifier variance on test data.
Probabilistic classifiers output a probability distribution on target classes rather than just a class prediction. Besides providing a clear separation of prediction and decision making, the main advantage of probabilistic models is their ability to represent uncertainty about predictions. In safety-critical applicatio…
In most classification tasks there are observations that are ambiguous and therefore difficult to correctly label. Set-valued classifiers output sets of plausible labels rather than a single label, thereby giving a more appropriate and informative treatment to the labeling of ambiguous instances. We introduce a framewo…
A new framework separates classifier calibration and discrimination.
problem Combining reliability and resolution in probabilistic predictions.
method Manokhin Probability Matrix separates reliability and resolution using Spiegelhalter Z-statistic and AUC-ROC.
result Classifiers are categorized into four archetypes: Eagle, Bull, Sloth, and Mole.
A new method calculates optimal decisions from classifier outputs, improving predictions in drug discovery.
problem Finding optimal decisions from classifier outputs in fields like medicine.
method Develops a transducer that calculates probabilities from classifier outputs, enabling expected-utility maximization.
result Improves prediction accuracy in drug discovery problems, sometimes close to theoretical maximum.
Numbers and numerical vectors account for a large portion of data. However, recently the amount of string data generated has increased dramatically. Consequently, classifying string data is a common problem in many fields. The most widely used approach to this problem is to convert strings into numerical vectors using …
Support vector machines (SVM) and other kernel techniques represent a family of powerful statistical classification methods with high accuracy and broad applicability. Because they use all or a significant portion of the training data, however, they can be slow, especially for large problems. Piecewise linear classifie…
Binary classification is highly used in credit scoring in the estimation of probability of default. The validation of such predictive models is based both on rank ability, and also on calibration (i.e. how accurately the probabilities output by the model map to the observed probabilities). In this study we cover the cu…
New method calibrates classifier probabilities with guaranteed coverage.
problem Inaccurate probability estimates by classifiers in high-risk applications.
method Adaptive temperature scaling algorithm for conformal prediction.
result Improves calibration error measures and standard metrics across various tasks.
Paper introduces a novel method for estimating model confidence in deep neural classifiers.
problem Reliable confidence estimation for deep neural classifiers in safety-critical applications.
method Proposes a novel target criterion (true class probability) and learns it from data with an auxiliary model.
result The proposed method outperforms strong baselines in various tasks and network architectures.
Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single corrective multiplicative factor for inputs to the last softmax layer. On non-neural models the exist…
Focal loss improves classification but not class-posterior probability estimation.
problem Improving class-posterior probability estimation from focal loss.
method Proved classification-calibration and derived a transformation to recover true class-posterior probabilities.
result A transformation of the confidence score from focal loss minimization allows recovery of true class-posterior probabilities.
Most binary classifiers work by processing the input to produce a scalar response and comparing it to a threshold value. The various measures of classifier performance assume, explicitly or implicitly, probability distributions and of the response belonging to either class, probability distributions for the…
Study classifies submanifolds in probability simplex.
problem Classifying submanifolds in the probability simplex.
method Complete classification through geometric analysis.
result Doubly totally-umbilical submanifolds identified and classified.
Relation extraction models suffer from limited qualified training data. Using human annotators to label sentences is too expensive and does not scale well especially when dealing with large datasets. In this paper, we use Auxiliary Classifier Generative Adversarial Networks (AC-GANs) to generate high-quality relational…
The study quantifies decision-making risks from suboptimal classifiers and proposes methods to reduce these risks.
problem Excess risk in decision-making from suboptimal probabilistic classifiers.
method Analytical expressions and upper/lower bounds for excess risk, calibration curve estimation, grouping loss estimator.
result Identifies regimes where recalibration alone or post-training is more effective.
The paper analyzes recalibration methods for binary classifiers under distribution shift.
problem Recalibrating binary classifiers to match a target prior probability.
method Analysis of distribution shift assumptions and proposal of new recalibration methods.
result QMM methods provide conservative results for risk weights functions.
In many real-world applications of machine learning classifiers, it is essential to predict the probability of an example belonging to a particular class. This paper proposes a simple technique for predicting probabilities based on optimizing a ranking loss, followed by isotonic regression. This semi-parametric techniq…
We study the effect of imperfect training data labels on the performance of classification methods. In a general setting, where the probability that an observation in the training dataset is mislabelled may depend on both the feature vector and the true label, we bound the excess risk of an arbitrary classifier trained…
We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., t…
This paper addresses the general problem of accurate identification of oil reservoirs. Recent improvements in well or borehole logging technology have resulted in an explosive amount of data available for processing. The traditional methods of analysis of the logs characteristics by experts require significant amount o…
Simpler majority vote of three classifiers achieves optimal error bounds.
problem Developing an optimal PAC learning algorithm in the realizable setting.
method Returning the majority vote of three ERM classifiers.
result Achieves optimal in-expectation bound on error.
New method provides formal uncertainty guarantees for image classifiers.
problem Uncertainty quantification for image classifiers without formal guarantees.
method Adapts conformal prediction to give stable, formal coverage guarantees.
result Method outperforms existing approaches in coverage and set size.
Develops a new minimax probability machine for imbalanced classification tasks.
problem Imbalanced classification tasks with non-decomposable performance measures.
method Derives an equivalent form of the MPMF model for solving linear and nonlinear classifiers.
result Demonstrates the effectiveness of the new model on real-world datasets.
Efficient methods estimate concordance probability for big data.
problem Efficiently calculating concordance probability in large datasets.
method Proposes two estimation methods for discrete and continuous settings.
result Estimators are accurate and computationally efficient.
A new L2D system produces calibrated probabilities of expert correctness without sacrificing accuracy.
problem Calibration of learning to defer systems for safety.
method One-vs-all classifiers with a consistent surrogate loss function.
result Proposes a calibrated L2D system that outperforms existing methods in accuracy and calibration.
A method for consensus prediction from probabilistic classifier outputs.
problem Combining probabilistic classifier outputs without prior information.
method Iteratively updating predictions based on classifier diversity and convergence.
result Consensus predictions converge to a decision based on classifier outputs' correspondence.
In this paper we present a new Bayesian network model for classification that combines the naive-Bayes (NB) classifier and the finite-mixture (FM) classifier. The resulting classifier aims at relaxing the strong assumptions on which the two component models are based, in an attempt to improve on their classification pe…
Study shows -NN classifier is not universally consistent on but consistent on discrete and specific measure spaces.
problem Consistency of -NN classifier under Wasserstein distance on measure spaces.
method Analysis of -NN classifier properties under Wasserstein distance, use of -finite metric dimension, geodesic structures of Wasserstein spaces.
result Consistency of -NN classifier on specific measure spaces (discrete, Gaussian, wavelet series) but not on .
Many modern machine learning classifiers are shown to be vulnerable to adversarial perturbations of the instances. Despite a massive amount of work focusing on making classifiers robust, the task seems quite challenging. In this work, through a theoretical study, we investigate the adversarial risk and robustness of cl…
This work introduces significativity indices for agreement values between classifiers.
problem Evaluating the quality of agreement measures between classifiers.
method Proposes general approach and two specific indices for evaluation.
result Introduces two new significativity indices for evaluating agreement values.
Two strategies extend multi-label chaining for imprecise probability estimates.
problem Handling imprecise probability estimates in multi-label classification.
method Adapting multi-label chaining to use convex sets of distributions (credal sets).
result Adapted approaches produce relevant cautiousness on hard-to-predict instances.
Bayesian Conditional Gaussian Network Classifiers with Applications to Mass Spectra Classificationcs.LG
Classifiers based on probabilistic graphical models are very effective. In continuous domains, maximum likelihood is usually used to assess the predictions of those classifiers. When data is scarce, this can easily lead to overfitting. In any probabilistic setting, Bayesian averaging (BA) provides theoretically optimal…
Obtaining accurate and well calibrated probability estimates from classifiers is useful in many applications, for example, when minimising the expected cost of classifications. Existing methods of calibrating probability estimates are applied globally, ignoring the potential for improvements by applying a more fine-gra…