A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.
In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with perfo…
A widespread approach in machine learning to evaluate the quality of a classifier is to cross -- classify predicted and actual decision classes in a confusion matrix, also called error matrix. A classification tool which does not assume distributional parameters but only information contained in the data is based on th…
Data collection and annotation are time-consuming in machine learning, expecially for large scale problem. A common approach for this problem is to transfer knowledge from a related labeled domain to a target one. There are two popular ways to achieve this goal: adversarial learning and self training. In this article, …
Exploits class similarity for better machine learning models with confidence labels and projective loss functions.
There are a variety of Domain Adaptation (DA) scenarios subject to label sets and domain configurations, including closed-set and partial-set DA, as well as multi-source and multi-target DA. It is notable that existing DA methods are generally designed only for a specific scenario, and may underperform for scenarios th…
Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data scientists need to assess classifiers' performances, evaluate their learning behavior over time, and c…
EAST aligns neural network classifiers with user-defined evaluation metrics.
The article explains how to estimate confusion matrices for classifiers using unlabeled data.
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradient confusion is high, stochastic gradients produced by different data samples may be negatively correlated, slowing down convergence. But wh…
Optimizes classification algorithms with bounds on error rates.
Unified framework for comparing classification metrics across different imbalance rates.
Paper recovers top-two answers and confusion probability in multi-choice crowdsourcing.
We study consistency of learning algorithms for a multi-class performance metric that is a non-decomposable function of the confusion matrix of a classifier and cannot be expressed as a sum of losses on individual data points; examples of such performance metrics include the macro F-measure popular in information retri…
Proposes FACT, a diagnostic for understanding group fairness trade-offs.
InstanceFlow visualizes classifier confusion over training epochs.
In this work we addressed the issue of applying a stochastic classifier and a local, fuzzy confusion matrix under the framework of multi-label classification. We proposed a novel solution to the problem of correcting label pairwise ensembles. The main step of the correction procedure is to compute classifier-specific c…
This paper compares unstructured and structured EM-based semi-supervised learning methods.
We define a notion of semi-conjugacy between orientation-preserving actions of a group on the circle, which for fixed point free actions coincides with a classical definition of Ghys. We then show that two circle actions are semi-conjugate if and only if they have the same bounded Euler class. This settles some existin…
Proposes a new method for deep ensembles that improves accuracy and calibration.
GS-BSE improves label shift estimation by smoothing priors on a graph.
Unified view of label shift estimation methods.
Randomizing the Fourier-transform (FT) phases of temporal-spatial data generates surrogates that approximate examples from the data-generating distribution. We propose such FT surrogates as a novel tool to augment and analyze training of neural networks and explore the approach in the example of sleep-stage classificat…
CILF learns adaptive embeddings for class-incremental learning with novel class detection and model update.
Ideally, what confuses neural network should be confusing to humans. However, recent experiments have shown that small, imperceptible perturbations can change the network prediction. To address this gap in perception, we propose a novel approach for learning robust classifier. Our main idea is: adversarial examples for…
Traditionally, in supervised machine learning, (a significant) part of the available data (usually 50% to 80%) is used for training and the rest for validation. In many problems, however, the data is highly imbalanced in regard to different classes or does not have good coverage of the feasible data space which, in tur…
Machine learning classification limits estimated using Kullback-Leibler divergence and Cohen's Kappa.
New method quantifies classifier uncertainty, revealing large variability in performance metrics.
This paper investigates how machine learning APIs change over time and proposes an efficient method to monitor these changes.
Combines human and model predictions for improved accuracy.
New algorithm optimizes complex metrics in online learning.
In the last few years, many different performance measures have been introduced to overcome the weakness of the most natural metric, the Accuracy. Among them, Matthews Correlation Coefficient has recently gained popularity among researchers not only in machine learning but also in several application fields such as bio…
This paper visualizes uncertainty in classifier performance metrics.
This paper revisits the problem of analyzing multiple ratings given by different judges. Different from previous work that focuses on distilling the true labels from noisy crowdsourcing ratings, we emphasize gaining diagnostic insights into our in-house well-trained judges. We generalize the well-known DawidSkene model…
ViTaX provides formal guarantees for targeted explanations in safety-critical systems.
Consistent algorithms for multiclass learning with complex metrics and constraints.
Several recent deep neural networks experiments leverage the generalist-specialist paradigm for classification. However, no formal study compared the performance of different clustering algorithms for class assignment. In this paper we perform such a study, suggest slight modifications to the clustering procedures, and…
Context: Conducting experiments is central to research machine learning research to benchmark, evaluate and compare learning algorithms. Consequently it is important we conduct reliable, trustworthy experiments. Objective: We investigate the incidence of errors in a sample of machine learning experiments in the domain …
IRT metrics improve model evaluation by assessing latent characteristics.
WAPPO optimizes feature distributions for better visual transfer in RL.
Unified approach optimizes neural network training for various metrics.
Improved multiclass classification with class-weighted nearest neighbors.
Proposes TFDF to learn transferable and discriminative features for unsupervised domain adaptation.
We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditi…
The aim of this paper was to compare soft confusion matrix approach and Bayes metaclassifier under the multi-label classification framework. Although the methods were successfully applied under the multi-label classification framework, they have not been compared directly thus far. Such comparison is of vital importanc…
In this work, we addressed the issue of applying a stochastic classifier and a local, fuzzy confusion matrix under the framework of multi-label classification. We proposed a novel solution to the problem of correcting label pairwise ensembles. The main step of the correction procedure is to compute classifier- specific…
The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the "truth" under the influence of their varying skill-levels and biases. Blindly treating these noisy labels as the ground trut…
In the past decades, intensive efforts have been put to design various loss functions and metric forms for metric learning problem. These improvements have shown promising results when the test data is similar to the training data. However, the trained models often fail to produce reliable distances on the ambiguous te…