A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper recovers top-two answers and confusion probability in multi-choice crowdsourcing.
We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditi…
New method quantifies classifier uncertainty, revealing large variability in performance metrics.
The article explains how to estimate confusion matrices for classifiers using unlabeled data.
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradient confusion is high, stochastic gradients produced by different data samples may be negatively correlated, slowing down convergence. But wh…
Optimizes classification algorithms with bounds on error rates.
A widespread approach in machine learning to evaluate the quality of a classifier is to cross -- classify predicted and actual decision classes in a confusion matrix, also called error matrix. A classification tool which does not assume distributional parameters but only information contained in the data is based on th…
PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.
Retirees who exhaust their savings while still alive are said to experience financial ruin. These savings are typically grown during the accumulation phase then spent during the retirement decumulation phase. Extensive research into invest-and-harvest decumulation strategies has been conducted, but recommendations diff…
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
Data collection and annotation are time-consuming in machine learning, expecially for large scale problem. A common approach for this problem is to transfer knowledge from a related labeled domain to a target one. There are two popular ways to achieve this goal: adversarial learning and self training. In this article, …
Machine learning classification limits estimated using Kullback-Leibler divergence and Cohen's Kappa.
In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with perfo…
Combines human and model predictions for improved accuracy.
This paper visualizes uncertainty in classifier performance metrics.
Proposes a new method for deep ensembles that improves accuracy and calibration.
Context: Conducting experiments is central to research machine learning research to benchmark, evaluate and compare learning algorithms. Consequently it is important we conduct reliable, trustworthy experiments. Objective: We investigate the incidence of errors in a sample of machine learning experiments in the domain …
Ideally, what confuses neural network should be confusing to humans. However, recent experiments have shown that small, imperceptible perturbations can change the network prediction. To address this gap in perception, we propose a novel approach for learning robust classifier. Our main idea is: adversarial examples for…
Robust variable selection for high-dimensional data with missing and measurement errors.
Randomizing the Fourier-transform (FT) phases of temporal-spatial data generates surrogates that approximate examples from the data-generating distribution. We propose such FT surrogates as a novel tool to augment and analyze training of neural networks and explore the approach in the example of sleep-stage classificat…
This paper investigates how machine learning APIs change over time and proposes an efficient method to monitor these changes.
There are a variety of Domain Adaptation (DA) scenarios subject to label sets and domain configurations, including closed-set and partial-set DA, as well as multi-source and multi-target DA. It is notable that existing DA methods are generally designed only for a specific scenario, and may underperform for scenarios th…
EAST aligns neural network classifiers with user-defined evaluation metrics.
This paper revisits the problem of analyzing multiple ratings given by different judges. Different from previous work that focuses on distilling the true labels from noisy crowdsourcing ratings, we emphasize gaining diagnostic insights into our in-house well-trained judges. We generalize the well-known DawidSkene model…
Consistent algorithms for multiclass learning with complex metrics and constraints.
IRT metrics improve model evaluation by assessing latent characteristics.
Proposes FACT, a diagnostic for understanding group fairness trade-offs.
This paper introduces a high frequency trade execution model to evaluate the economic impact of supervised machine learners. Extending the concept of a confusion matrix, we present a 'trade information matrix' to attribute the expected profit and loss of the high frequency strategy under execution constraints, such as …
WAPPO optimizes feature distributions for better visual transfer in RL.
Exploits class similarity for better machine learning models with confidence labels and projective loss functions.
Probabilistic models can handle causal inference without special tools.
InstanceFlow visualizes classifier confusion over training epochs.
Unified approach optimizes neural network training for various metrics.
In this work we addressed the issue of applying a stochastic classifier and a local, fuzzy confusion matrix under the framework of multi-label classification. We proposed a novel solution to the problem of correcting label pairwise ensembles. The main step of the correction procedure is to compute classifier-specific c…
Unified framework for comparing classification metrics across different imbalance rates.
Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data scientists need to assess classifiers' performances, evaluate their learning behavior over time, and c…
New PAC-Bayes bound controls multiple error types simultaneously.
In this paper, an issue of building the RRC model using probability distributions other than beta distribution is addressed. More precisely, in this paper, we propose to build the RRR model using the truncated normal distribution. Heuristic procedures for expected value and the variance of the truncated-normal distribu…
The aim of this paper was to compare soft confusion matrix approach and Bayes metaclassifier under the multi-label classification framework. Although the methods were successfully applied under the multi-label classification framework, they have not been compared directly thus far. Such comparison is of vital importanc…
In this work, we addressed the issue of applying a stochastic classifier and a local, fuzzy confusion matrix under the framework of multi-label classification. We proposed a novel solution to the problem of correcting label pairwise ensembles. The main step of the correction procedure is to compute classifier- specific…
Algorithm constructs prediction sets with PAC guarantees in label shift settings.
The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the "truth" under the influence of their varying skill-levels and biases. Blindly treating these noisy labels as the ground trut…
The study improves life insurance surrender risk modeling using various machine learning techniques.
In the past decades, intensive efforts have been put to design various loss functions and metric forms for metric learning problem. These improvements have shown promising results when the test data is similar to the training data. However, the trained models often fail to produce reliable distances on the ambiguous te…
We present a new method for computing ASR word confidences that effectively mitigates the effect of ASR errors for diverse downstream applications, improves the word error rate of the 1-best result, and allows better comparison of scores across different models. We propose 1) a new method for modeling word confidence u…
Paper proposes contrastive training to improve OOD detection without needing explicit OOD examples.
The St. Petersburg Paradox, an important topic in probability theory, has not been solved in the last 280 years. Since Nicolaus Bernoulli proposed the St. Petersburg Paradox in 1738, many people had tried to solve it and had proposed various explanations, but all were not satisfactory. In this paper we propose a new pr…
Algorithm identifies the best arm in linear bandits with high probability.