Develops rough set classifiers using confusion matrices.
problem Evaluating classifier quality in machine learning.
method Combines rough set theory with confusion matrices.
result Defines indices and classifiers based on rough confusion matrices.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “Confusability Index”
The paper investigates how neural network width and depth impact training speed through gradient confusion.
problem Gradient confusion in neural networks during training.
method Formal analysis using gradient confusion, theoretical and experimental results.
result Increasing network width reduces gradient confusion, leading to faster training.
The article explains how to estimate confusion matrices for classifiers using unlabeled data.
problem Estimating sensitivity and specificity of binary medical diagnostic tests without gold standard tests.
method Modifying diagnostic test solutions to estimate confusion matrices for classifiers on unlabeled data.
result The approach can be used to estimate accuracy statistics for supervised or unsupervised binary classifiers on unlabeled data.
DCSE combines domain confusion and self-ensembling for unsupervised adaptation.
problem Unsupervised domain adaptation with time-consuming data collection and annotation.
method DCSE combines domain confusion and self-ensembling to learn invariant representations.
result DCSE outperforms existing methods in various unsupervised domain adaptation benchmarks.
PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.
problem Inaccurate and insufficient confusion matrices for annotators in multi-class classification tasks.
method PTBCC (ProtoType learning-driven Bayesian Classifier Combination) uses prototype confusion matrices to capture annotator expertise.
result PTBCC achieves up to 15% accuracy improvement and 3% higher average accuracy compared to existing methods.
Paper recovers top-two answers and confusion probability in multi-choice crowdsourcing.
problem Recovering top-two answers and confusion probability in multi-choice crowdsourcing tasks.
method Proposes a two-stage inference algorithm based on a model quantifying task difficulty and worker reliability.
result Achieves minimax optimal convergence rate and outperforms other algorithms in synthetic and real data experiments.
Paper proposes MCC to reduce class confusion for versatile DA.
problem Class confusion in DA methods limits their performance across different scenarios.
method Introduces Minimum Class Confusion (MCC) loss function to handle various DA scenarios.
result MCC significantly improves performance on diverse DA scenarios, including Multi-Source and Multi-Target DA.
Study uses spoofing countermeasures to assess speech processing artifacts in voice conversion.
problem Difficulty in objectively assessing speech processing artifacts in voice conversion.
method Configured a constant-Q cepstral coefficient (CQC) model to measure artifact extent.
result Identified two clusters of VCC'18 entries: low-quality with detectable artifacts and higher quality with less artifacts.
In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with perfo…
Bayes metaclassifier and Soft-confusion-matrix classifier in the task of multi-label classificationcs.LG
Paper compares two multi-label classification methods: Bayes metaclassifier and soft-confusion-matrix.
problem Comparing multi-label classification methods under the same framework.
method Bayes metaclassifier combined with problem-transformation approach, and soft-confusion-matrix classifier.
result Both methods perform similarly, but statistical analysis provides insights.
ConfusionFlow visualizes classifier confusion over time for model comparison.
problem Insufficient performance analysis of classifiers.
method Interactive, model-agnostic visualization tool combining confusion matrices and temporal analysis.
result ConfusionFlow facilitates detailed, comparative analysis of classifier performance over time.
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
Method learns true labels from noisy annotators using regularization.
problem Learning from noisy labels in supervised learning.
method Regularized estimation of annotator confusion matrices.
result Method outperforms state-of-the-art methods in image classification.
New method quantifies classifier uncertainty, revealing large variability in performance metrics.
problem Uncertainty in classifier performance metrics due to small data sets.
method Probability model of the confusion matrix to quantify uncertainty.
result Large uncertainties in classification performance metrics can lead to misleading conclusions.
Clarifies confusion on feature relevance quantification in explainable AI.
problem Confusion between observational and interventional conditional probabilities in feature relevance quantification.
method Uses Shapley values and clarifies the distinction between observational and interventional conditional probabilities based on Pearl's causality theory.
result Unconditional expectations are the right notion for dropping features, contradicting theoretical justification of SHAP.
This paper investigates how machine learning APIs change over time and proposes an efficient method to monitor these changes.
problem Understanding and assessing changes in machine learning APIs over time.
method Systematic investigation of ML API shifts, proposing a principled adaptive sampling algorithm (MASA) for efficient estimation of confusion matrix shifts.
result MASA can accurately estimate confusion matrix shifts using up to 90% fewer samples compared to random sampling.
Aims to make neural networks robust to small, imperceptible perturbations.
problem Neural networks can be fooled by small, imperceptible changes.
method Proposes a Generative Adversarial Networks (GAN) framework to learn robust classifiers.
result Our method outperforms adversarial training in robustness.
A new weighting scheme corrects label ensembles in multi-label classification.
problem Improving the reliability of multi-label classification with imbalanced data.
method Proposed a novel weighting scheme based on fuzzy confusion matrix and information theory.
result The proposed method reduces the vulnerability to imbalanced class distribution and improves classification quality.
Machine learning experiments often contain errors, especially in confusion matrices and statistical tests.
problem Errors in machine learning experiments, particularly in confusion matrices and statistical tests.
method Analyzed 49 papers describing 2456 experiments, checking for errors in confusion matrices and statistical significance.
result 22 out of 49 papers contain demonstrable errors, with 7 statistical and 16 related to confusion matrix inconsistency.
EAST aligns neural network classifiers with user-defined evaluation metrics.
problem Mismatch between neural network training and evaluation metrics leads to suboptimal performance.
method EAST uses dynamic thresholding, soft-set confusion matrix, and annealing to align neural network predictions with target evaluation metrics.
result EAST improves alignment between training objectives and evaluation metrics, outperforming existing methods.
This paper revisits the problem of analyzing multiple ratings given by different judges. Different from previous work that focuses on distilling the true labels from noisy crowdsourcing ratings, we emphasize gaining diagnostic insights into our in-house well-trained judges. We generalize the well-known DawidSkene model…
Consistent algorithms for multiclass learning with complex metrics and constraints.
problem Learning with complex performance metrics and constraints.
method General framework for designing consistent algorithms by viewing the problem as an optimization over feasible confusion matrices.
result Rates of convergence to the optimal (feasible) classifier, showing asymptotic consistency.
IRT metrics improve model evaluation by assessing latent characteristics.
problem Limitations of classic metrics like precision and F1.
method Introducing psychometric metrics like Item Response Theory (IRT).
result IRT complements classical metrics, offering new insights.
Proposes FACT, a diagnostic for understanding group fairness trade-offs.
problem Group fairness notions often conflict with each other, requiring a cost in model performance.
method Characterizes trade-offs via the fairness-confusion tensor and optimizes accuracy and fairness objectives.
result Demonstrates the use of FACT on synthetic and real datasets to understand accuracy-fairness trade-offs.
WAPPO optimizes feature distributions for better visual transfer in RL.
problem Improving visual transfer in reinforcement learning.
method WAPPO uses Wasserstein Confusion to minimize feature distribution distance.
result WAPPO outperforms previous methods in visual transfer across different environments.
Exploits class similarity for better machine learning models with confidence labels and projective loss functions.
problem Poor model performance due to confusing similar classes.
method Exploits class similarity with confidence labels and projective loss functions.
result Improved model performance on noisy labels.
InstanceFlow visualizes classifier confusion over training epochs.
problem Limited model interpretability through aggregate performance measures.
method Dual-view visualization tool showing instance-level learning behavior.
result Allows temporal analysis of training process and instance-level performance.
Unified approach optimizes neural network training for various metrics.
problem Training and evaluation of neural network binary classifiers often use different metrics.
method Combines differentiable approximation and probabilistic soft sets.
result Effective in optimizing for metrics like F1-Score across various domains.
Optimizes classification algorithms with bounds on error rates.
problem Bounding uncertainties in classifier outputs for diagnostic testing.
method Set-theoretic and probabilistic arguments to derive uniform error bounds.
result Optimal partition minimizes the largest Gershgorin radius of the confusion matrix.
Unified framework for comparing classification metrics across different imbalance rates.
problem Differences in scale and sensitivity to class imbalance rates in classification metrics.
method Introduces outperformance standardization (OPS) function to map metrics to a common scale.
result Unified o-value metric provides clear comparison across different imbalance rates.
New correlation measures improve classifier performance assessment.
problem Improving assessment of classifiers and raters.
method Introducing CO-, ANTI-, and COANTI-correlation coefficients.
result Demonstrated new measures are powerful for classifying confusion matrices.
Elicit performance metrics from classifier comparisons.
problem Discover the performance metric a practitioner prefers for binary classification.
method Formalize and exploit geometric properties of confusion matrices for efficient metric elicitation.
result Provably efficient algorithms for eliciting linear and linear-fractional metrics from pairwise feedback.
Acoustic Neighbor Embeddingseess.AS
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
problem Mapping speech and text to fixed dimensions for phonetic confusability.
method Adapting SNE to sequential inputs, training two encoder neural networks.
result More accurate results with low-dimensional embeddings in word recognition tasks.
AML addresses distribution bias in metric learning.
problem Distribution bias between training and test sets.
method Adversarial pairs generation and learning.
result AML significantly improves discriminability.
A method to correct binary classifier errors in multi-label pairwise models.
problem Improving the accuracy of binary classifiers in multi-label pairwise models.
method Computing competence and cross-competence measures to estimate and correct label errors.
result The proposed correction methods significantly outperform the reference method in terms of zero-one loss.
Paper tackles noisy reinforcement learning with perturbed rewards, improving agent performance.
problem Noisy rewards in reinforcement learning scenarios, leading to unreliable model performance.
method Develops a robust RL framework using a confusion matrix to estimate unbiased surrogate rewards.
result Trained policies using estimated surrogate rewards achieve higher expected rewards and faster convergence.
The study finds that firm membership in flagship indices and TCFD endorsement are strong predictors of a wider Disclosure-Performance Gap.
problem The Aggregate Confusion hypothesis and the measurement of greenwashing in environmental disclosures.
method The study uses a Disclosure-Performance Gap (DPG) model to measure the divergence between voluntary environmental disclosures and realised emissions performance for 200 large European firms. The model selection process involved multiple stages and robust standard errors.
result Firm membership in flagship indices and TCFD endorsement are strong predictors of a wider gap, while renewable energy use and environmental capital expenditure significantly narrow the gap.
The study explores Hesse manifolds and their symmetries in multifield cosmological models.
problem Understanding symmetries in multifield cosmological models.
method Analyzes Hesse functions and their properties on Riemannian manifolds.
result Complete Hesse manifolds are characterized by their index and are hyperbolic.
Paper introduces a new model to assess machine learning strategies in high-frequency trading.
problem Evaluating the economic impact of supervised machine learning in high-frequency trading.
method Developed a 'trade information matrix' to attribute profit and loss to correct and incorrect predictions under execution constraints.
result Demonstrated an estimation approach for measuring the sensitivity of P&L to prediction error in a market making strategy.
New method improves ASR word confidence for diverse applications.
problem Mitigating ASR errors and improving word error rate.
method Heterogeneous Word Confusion Network (HWCN) with score calibration.
result Word sequence with best overall confidence is more accurate than 1-best result.
This paper visualizes uncertainty in classifier performance metrics.
problem Overemphasis on model performance metrics risks overlooking uncertainty.
method Developed visualizations of confusion matrix metric distributions.
result Uncertainty in performance metrics can overshadow model differences.
Research tackles investor confusion in ESG rankings, offering tailored strategies.
problem Widespread confusion among investors regarding ESG rankings.
method Developed ESG ensemble strategies, integrated ESG scores into RL model, proposed Double-Mean-Variance model, introduced ESG-adjusted CAPMs.
result Optimized portfolios that balance financial returns and ESG-focused outcomes.
DPTA improves CIL by adapting PTMs with dual prototypes.
problem Catastrophic forgetting in incremental learning with pre-trained models.
method Dual-Prototype Network with Task-wise Adaptation (DPTA).
result DPTA consistently outperforms recent methods by 1\%-5\% on multiple benchmarks.
Paper proposes using truncated normal distribution for RRC model, improving detection of minority classes.
problem Improving weak classifiers in RRC models.
method Proposes using truncated normal distribution and soft confusion matrix for RRC model.
result Truncated-normal-based SCM algorithm outperforms beta distribution in discovering minority classes.
Probabilistic analysis reveals substantial losses for reverse convertible note holders.
problem Substantial losses to reverse convertible note holders due to complex pricing.
method Probabilistic analysis using Law of Total Expectation.
result Note-holders likely suffered substantial losses under various market scenarios.
Combines human and model predictions for improved accuracy.
problem Improving classification accuracy when both human and model predictions are imperfect.
method Uses confusion matrices and calibration to combine probabilistic model outputs with human class-level predictions.
result Human-model combinations consistently outperform either alone, with accuracy gains even with limited human input.
New algorithm optimizes complex metrics in online learning.
problem Optimizing non-decomposable metrics in sequential learning.
method General online algorithm for various metrics.
result Achieves regret for concave and smooth metrics.
We study consistency of learning algorithms for a multi-class performance metric that is a non-decomposable function of the confusion matrix of a classifier and cannot be expressed as a sum of losses on individual data points; examples of such performance metrics include the macro F-measure popular in information retri…