arXiv research

We propose a novel classifier accuracy metric: the Bayesian Area Under the Receiver Operating Characteristic Curve (CBAUC). The method estimates the area under the ROC curve and is related to the recently proposed Bayesian Error Estimator. The metric can assess the quality of a classifier using only the training datase…

The paper proposes a test to assess rater accuracy while accounting for rater covariates.

Many problems that appear in biomedical decision making, such as diagnosing disease and predicting response to treatment, can be expressed as binary classification problems. The costs of false positives and false negatives vary across application domains and receiver operating characteristic (ROC) curves provide a visu…

Study predicts antimicrobial resistance in ICU patients quickly.

Geometric analysis of ROC and PR curves for binary classification.

Machine learning methods may have the potential to significantly accelerate drug discovery. However, the increasing rate of new methodological approaches being published in the literature raises the fundamental question of how models should be benchmarked and validated. We reanalyze the data generated by a recently pub…

Receiver operating characteristic (ROC) analysis is widely used for evaluating diagnostic systems. Recent studies have shown that estimating an area under ROC curve (AUC) with standard cross-validation methods suffers from a large bias. The leave-pair-out (LPO) cross-validation has been shown to correct this bias. Howe…

New ROC tools assess predictive abilities for any linearly ordered outcomes.

LxCIM metric improves binary classification performance evaluation.

The Receiver Operating Characteristic (ROC) curve is a representation of the statistical information discovered in binary classification problems and is a key concept in machine learning and data science. This paper studies the statistical properties of ROC curves and its implication on model selection. We analyze the …

Proposes a new prior for VAEs to improve out-of-distribution detection.

A new method QMS22 for semi-supervised anomaly detection outperforms existing methods.

Unified four trade-off curves for assessing generative model proximity.

Modified AUC improves CNN training by considering model confidence.

When sufficient labeled data are available, classical criteria based on Receiver Operating Characteristic (ROC) or Precision-Recall (PR) curves can be used to compare the performance of un-supervised anomaly detection algorithms. However , in many situations, few or no data are labeled. This calls for alternative crite…

Deep learning has significant potential for medical imaging. However, since the incident rate of each disease varies widely, the frequency of classes in a medical image dataset is imbalanced, leading to poor accuracy for such infrequent classes. One possible solution is data augmentation of infrequent classes using syn…

We examine the supervised learning problem in its continuous setting and give a general optimality condition through techniques of functional analysis and the calculus of variations. This enables us to solve the optimality condition for the desired function u numerically and make several comparisons with other widely u…

Deep learning diagnoses MS from smartphone data.

In many healthcare settings, intuitive decision rules for risk stratification can help effective hospital resource allocation. This paper introduces a novel variant of decision tree algorithms that produces a chain of decisions, not a general tree. Our algorithm, $α$-Carving Decision Chain (ACDC), sequentially carves o…

Intensive care clinicians are presented with large quantities of patient information and measurements from a multitude of monitoring systems. The limited ability of humans to process such complex information hinders physicians to readily recognize and act on early signs of patient deterioration. We used machine learnin…

This paper aims to provide a better understanding of a symmetric loss. First, we emphasize that using a symmetric loss is advantageous in the balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization from corrupted labels. Second, we prove general theoretical p…

Imbalanced data sets containing much more background than signal instances are very common in particle physics, and will also be characteristic for the upcoming analyses of LHC data. Following up the work presented at ACAT 2008, we use the multivariate technique presented there (a rule growing algorithm with the meta-m…

Drug-drug interactions are preventable causes of medical injuries and often result in doctor and emergency room visits. Computational techniques can be used to predict potential drug-drug interactions. We approach the drug-drug interaction prediction problem as a link prediction problem and present two novel methods fo…

Electronic medical records (EMRs) supports the development of machine learning algorithms for predicting disease incidence, patient response to treatment, and other healthcare events. But insofar most algorithms have been centralized, taking little account of the decentralized, non-identically independently distributed…

Optimal cutoff interval for risk scores improves binary classification accuracy.

Machine learning improves early detection of patient deterioration in Brazilian hospitals.

Identifying patients who will be discharged within 24 hours can improve hospital resource management and quality of care. We studied this problem using eight years of Electronic Health Records (EHR) data from Stanford Hospital. We fit models to predict 24 hour discharge across the entire inpatient population. The best …

Paper discusses extending Gini score for tied rankings and case weights.

Paper extends RPD for better handling multiple modalities and non-convexity.

Neural network models improve ROC curve evaluation of biomarkers, focusing on age's role in physical activity-mortality association.

Area under the receiver operating characteristics curve (AUC) is an important metric for a wide range of signal processing and machine learning problems, and scalable methods for optimizing AUC have recently been proposed. However, handling very large datasets remains an open challenge for this problem. This paper prop…

Wide-band Electromagnetic Induction Sensors (WEMI) have been used for a number of years in subsurface detection of explosive hazards. While WEMI sensors have proven effective at localizing objects exhibiting large magnetic responses, detecting objects lacking or containing very low amounts of conductive materials can b…

New method assesses prediction intervals across different operating points.

Study evaluates data augmentation methods for prostate cancer detection in MRI.

This paper visualizes uncertainty in classifier performance metrics.

A common approach in positive-unlabeled learning is to train a classification model between labeled and unlabeled data. This strategy is in fact known to give an optimal classifier under mild conditions; however, it results in biased empirical estimates of the classifier performance. In this work, we show that the typi…

We consider a document classification problem where document labels are absent but only relevant keywords of a target class and unlabeled documents are given. Although heuristic methods based on pseudo-labeling have been considered, theoretical understanding of this problem has still been limited. Moreover, previous me…

Early detection of breast cancer can increase treatment efficiency. Architectural Distortion (AD) is a very subtle contraction of the breast tissue and may represent the earliest sign of cancer. Since it is very likely to be unnoticed by radiologists, several approaches have been proposed over the years but none using …

We present the qGaussian generalization of the Merton framework, which takes into account slow fluctuations of the volatility of the firms market value of financial assets. The minimal version of the model depends on the Tsallis entropic parameter q and the generalized distance to default. The empirical foundation and …

Proposes and evaluates three diagnostic graphics for probabilistic classifiers.

New algorithm optimizes AUC in binary classification and changepoint detection.

We investigate a long-debated question, which is how to create predictive models of recidivism that are sufficiently accurate, transparent, and interpretable to use for decision-making. This question is complicated as these models are used to support different decisions, from sentencing, to determining release on proba…

The paper develops a test for EU portfolio efficiency in high dimensions.

The consultative papers for the Basel II Accord require rating systems to provide a ranking of obligors in the sense that the rating categories indicate the creditworthiness in terms of default probabilities. As a consequence, the default probabilities ought to present a monotonous function of the ordered rating catego…

The fuzzy ROC extends Receiver Operating Curve (ROC) visualization to the situation where some data points, falling in an indeterminacy region, are not classified. It addresses two challenges: definition of sensitivity and specificity bounds under indeterminacy; and visual summarization of the large number of possibili…

New optimization method improves AUC for binary classification and changepoint detection.

Glioma grading before surgery is very critical for the prognosis prediction and treatment plan making. We present a novel wavelet scattering-based radiomic method to predict noninvasively and accurately the glioma grades. The method consists of wavelet scattering feature extraction, dimensionality reduction, and glioma…

Study predicts startup outcomes like funding, patenting, IPOs using machine learning.