The paper proposes a test to assess rater accuracy while accounting for rater covariates.
problem Assessing the accuracy of raters in medical imaging and forensic studies.
method Covariate-adjusted homogeneity test to determine differences in accuracy among multiple rater groups.
result The proposed test identifies statistically significant differences among five participant groups in a face recognition study.
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
The article explains how to estimate confusion matrices for classifiers using unlabeled data.
problem Estimating sensitivity and specificity of binary medical diagnostic tests without gold standard tests.
method Modifying diagnostic test solutions to estimate confusion matrices for classifiers on unlabeled data.
result The approach can be used to estimate accuracy statistics for supervised or unsupervised binary classifiers on unlabeled data.
TARP tests accuracy of generative posterior estimators.
problem Assessing the accuracy of posterior estimators from generative models.
method TARP coverage testing method.
result TARP can detect inaccurate inferences in high-dimensional spaces.
New test sets show current image classifiers struggle with harder images.
problem Current image classifiers overfit to original test sets.
method Created new test sets for CIFAR-10 and ImageNet, evaluated various models.
result Accuracy drops of 3% - 15% on CIFAR-10 and 11% - 14% on ImageNet.
Enhances image quality to improve test-time adaptation accuracy.
problem Reducing accuracy loss due to distribution shift in deep networks.
method Integrates image enhancement with TTA methods to reduce prediction uncertainty.
result TECA method increases accuracy of TTA methods without hyperparameters.
Test log-likelihood comparisons can be misleading.
problem Misinterpretation of test log-likelihood in model comparison.
method Simple examples of model comparison and forecast accuracy.
result Test log-likelihood does not always correlate with model accuracy.
Regularizing for or against class selectivity in DNNs improves test accuracy.
problem The necessity and sufficiency of class selectivity in DNNs.
method Direct regularization of class selectivity in convolutional neural networks.
result Reducing class selectivity improves test accuracy, while increasing it decreases it.
Deep learning fails in classifying handwritten historical documents, traditional methods perform better.
problem Classifying handwritten historical documents using deep learning methods.
method Traditional and deep learning methods were tested on a large collection of handwritten historical manuscripts.
result Deep learning methods performed poorly, traditional methods performed consistently well.
This paper removes near-duplicates from Fashion-MNIST to improve testing accuracy.
problem Near-duplicate images in Fashion-MNIST increase testing accuracy, reducing dataset quality.
method Identified and removed near-duplicate images between training and testing sets.
result Improved dataset quality for better testing accuracy in machine learning models.
SGD with large learning rates can achieve better test accuracy than expected.
problem SGD with large learning rates often outperforms expected convergence bounds.
method Proved that SGD with small learning rates stays close to gradient flow path on modified loss.
result Explicitly adding an implicit regularizer to the loss improves test accuracy.
Single tree outperforms random forest in testing accuracy.
problem The challenge of improving single decision tree performance.
method Gradient-based entire tree optimization framework, scaled sigmoid approximation, numerical stability algorithm, subtree polish strategy.
result Optimized single tree outperforms classic random forest by 2.03% on average.
A new neural network using chi-square test for binary classification.
problem Improving binary classification accuracy.
method Backpropagation neural network with chi-square test redefined cost and error functions.
result Significantly improved classification accuracy compared to related approaches.
FEAST improves FNN's range accuracy in ocean waveguides.
problem Evaluating FNN's range accuracy on unlabeled test data in ocean waveguides.
method Introducing FEAST to evaluate and improve FNN's range accuracy.
result FEAST helps improve FNN's ranging accuracy on test data.
New group testing method uses Belief Propagation for accurate screening.
problem Efficiently identifying infected samples in large groups with minimal tests.
method Belief Propagation algorithm for inference in group testing schemes.
result Significantly increased accuracy of infection identification with fewer tests.
Research proposes an ensemble learning model for efficient software defect prediction.
problem Efficient and cost-effective software testing to minimize project resources.
method Machine learning analysis on different datasets using KNN, Decision Tree, SVM, and Naïve Bayes.
result Ensemble learning model outperforms other techniques in accuracy, precision, recall, and F1-score.
Improved deep kernel machine achieves 94.5% test accuracy on CIFAR-10.
problem Achieving high accuracy on complex tasks like image classification.
method Stochastic kernel regularisation to add noise to learned Gram matrices.
result 94.5% test accuracy on CIFAR-10.
MU-SVM improves multiclass classification accuracy.
problem Multiclass classification problems.
method Proposes MU-SVM for multiclass learning and an analytic span bound for model selection.
result Achieves > 20% improvement in test accuracies compared to multi-class SVM.
Efficient deep learning for wireless source identification using test SNR estimates.
problem Training deep classifiers for wireless signal modulation and technology identification.
method Greedy training SNR Boosting and bootstrap aggregating (Bagging) based on training SNR values.
result Uniform improvement in accuracy across all SNR values with small subsets of training SNR values.
This paper presents a comparison of six machine learning (ML) algorithms: GRU-SVM (Agarap, 2017), Linear Regression, Multilayer Perceptron (MLP), Nearest Neighbor (NN) search, Softmax Regression, and Support Vector Machine (SVM) on the Wisconsin Diagnostic Breast Cancer (WDBC) dataset (Wolberg, Street, & Mangasarian, 1…
Optimizes test set size for accurate diagnosis using machine learning.
problem Determining the minimum test set size for accurate diagnosis.
method Proposes machine learning methods (LASSO and SVM) to predict optimal test set size.
result SVM achieves 90.4% accuracy with a reduced test set by 35.24%.
New theory predicts which large DNNs will have best test accuracy.
problem Predicting which large pre-trained DNNs will have the best test accuracy.
method Heavy-Tailed Self-Regularization (HT-SR) and Universal capacity control metric based on power law exponents.
result Universal capacity control metric correlates well with reported test accuracies of large-scale DNNs.
New measure assesses neural network models' functional similarity.
problem Measuring functional similarity between similar-performing neural networks.
method Robust nonparametric hypothesis testing framework.
result Proposed measure assesses neural networks' functional similarity.
Machine learning is currently dominated by largely experimental work focused on improvements in a few key tasks. However, the impressive accuracy numbers of the best performing models are questionable because the same test sets have been used to select these models for multiple years now. To understand the danger of ov…
A neural network-based two-sample test improves classification accuracy.
problem Differentiating between two sub-exponential densities.
method Difference of logit function from trained classification neural network.
result Network complexity scales with intrinsic dimensionality for low-dimensional manifolds.
Proposes a new method to improve Bayesian computation accuracy using flexible classification.
problem Bayesian computations accuracy check using rank-based simulation-based calibration has limitations.
method Replaces marginal rank test with a flexible classification approach that learns from data.
result Improves statistical power and provides an interpretable divergence measure of miscalibration.
Deep reinforcement learning improves medical test suggestions.
problem Improving accuracy of disease diagnosis through better medical test recommendations.
method Formulated as a stage-wise Markov decision process, trained using reinforcement learning with a new action policy representation and exploration scheme.
result Significant improvement in disease diagnosis accuracy with better medical test suggestions.
Pruning neural networks can improve test accuracy even with significant parameter reduction.
problem The tradeoff between generalization and stability in neural network pruning.
method Analysis of pruning behavior over training, focusing on instability and its relation to generalization.
result Pruning's benefit to generalization increases with its instability.
SliceOut speeds up deep learning training without sacrificing accuracy.
problem Frequent model re-training and large model training workloads in deep learning.
method SliceOut uses dropout-inspired scheme to drop contiguous sets of units at random, leveraging GPU memory layout.
result 10-40% speedups and memory reduction with minimal accuracy loss.
Efficiently selects best machine learning config from large dataset.
problem Finding the best machine learning configuration from a large dataset.
method CI-based progressive sampling and pruning strategy.
result Achieves more than two orders of magnitude speedup while maintaining similar accuracy.
Paper introduces MultiTargeted testing, improving adversarial testing efficiency.
problem Improving efficiency and effectiveness of adversarial testing methods.
method Introduces MultiTargeted testing, using alternative surrogate losses.
result MultiTargeted outperforms other PGD-based methods, requiring fewer iterations.
New method improves prediction accuracy by using future test points.
problem Improving prediction accuracy by leveraging future test points.
method Transductive inference techniques from semi-parametric inference.
result Non-asymptotic upper bounds on transductive prediction errors.
Study on ensemble, distillation, and self-distillation in deep learning models.
problem Improving test accuracy in deep learning models using ensemble and distillation methods.
method Formal study of ensemble and distillation, considering multi-view data structure.
result Proven that ensemble and distillation can improve test accuracy in deep learning models, and the superior performance can be distilled into a single model.
Statistical test detects model degradations in optimized language models.
problem Detecting model degradations in optimized language models.
method Statistical hypothesis testing framework based on McNemar's test.
result Even small accuracy degradations (0.3%) can be attributed to actual degradations, not noise.
FL's early training phase significantly impacts final test accuracy.
problem Understanding how early phases affect FL's final test accuracy.
method Generalized Fisher Information Matrix (FedFIM) to FL.
result FL exhibits critical learning periods where small errors can have large impacts.
New research shows no trade-off between fairness and accuracy in machine learning.
problem The trade-off between fairness and accuracy in machine learning is a widely accepted belief.
method Using mismatched hypothesis testing and Chernoff information, the study demonstrates that optimal fairness and accuracy can be achieved simultaneously.
result There is no inherent trade-off between fairness and accuracy in ideal distributions, but it exists when measured with respect to biased datasets.
Mixed-size training improves CNN accuracy and speed.
problem Training CNNs on fixed image sizes limits their adaptability to various image sizes.
method Mixed-size training: training on multiple image sizes at once.
result Models trained with mixed-size images achieve higher accuracy and faster inference.
In this paper we propose strategies for estimating performance of a classifier when labels cannot be obtained for the whole test set. The number of test instances which can be labeled is very small compared to the whole test data size. The goal then is to obtain a precise estimate of classifier performance using as lit…
Paper shows adversarial training can be fooled by new type of noise.
problem Adversarial training can be fooled by new types of noise.
method Designing ADVIN, a new type of inducing noise.
result ADVIN can degrade adversarial training robustness by 99.9%.
Automated testing improves deep learning model accuracy by 259.2%.
problem Ensuring robustness of deep learning models through automated testing.
method Jointly optimizing differential behaviors and neuron coverage; generating corner-cases; applying transformations and GANs.
result Deep learning model accuracy increased by 259.2% using automated generated corner cases.
In this study, a novel sparsity-driven weighted ensemble classifier (SDWEC) that improves classification accuracy and minimizes the number of classifiers is proposed. Using pre-trained classifiers, an ensemble in which base classifiers votes according to assigned weights is formed. These assigned weights directly affec…
A test confirms if a local maximum is globally optimal in imaging problems.
problem Confirming if a local maximum is globally optimal in imaging problems.
method Reparameterizing the likelihood function to embed its domain into a higher dimensional parameter space.
result Improved accuracy and reduced computation for camera-blur estimation.
Paper improves power of conditional randomization tests.
problem Improving power of conditional randomization tests.
method Introducing a new cost function to maximize test statistic power.
result Consistently increases the number of correct discoveries.
FastForest boosts Random Forest speed by 24%.
problem Efficiency in processing speed for Random Forest.
method Subsample Aggregating, Logarithmic Split-Point Sampling, Dynamic Restricted Subspacing.
result Average 24% increase in processing speed with accuracy maintained.
Recent works on adversarial perturbations show that there is an inherent trade-off between standard test accuracy and adversarial accuracy. Specifically, they show that no classifier can simultaneously be robust to adversarial perturbations and achieve high standard test accuracy. However, this is contrary to the stand…
A new test evaluates risk estimation accuracy using probability integral transform.
problem Measuring the accuracy of financial market risk estimations.
method Probability Integral Transform (PIT) of ex post realized returns against ex ante probability distributions.
result The new test shows the importance of capturing the dynamic of financial markets.
Usually one compares the accuracy of two competing classifiers via null hypothesis significance tests (nhst). Yet the nhst tests suffer from important shortcomings, which can be overcome by switching to Bayesian hypothesis testing. We propose a Bayesian hierarchical model which jointly analyzes the cross-validation res…
Enhances k-NN accuracy through randomized hyperstructure.
problem Improves k-NN accuracy by optimizing neighbor selection.
method Constructs a random n-dimensional hyperstructure around test instances to refine neighbor selection.
result 85.71% accuracy on Haberman's Cancer Survival dataset, compared to 80.95% for conventional k-NN.