ELM detects mislabels in Finnish academic publication ranks.
problem Detecting mislabeled academic publication ranks in Finland.
method Used Extreme Learning Machine (ELM) with features characterizing publication channels.
result ELM-based approach accurately detected mislabels compared to reference results.
Improves fault detection models in noisy data.
problem Poor generalization due to mislabeled samples in fault detection.
method Two-step framework: outlier identification and data modification.
result Significantly improved model generalization under label noise.
The paper proposes a method to detect and filter noisy or mislabeled data using pointwise mutual information.
problem Detecting and filtering noisy or mislabeled data in deep learning models.
method A mutual information-based framework quantifying statistical dependencies between inputs and labels.
result The method effectively filters low-quality samples, improving classification accuracy by up to 15%.
Do-AIQ framework evaluates AI algorithms' quality using DOE.
problem Quality evaluation of AI mislabel detection algorithms.
method Design-of-experiment approach with high-dimensional constraint space design and surrogate modeling.
result Established framework for evaluating AI algorithm quality robustly.
The paper analyzes methods to identify influential data points in deep models.
problem Interpreting deep learning models and debugging datasets.
method Curated experiments to analyze influence of data points on classifiers.
result Training loss-based sample selection outperformed other methods in detecting mislabels.
Paper identifies and removes mislabeled data to improve neural network training.
problem Improving neural network training by identifying and removing ambiguous or mislabeled data.
method Introduces Area Under the Margin (AUM) statistic to identify mislabeled data and a simple procedure to remove it.
result Consistently improves test error on synthetic and real-world datasets.
Unified and noise-reduced data valuation framework for machine learning.
problem Quantifying the contribution of individual data points in machine learning.
method Beta Shapley, a generalization of Data Shapley, relaxes the efficiency axiom.
result Beta Shapley outperforms state-of-the-art data valuation methods on various ML tasks.
Hölder-DPO aligns models robustly with noisy human feedback.
problem No existing alignment methods can handle severe label noise.
method Proposes Hölder-DPO, a principled alignment loss with provable redescending property.
result Hölder-DPO enables scalable human feedback valuation and improves model alignment.
We investigate the problem of machine learning with mislabeled training data. We try to make the effects of mislabeled training better understood through analysis of the basic model and equations that characterize the problem. This includes results about the ability of the noisy model to make the same decisions as the …
Improves neural network performance with double regularization.
problem Challenges of mislabeled data in neural networks.
method Double regularization combining model complexity and observation reweighting.
result Stronger robustness and generalization against mislabeling.
Improved KNN data valuation method with reduced computation time.
problem Efficiently valuing individual data points in KNN models.
method Proposed a new utility function and derived its calculation for KNN classifiers/regressors, achieving similar time complexity as the original method.
result Soft-label KNN-SV outperforms the original method in mislabeled data detection.
A key requirement for supervised machine learning is labeled training data, which is created by annotating unlabeled data with the appropriate class. Because this process can in many cases not be done by machines, labeling needs to be performed by human domain experts. This process tends to be expensive both in time an…
Efficient active learning method defends against malicious mislabeling and data poisoning attacks.
problem Malicious mislabeling and data poisoning attacks on deep neural networks.
method Adversarial retraining and active learning with random sampling strategy.
result The proposed method achieves 89% accuracy with only one-third of the labeled data, significantly outperforming the baseline method.
Paper presents robust clustering methods for general mixture models.
problem Clustering with sub-Gaussian error assumptions often invalid in practice.
method Hybrid clustering with robust centroid estimate and data-driven initialization.
result Provably near-optimal mislabeling guarantees for general error distributions.
New attack tricks certifiably robust models into mislabeling images.
problem Defeating certified defenses against adversarial examples.
method Spoofed robustness certificates and large perturbations.
result Certifiably robust models can be fooled by large perturbations.
Paper presents attacks on real-time object detection systems.
problem Adversarial attacks on real-time object detection systems.
method Three targeted adversarial Objectness Gradient attacks (TOG).
result Adversarial attacks can cause object-vanishing, object-fabrication, and object-mislabeling.
We introduce a novel algorithm for the detection of possible sample corruption such as mislabeled samples in a training dataset given a small clean validation set. We use a set of inclusion variables which determine whether or not any element of the noisy training set should be included in the training of a network. We…
TimeInf estimates data contribution in time series data, improving model performance and anomaly detection.
problem Estimating data contribution in time series datasets with temporal dependencies.
method Model-agnostic data contribution estimation method using influence scores.
result TimeInf effectively detects time series anomalies and outperforms existing methods.
New method improves few-shot learning with noisy labels.
problem Robustness to label noise in few-shot learning.
method Feature aggregation and Transformer model for noisy samples.
result TraNFS outperforms other methods in noisy conditions.
There is an especially strong need in modern large-scale data analysis to prioritize samples for manual inspection. For example, the inspection could target important mislabeled samples or key vulnerabilities exploitable by an adversarial attack. In order to solve the "needle in the haystack" problem of which samples t…
The positive-unlabeled (PU) classification is a common scenario in real-world applications such as healthcare, text classification, and bioinformatics, in which we only observe a few samples labeled as "positive" together with a large volume of "unlabeled" samples that may contain both positive and negative samples. Bu…
Recently, a backdoor data poisoning attack was proposed, which adds mislabeled examples to the training set, with an embedded backdoor pattern, aiming to have the classifier learn to classify to a target class whenever the backdoor pattern is present in a test sample. Here, we address post-training detection of innocuo…
Noisy PN learning is the problem of binary classification when training examples may be mislabeled (flipped) uniformly with noise rate rho1 for positive examples and rho0 for negative examples. We propose Rank Pruning (RP) to solve noisy PN learning and the open problem of estimating the noise rates, i.e. the fraction …
CNNs improve transient detection in DES-SN images.
problem Automated identification of transients in astronomical images.
method Convolutional Neural Networks (CNNs) for image classification.
result CNNs achieve efficiency in identifying non-artifacts from artifacts.
DVGS identifies low-quality data quickly and accurately.
problem Identifying and filtering mislabeled or noisy data in machine learning.
method Data Valuation with Gradient Similarity (DVGS) algorithm.
result DVGS outperforms baseline methods in identifying low-value data across various domains.
Proposes a new loss function for better handling mislabeling costs.
problem Handling mislabeling costs in machine learning models.
method Introduces Real-World-Weight Crossentropy loss function.
result Demonstrates improved performance in scenarios of mislabeling.
Large datasets often have unreliable labels-such as those obtained from Amazon's Mechanical Turk or social media platforms-and classifiers trained on mislabeled datasets often exhibit poor performance. We present a simple, effective technique for accounting for label noise when training deep neural networks. We augment…
Machine Learning (ML) has seen massive progress in the last decade and as a result, there is a pressing need for validating ML-based systems. To this end, we propose, design and evaluate CALLISTO - a novel test generation and data quality assessment framework. To the best of our knowledge, CALLISTO is the first blackbo…
Efficiently approximates neural network function space distance.
problem Estimating the average discrepancy between neural network outputs.
method Linearized Activation Function TRick (LAFTR) for ReLU networks.
result Parametric approximation outperforms nonparametric methods in memory and accuracy.
In this paper we introduce a significant improvement to the popular tree-based Stochastic Gradient Boosting algorithm using a wavelet decomposition of the trees. This approach is based on harmonic analysis and approximation theoretical elements, and as we show through extensive experimentation, our wavelet based method…
One-bit quantization and sparsification improve multiclass classification with strong regularization.
problem Overfitting mislabeled data in multiclass classification.
method Linear regression with regularization and one-bit quantization/sparsification.
result Sparse and one-bit solutions perform almost as well as the optimal solution with f(⋅)=∥⋅∥22. The study analyzes how deep neural networks treat instances with regular and irregular patterns.
problem Understanding how deep neural networks handle both common and rare patterns in data.
method Characterizes instances using a consistency score based on training data sets of varying sizes.
result The consistency score identifies out-of-distribution and mislabeled examples, distinguishing them from strongly regular examples.
New generalization concept considers distribution of errors, not just average error.
problem Classical generalization fails to capture distributional differences in classifier outputs.
method Formal conjectures about distributional generalization based on model architecture, training procedure, and data distribution.
result Distributional generalization can be expected in specific conditions, as evidenced by empirical results.
SAP corrects model for label noise by identifying and removing noisy samples.
problem Label corruption degrades model performance; acquiring perfect labels is costly.
method SAP uses SVD to identify and project model weights onto a clean activation space.
result SAP improves model generalization by up to 6% on CIFAR dataset with 25% synthetic corruption.
Adapts AUM to identify ambiguous tasks in crowdsourced learning, improving generalization.
problem Discerning ambiguous tasks in crowdsourced labels to prevent mislabeling.
method Introduces Weighted Areas Under the Margin (WAUM) to average AUMs weighted by task-specific scores.
result Improves generalization performance by discarding ambiguous tasks.
Study shows influence functions are poor for neural networks but useful for identifying influential examples.
problem Influence functions misalign with leave-one-out retraining in neural networks.
method Decomposed the discrepancy into five terms and studied their contributions across different architectures and datasets.
result Influence functions are a good approximation to the proximal Bregman response function (PBRF), useful for identifying influential examples.
Self-training algorithm improves classifier performance with labeled and unlabeled data.
problem Improving classifier performance with limited labeled data.
method Iterative learning of halfspaces, exploration and pruning phases.
result Misclassification error is bounded and never degrades compared to initial labeled set.
Class labels are often imperfectly observed, due to mistakes and to genuine ambiguity among classes. We propose a new semi-supervised deep generative model that explicitly models noisy labels, called the Mislabeled VAE (M-VAE). The M-VAE can perform better than existing deep generative models which do not account for l…
DataInf efficiently approximates data influence in large models, improving transparency and identifying mislabeled data.
problem Efficiently estimating data influence in large-scale models like LoRA-tuned LLMs and diffusion models.
method DataInf uses a closed-form expression to approximate influence scores efficiently.
result DataInf outperforms existing methods in computational and memory efficiency, accurately identifying influential data points.
A method to approximate instance-dependent label noise using instance-confidence embedding.
problem Real-world label noise that depends on individual instances.
method Variational approximation with instance embedding to capture instance-specific label corruption.
result ICE method effectively approximates instance-dependent noise and detects ambiguous instances.
Develops new methods to evaluate data influence in SAM for improved model training.
problem Challenges in mislabeled noisy data and privacy concerns in SAM.
method Two innovative data valuation methods based on influence functions (IF) for SAM.
result Demonstrates effectiveness in identifying mislabeled data and enhancing interpretability.
Simple k-NN filtering improves model accuracy on noisy labels.
problem Training models with noisy labels reduces performance and is hard to identify.
method A simple k-nearest neighbor-based filtering approach on the logit layer. result Improves model accuracy compared to recent methods.
New method estimates data influence efficiently by leveraging test samples.
problem Efficiently estimating influence of training data on model predictions.
method Mirrored Influence Hypothesis, forward pass for test samples.
result Significant improvement in efficiency over existing methods.
PUMA augments models to remove unique data points without performance loss.
problem Preserving model performance while removing unique training data points.
method Explicitly models data influence, reweights remaining data optimally.
result PUMA effectively removes unique data points without performance degradation.
Boosting is known to be sensitive to label noise. We studied two approaches to improve AdaBoost's robustness against labelling errors. One is to employ a label-noise robust classifier as a base learner, while the other is to modify the AdaBoost algorithm to be more robust. Empirical evaluation shows that a committee of…
EP algorithm for efficient feature selection in binary classification.
problem Sparse feature selection in binary classification.
method Statistical mechanics inspired expectation propagation (EP) on a diluted Bayesian classifier.
result EP is a robust and competitive algorithm in terms of variable selection, estimation accuracy, and computational complexity.
FASC clusters data with latent factors, improving on naive methods.
problem Clustering high-dimensional data with correlated variables.
method Factor Adjusted Spectral Clustering (FASC) algorithm.
result FASC achieves an exponentially low mislabeling rate under general assumptions.
Annotation errors can significantly hurt classifier performance, yet datasets are only growing noisier with the increased use of Amazon Mechanical Turk and techniques like distant supervision that automatically generate labels. In this paper, we present a robust extension of logistic regression that incorporates the po…