SSMs can be poisoned with clean labels, leading to generalization failure.
problem The implicit bias of SSMs can be manipulated by including special training examples with clean labels.
method Formal proof and empirical demonstration of the phenomenon.
result SSMs can fail to generalize even with clean labels, due to the inclusion of special training examples.
Adversarial training leads to clean data generalization with significant robust overfitting gap.
problem Significant robust generalization gap in adversarial training.
method Two theoretical views: representation complexity and training dynamics.
result ReLU nets with O(ND) extra parameters can achieve CGRO. Neural networks learn clean data patterns first, then noisy data, leading to improved performance initially but deteriorating later.
problem Improvement in prediction error on clean data during early training of neural networks with noisy labels.
method Theoretical analysis and experiments to explore the dynamics of gradient descent and the impact of clean and noisy data.
result Neural networks prioritize learning clean data patterns first, leading to improved performance initially but deteriorating later due to diminishing gradient dominance of clean samples over noisy ones.
Improved speech translation model using cleaned data and ensemble decoding.
problem End-to-end speech translation from English to German.
method Fine-tuning on cleaned data, weight normalization, label smoothing, checkpoint averaging, ensemble decoding.
result Ensemble model achieved a BLEU score of 10.24 on test data.
CUTS removes corruption from models without clean data, improving utility and security.
problem Removing corruption from models without access to clean training data.
method CUTS uses a proxy set to amplify corruption and subtract it from model weights.
result CUTS recovers a large fraction of lost utility and nearly eliminates attacks with minimal damage.
SAT improves adversarial training by smoothing the loss landscape through curriculum learning.
problem Adversarial training sacrifices clean accuracy for robustness and suffers from large generalization error.
method SAT uses curriculum learning to smooth the adversarial loss landscape, improving both clean and robust accuracy.
result SAT models improve clean and robust accuracy significantly compared to adversarial training and other baselines.
Collecting large training datasets, annotated with high-quality labels, is costly and time-consuming. This paper proposes a novel framework for training deep convolutional neural networks from noisy labeled datasets that can be obtained cheaply. The problem is formulated using an undirected graphical model that represe…
Consensus NN learns from noisy data only for medical image denoising.
problem Lack of clean training data for medical image denoising.
method Trains neural network using only noisy data by splitting and combining subsets.
result Improved performance on denoising medical images compared to existing methods.
New method finds optimal training stop point with noisy labeled data.
problem Finding optimal training stop point with noisy labeled data.
method Analyzed training accuracy rate changes for different noise ratios to identify a training stop region. Developed a heuristic algorithm based on a small-learning assumption.
result Identified optimal training stop point at or close to maximum obtainable test accuracy.
This paper presents a statistical method of single-channel speech enhancement that uses a variational autoencoder (VAE) as a prior distribution on clean speech. A standard approach to speech enhancement is to train a deep neural network (DNN) to take noisy speech as input and output clean speech. Although this supervis…
Algorithm detects corrupted data using gradient descent.
problem Detecting mislabeled samples in noisy training datasets.
method Optimizing network performance on clean validation set via gradient descent.
result Algorithm improves network performance on validation set.
Method cleans noisy training labels for biomedical data.
problem Accurately labeling biomedical data is challenging.
method Reliability-based training data cleaning with inductive conformal prediction.
result Significant enhancements in classification performance across multiple tasks.
Cincer cleans both new and past data by identifying and relabeling suspicious and counter-examples.
problem Sequential learning under label noise, especially in applications with human supervision.
method Cincer uses example-based explanations to identify and relabel suspicious and counter-examples, leveraging Fisher information matrix approximation.
result Cincer achieves better data and models by clarifying the model's suspicions, especially with FIM approximation.
Detects poisoned training samples in deep neural networks.
problem Data poisoning attacks on deep neural networks.
method Two approaches: parametric probability distributions and Bayesian deep neural networks.
result Uncertainty estimates from trained models can discriminate clean from poisoned samples.
Adversarial training adds dynamic perturbations to neural networks for robustness.
problem Accuracy trade-off and lack of diversity in adversarial examples.
method Dynamic adversarial perturbations in the parameter space of neural networks, updating perturbation biases during training.
result Adversarial training with negligible cost and reduced accuracy trade-off.
INN method refines clean labeled data from noisy labels.
problem Handling noisy labels in deep neural networks.
method INN method based on memorization effect at neighbor regions.
result INN method resolves memorization effect shortcomings.
Paper tackles speaker verification by removing reverberation using deep LSTM networks.
problem Improving speaker verification accuracy in reverberant environments.
method Dual-label deep LSTM networks trained to map reverberant to clean speech features.
result Evaluates performance using EERs, showing improved accuracy.
MetaPoison poisons neural networks by learning to craft imperceptible changes to training data.
problem Data poisoning attacks on neural networks.
method MetaPoison uses meta-learning to approximate bilevel optimization for crafting poisons.
result MetaPoison outperforms previous methods and works in various scenarios.
New attack method makes poisoned images transferable without access to victim model.
problem Injecting poisoned images into training data to mislead deep nets.
method Designing poison images to surround targeted images in feature space, using Dropout.
result Achieves over 50% transferable success with only 1% poisoning of training set.
FR-Train improves fair and robust AI training by detecting and reducing poisoned data.
problem Training AI models that are fair and robust in the presence of data bias and poisoning.
method Mutual information-based adversarial training with an additional discriminator.
result FR-Train maintains fairness and accuracy even in the presence of poisoned data.
Paper proposes training deep nets on noisy labels without manual annotation.
problem Training deep neural networks on noisy labels without manual annotation.
method Directly train deep neural network on noisy candidate labels, early stopping to avoid overfitting.
result Training on noisy candidate labels yields higher test performance than on clean data.
Self-supervised method predicts clean signal and noise distribution from noisy images.
problem Blind denoising and noise estimation in biomedical images with limited clean data.
method Two neural networks jointly predict clean signal and noise distribution from noisy observations.
result Significantly outperforms state-of-the-art algorithms on six biomedical image datasets.
New findings explain why online methods outperform offline methods in noisy expert feedback settings.
problem The challenge of learning from imperfect expert feedback in sequential decision-making systems.
method Introduced a noisy expert model and a novel variant of on-policy distillation (OPD) to address the gap between offline and online imitation learning.
result Online interaction with a noisy expert via OPD enables polynomial dependence on the horizon, unlike offline methods which require exponential growth in sample complexity.
CAT improves robustness of neural networks by customizing perturbation levels.
problem Poor generalization of adversarial training on clean and perturbed data.
method CAT adapts perturbation level and label for each sample.
result CAT achieves better clean and robust accuracy than previous methods.
Game theory helps machine learn better from adversarial queries.
problem Adversarial evasion in machine learning prediction.
method Repeated Bayesian Sequential Game to balance classifier selection and query type.
result Learner selects appropriate classifier for clean vs. adversarial queries.
Paper proposes iterative trimmed loss minimization for learning from corrupted data.
problem Learning from corrupted training data.
method Iterative trimmed loss minimization, alternating between selecting and retraining samples.
result Recovery of ground truth with linear convergence rate in generalized linear models.
This work benchmarks and theorizes robust NAS under adversarial training.
problem Lack of benchmark evaluations and theoretical guarantees for robust NAS architectures under adversarial training.
method Released a comprehensive data set and established a generalization theory using the neural tangent kernel.
result Established a generalization theory for robust NAS architectures under adversarial training.
EntProp increases entropy of clean samples to generate out-of-distribution data for better DNN performance.
problem Improving deep neural networks' accuracy and robustness to out-of-distribution data.
method High entropy propagation using data augmentation and free adversarial training.
result EntProp achieves higher standard accuracy and robustness with lower training cost.
Study improves resilience against adversarial clean-label attacks in real and noisy settings.
problem Ensuring accurate predictions in the presence of adversarial clean-label samples.
method Sequential learning from a stream of i.i.d. data, allowing abstention for uncertain predictions.
result Theoretical analysis and adaptations for the agnostic setting with a clean-label adversary and noise.
Paper develops an algorithm with PAC guarantees for detecting alien categories.
problem Detecting alien categories not seen in training data reliably.
method Develops an algorithm with PAC-style guarantees for alien detection under known upper bounds on alien fraction.
result Empirical results show the algorithm's effectiveness in detecting aliens.
This work improves ASR noise robustness using parallel data and T/S learning.
problem Noise robustness in automatic speech recognition.
method Teacher-student learning with parallel clean and noisy data, logits selection.
result Best student model yields significant WER reductions in noisy conditions.
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the …
ENSURE framework trains deep image recon algorithms without clean data.
problem Lack of clean, fully sampled ground-truth data for deep learning image reconstruction.
method Introduces ENSURE framework, a generalization of SURE and GSURE to random sampling patterns.
result ENSURE loss function is an unbiased estimate for true mean-square error.
We apply basic statistical reasoning to signal reconstruction by machine learning -- learning to map corrupted observations to clean signals -- with a simple and powerful conclusion: it is possible to learn to restore images by only looking at corrupted examples, at performance at and sometimes exceeding training using…
Butterfly tackles wild unsupervised domain adaptation with noisy labeled data.
problem Training classifiers with noisy labeled data from source domain and unlabeled data from target domain.
method Butterfly framework, maintaining four deep networks for simultaneous adaptations.
result Butterfly significantly outperforms existing methods in wild unsupervised domain adaptation.
Enhances deep learning robustness to noise without sacrificing clean data accuracy.
problem Robustness of deep neural networks to input noise.
method Discriminative loss at penultimate layer and class-wise feature alignment with Gaussian noise.
result Improves robustness to various perturbations without degrading clean data accuracy.
A method to learn from noisy labels without a clean validation set.
problem Learning from samples with noisy labels.
method Limited Gradient Descent: modifying labels to estimate optimal stopping timing.
result Comparable and sometimes superior generalization performance compared to methods relying on clean validation sets.
PA-GNN enhances GNN robustness against poisoning attacks using clean graph knowledge.
problem Improving robustness of GNNs against poisoning attacks.
method PA-GNN uses a penalized aggregation mechanism and meta-optimization to transfer robustness from clean graphs.
result PA-GNN significantly improves GNN robustness against poisoning attacks on real-world graphs.
New method denoises and fills in missing image data without clean training data.
problem Denoising and inpainting images with noisy, incomplete data.
method Robust Hadamard Autoencoders trained on noisy, incomplete data.
result Autoencoders can perform denoising and inpainting simultaneously.
Deep networks learn clean structure before memorizing corrupted labels, leaving a spectral signature in gradient centered scatter.
problem Deep networks' transition from learning clean structure to memorizing corrupted labels under label noise.
method Analysis of the centered scatter of per-example last-layer gradients to identify Fisher Rank Inflation.
result Fisher Rank Inflation is a spectral signature of memorization under label noise, with effective rank expanding during memorization.
Adversarial robustness varies with input data distribution.
problem Vulnerability of adversarial training to input data distribution.
method Study of adversarial training and Bayes classifier behavior.
result Adversarial robustness is sensitive to input data distribution.
DynaCor detects noisy labels by learning from corrupted training signals.
problem Label noise in real-world datasets hinders model generalization.
method DynaCor introduces label corruption to indirectly simulate noisy labels and learns to distinguish clean from noisy instances.
result DynaCor outperforms state-of-the-art competitors in noisy label detection.
Develops a robust training framework to detect backdoor attacks in DNNs.
problem Vulnerability of DNNs to backdoor attacks by poisoned training data.
method Collider framework selects prominent samples based on geometric structures and coreset selection objective.
result Significantly reduces backdoor success rate in various poisoned datasets.
A new method selects clean samples to train DNNs with noisy labels.
problem Training deep neural networks with noisy labeled data.
method Adaptive k-set selection to choose clean samples at each epoch.
result The method guarantees performance with a theoretical bound on regret.
DIP-FAT improves adversarial training by diversifying perturbations.
problem Adversarial examples fool deep neural networks, leading to overfitting and poor performance.
method DIP-FAT uses random directions to diversify perturbations in adversarial training.
result DIP-FAT reduces overfitting and improves clean data accuracy.
Selective forgetting method cleans deep network weights of forgotten data.
problem Selective forgetting of specific data subsets in deep neural networks.
method A method to scrub weights clean of forgotten data without retraining.
result The method ensures indistinguishability of probing functions from a non-forgotten network.
A new method aggregates generative classifiers to resist adversarial attacks.
problem Adversarial attacks on deep neural networks.
method Rank-aggregating ensemble of generative classifiers trained on intermediate layer responses.
result The ensemble of generative classifiers shows robustness to adversarial attacks.
PrIU optimizes machine learning model updates after data cleaning.
problem Incrementally updating machine learning models after removing problematic training samples.
method Provenance-based approach for efficient model parameter updates.
result PrIU-opt achieves up to two orders of magnitude speed-up compared to retraining from scratch.