Noise improves model quality in non-linear neural networks during decentralized training.
problem Improving generalization of locally trained neural networks.
method Injecting noise into the weights of neural networks during decentralized training.
result Noise injection improves model quality for non-linear neural networks, but not for linear models.
Paper proposes a method to train deep text classification models robust to label noise.
problem Training deep text classification models with noisy labels.
method Introduces a non-linear processing layer (noise model) into CNN architecture, learned jointly with CNN weights.
result The approach enables better sentence representations and robustness to extreme label noise.
Synthetic noise training improves machine translation robustness to spelling mistakes.
problem Making machine translation robust to spelling mistakes and natural noise.
method Training on synthetic noise to improve robustness to natural noise.
result Training on synthetic noise improves robustness to natural noise without diminishing performance on clean text.
Label noise in adversarial training leads to robust overfitting, explained and mitigated.
problem Label noise in adversarial training causes robust overfitting.
method Proposed a method to automatically calibrate labels.
result Consistent performance improvements across various models and datasets.
Improved neural language models trained with dynamic noise-contrastive estimation.
problem Training large-scale language models efficiently and avoiding overfitting.
method Dynamic Noise-Contrastive Estimation (DNCE) to train neural trans-dimensional random field language models.
result DNCE reduces training cost and improves model performance on large datasets.
GROVER improves word representations by gradually adding random noises during training.
problem Improving word representations for better model performance.
method Gradually adding random noises to word embeddings during training.
result GROVER improves model performances on most text classification datasets.
FANNet analyzes noise tolerance and training bias in neural networks.
problem Low noise tolerance and input sensitivity in neural networks lead to failures on unseen inputs.
method Formal analysis using model checking under different noise ranges.
result Noise tolerance of ±11% for the trained network, sensitive input nodes identified, and biasness confirmed. TrustNet robustly learns noise patterns from trusted data to improve weakly-supervised classification.
problem Robustness to label noise in weakly-supervised learning.
method TrustNet learns noise patterns from trusted data, then trains a robust classifier using these patterns.
result TrustNet outperforms state-of-the-art methods in robustness to various noise patterns.
Paper tackles robust training with noisy labels using a trusted set and pseudo labels.
problem Training deep neural networks with high label noise.
method Leveraging a small trusted set to estimate exemplar weights and pseudo labels for noisy data.
result Achieves high accuracy on large-scale datasets with real-world label noise.
Noise enhancement improves generalization in training.
problem Improving generalization in training with controlled noise.
method Noise enhancement method to control SGD noise without changing learning rate or minibatch size.
result Noise enhancement improves generalization for real datasets.
A deep abstaining classifier tackles label noise in deep learning.
problem Label noise in deep learning training data.
method Proposes a loss function allowing deep neural networks to abstain from making predictions on confusing samples.
result Deep abstaining classifier (DAC) improves robust learning in various types of label noise.
NR-GANs learn clean images from noisy data.
problem Learning clean images from noisy training data.
method Introduced a noise generator and distribution/transformation constraints.
result NR-GANs can generate clean images from noisy data.
Noise helps neural networks escape local optima.
problem Understanding the role of noise in neural network training.
method Perturbed gradient descent and noise annealing.
result Noise guarantees convergence to global optimum in polynomial time.
Self-adaptive training improves deep learning robustness.
problem Improving deep learning performance on corrupted data.
method Dynamic correction of problematic labels using model predictions.
result Self-adaptive training significantly improves generalization over ERM under various levels of noise.
S-SGD adds symmetrical noise to weights to avoid sharp minima in deep learning.
problem SGD does not always converge to a flat minimum, leading to poor generalization.
method Symmetrical weight noise injection in SGD.
result S-SGD outperforms conventional SGD and weight-noise injection methods in large batch training.
Meta-trained optimizers improve model robustness to image corruptions.
problem Robustness of deep learning models to input noise.
method Meta-training a learned optimizer to produce robust models.
result Meta-trained optimizers improve model robustness to Gaussian noise.
Analytic expressions for deep neural network output under stochastic training.
problem Understanding the impact of noise and hyperparameters on deep neural network performance.
method Taylor expansion of network output to derive analytical expressions for weights and output.
result Noise in training affects generalization by preventing the output from fully converging on train data, but does not provide explicit regularization.
Paper shows adversarial training can be fooled by new type of noise.
problem Adversarial training can be fooled by new types of noise.
method Designing ADVIN, a new type of inducing noise.
result ADVIN can degrade adversarial training robustness by 99.9%.
CNN trained with noise improves multi-speaker localization accuracy.
problem Multi-speaker localization in noisy environments.
method Convolutional Neural Network (CNN) trained with synthesized noise.
result The CNN-based method outperforms the steered response power method.
New method improves deep learning models robustness to label noise.
problem Improving deep learning models' robustness to corrupted labels.
method Sparse over-parameterization and implicit regularization.
result State-of-the-art test accuracy against label noise on various datasets.
New bounds for KANs trained with DP-SGD, addressing correlated noise.
problem Risk bounds for Kolmogorov-Arnold Networks trained by DP-SGD with correlated noise.
method Established new optimization and population risk analysis for KANs trained with DP-SGD, addressing correlated noise.
result First optimization and population risk analysis of correlated-noise mechanisms for DP training in non-convex settings, including neural networks.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
Noise Injection probes deep learning dynamics during training phases.
problem Understanding the learning mechanism of deep neural networks.
method Noise Injection Nodes (NINs) are used to perturb DNN architectures without altering the optimization algorithm.
result Distinct training phases are observed based on the scale of injected noise.
RAD improves robustness to domain annotation noise without explicit domain annotations.
problem Robustness to domain annotation noise in training data.
method Regularized Annotation of Domains (RAD) for last layer retraining.
result RAD outperforms state-of-the-art methods even with 5% noise in training data.
Interpolating noisy data in linear regression leads to zero training error.
problem Understanding why deep neural networks generalize well with noisy data.
method Investigated overparameterized linear regression, analyzing generalization error and proposing a hybrid scheme.
result Interpolating solutions in noisy data can generalize well, with error decaying to zero with more features.
The paper examines how SGD noise deviates from Gaussian distribution.
problem Understanding why SGD outperforms GD in neural networks.
method Analysis of SGN vectors' distribution during training.
result For large batch sizes, SGN vectors are mostly Gaussian in early phases.
Early stopping helps prevent overfitting to noisy labels in neural networks.
problem Overfitting to noisy labels in real-world training data.
method Two-phase training method (Prestopping) that early stops training and resumes using a maximal safe set.
result Significantly outperforms state-of-the-art methods in test error under label noise.
The study improves generalization in large-batch training by adding structured covariance noise to gradients.
problem Improving generalization in large-batch training while maintaining optimal convergence.
method Adding covariance noise to the gradients to improve generalization performance.
result The method improves generalization performance without degrading optimization performance and training duration.
New algorithm reduces noise in GAN training, improving stability and efficiency.
problem Stochastic gradient noise hinders GAN convergence.
method Stochastic variance-reduced extragradient (SVRE) optimization algorithm.
result SVRE improves GAN training stability and efficiency compared to standard methods.
Paper presents faster, robust adversarial training methods.
problem Increasing neural network robustness against adversarial attacks.
method Integrates FGSM with Pixelwise Noise Injection Layer (PNIL) and uniform noise.
result Achieves comparable results to PGD-based adversarial training but faster.
Large datasets often have unreliable labels-such as those obtained from Amazon's Mechanical Turk or social media platforms-and classifiers trained on mislabeled datasets often exhibit poor performance. We present a simple, effective technique for accounting for label noise when training deep neural networks. We augment…
Paper tackles robustness in adversarial noise with a meta-optimizer.
problem Sensitivity to adversarial noise hinders machine learning deployment.
method Meta-optimizer learns to robustly optimize models using adversarial examples.
result Meta-optimizer transfers adversarial knowledge to new models without generating new examples.
SGD-trained neural networks generalize well even with adversarial label noise.
problem Generalization of neural networks trained on adversarial label noise.
method Training a one-hidden-layer neural network with SGD on arbitrary width networks.
result SGD-trained networks achieve classification accuracy competitive with the best halfspace over adversarial label noise.
IEBN normalizes noise by enhancing instance-specific information, improving deep learning performance.
problem Improving deep learning performance by regulating noise in batch normalization.
method Integrates self-attention mechanism to recalibrate channel information in BN.
result IEBN outperforms BN with improved generalization and stability.
Stochastic Gradient Descent introduces noise in training, affecting model decision boundaries.
problem Understanding the impact of noise in SGD on model decision boundaries.
method Characterized SGD and persistent SGD dynamics in a neural network model, measuring noise magnitude in both under- and over-parametrized regimes.
result Noisier algorithms lead to wider decision boundaries in constraint satisfaction problems.
Extends neural network training framework to handle noise and uncertainty.
problem Handling noise and uncertainty in neural network training.
method Integrates non-zero aleatoric noise and derives posterior covariance for epistemic uncertainty.
result Derives an estimator for posterior covariance, providing a handle on epistemic uncertainty.
Self-supervised method predicts clean signal and noise distribution from noisy images.
problem Blind denoising and noise estimation in biomedical images with limited clean data.
method Two neural networks jointly predict clean signal and noise distribution from noisy observations.
result Significantly outperforms state-of-the-art algorithms on six biomedical image datasets.
This work improves ASR noise robustness using parallel data and T/S learning.
problem Noise robustness in automatic speech recognition.
method Teacher-student learning with parallel clean and noisy data, logits selection.
result Best student model yields significant WER reductions in noisy conditions.
A convolution neural network (CNN) based classification method for broadband DOA estimation is proposed, where the phase component of the short-time Fourier transform coefficients of the received microphone signals are directly fed into the CNN and the features required for DOA estimation are learnt during training. Si…
Noise improves deep neural network performance, especially in knowledge distillation.
problem Improving deep neural network performance and reducing performance gap.
method Injecting constructive noise at different levels in the collaborative learning framework.
result Constructive noise enables effective training and distillation of desirable characteristics.
NoL approach improves adversarial robustness by modeling random noise during training.
problem Improving neural network robustness against adversarial attacks.
method Implicit generative modeling of random noise during training.
result Models trained with NoL perform better against a wide range of adversarial attacks.
Noise Injection Node Regularization improves robustness in neural networks.
problem Improving robustness of neural networks against various perturbations.
method Injecting structured noise into neural networks during training.
result Significant improvement in robustness against data perturbations.
CNNs trained by gradient descent can learn intrinsic image rank robustly to background noises.
problem Understanding the intrinsic dimension of data in over-parameterized CNNs.
method Theoretical analysis and experiments on synthetic and real datasets.
result CNNs trained by gradient descent can learn the intrinsic dimension of clean images robustly to background noises.
New method calibrates noise for attack risk, improving ML model accuracy.
problem Improving accuracy of privacy-preserving ML models while maintaining privacy.
method Directly calibrates noise scale to a desired attack risk level, bypassing the standard ε-calibration. result Significantly decreases noise scale, leading to increased utility at the same risk level.
Paper improves image classification accuracy with a new Noise Modeling Network.
problem Improving performance of multi-label image classifiers with noisy or missing labels.
method Integrates a Noise Modeling Network (NMN) with a CNN to jointly learn noise distribution and CNN parameters.
result Consistently improves classification performance on MSR-COCO and MSR-VTT datasets.
Colored noise improves neural network robustness against adversarial attacks.
problem Vulnerability of neural networks to adversarial perturbations.
method Injection of colored noise into network weights and activations during adversarial training.
result Our approach outperforms previous methods in terms of adversarial accuracy on CIFAR-10 and CIFAR-100 datasets.
DynaCor detects noisy labels by learning from corrupted training signals.
problem Label noise in real-world datasets hinders model generalization.
method DynaCor introduces label corruption to indirectly simulate noisy labels and learns to distinguish clean from noisy instances.
result DynaCor outperforms state-of-the-art competitors in noisy label detection.
Noise in linear networks minimizes sharpness and leads to shrinkage-thresholding.
problem Minimizing sharpness in diagonal linear networks.
method Stochastic sharpness-aware minimization (SAM) with isotropic noise.
result Noise forces shrinkage-thresholding of true parameters.