Deep networks can handle noisy labels up to a certain threshold.
problem Deep learning's robustness to noisy labels.
method Applying classical statistical theory and universal consistency of DNNs.
result Certain DNNs can tolerate massive symmetric label noise up to the information-theoretic threshold.
Recent developments have established the vulnerability of deep reinforcement learning to policy manipulation attacks via intentionally perturbed inputs, known as adversarial examples. In this work, we propose a technique for mitigation of such attacks based on addition of noise to the parameter space of deep reinforcem…
Additive asynchronous and cyclostationary impulsive noise limits communication performance in OFDM powerline communication (PLC) systems. Conventional OFDM receivers assume additive white Gaussian noise and hence experience degradation in communication performance in impulsive noise. Alternate designs assume a parametr…
Omega method mitigates noise in stochastic game optimization.
problem Noise sensitivity and convergence issues in stochastic game optimization.
method Omega method incorporates EMA of historic gradients in its update rule.
result Omega method outperforms optimistic gradient method in stochastic games.
RDP-GAN improves GAN privacy by adding random noises to loss function.
problem Protecting sensitive information in GANs while maintaining quality of generated samples.
method Integrates Rényi-differential privacy into GAN training process by adding random noises to loss function.
result Achieves better privacy protection with high-quality samples compared to existing methods.
It is known that evolution strategies in continuous domains might not converge in the presence of noise. It is also known that, under mild assumptions, and using an increasing number of resamplings, one can mitigate the effect of additive noise and recover convergence. We show new sufficient conditions for the converge…
A new hybrid framework reduces quantum runtime and noise effects.
problem Challenges in deploying deep QFMs on real quantum hardware.
method Iterative Quantum Feature Maps (IQFMs) combining shallow QFMs and classical augmentation weights.
result Numerical experiments show IQFMs outperforming quantum convolutional neural networks.
New method improves nonlinear filtering accuracy with reduced computation.
problem Complex nonlinear filtering with small system noise.
method Asymptotic expansion with ordinary differential equations and Edgeworth-type correction.
result Significantly lower computational cost with improved accuracy.
New method bounds hardware noise without assumptions.
problem Estimating hardware noise without assumptions.
method Machine Learning and Conformal Prediction.
result Theoretical upper bounds of fidelity.
Label noise in adversarial training leads to robust overfitting, explained and mitigated.
problem Label noise in adversarial training causes robust overfitting.
method Proposed a method to automatically calibrate labels.
result Consistent performance improvements across various models and datasets.
Label smoothing improves model performance even with noisy labels.
problem Mitigating label noise in deep learning models.
method Examined label smoothing as a technique to cope with label noise and compared it to loss-correction methods.
result Label smoothing is competitive with loss-correction techniques under label noise and beneficial for distillation from noisy data.
Because large, human-annotated datasets suffer from labeling errors, it is crucial to be able to train deep neural networks in the presence of label noise. While training image classification models with label noise have received much attention, training text classification models have not. In this paper, we propose an…
We introduce a training method for both better word representation and performance, which we call GROVER (Gradual Rumination On the Vector with maskERs). The method is to gradually and iteratively add random noises to word embeddings while training a model. GROVER first starts from conventional training process, and th…
Holographic Invariant Storage uses vector architectures to ensure LLM safety at design time.
problem Mitigating context drift in large language models (LLMs) during deployment.
method Introduces Holographic Invariant Storage (HIS) protocol that combines known properties of bipolar Vector Symbolic Architectures into a design-time safety contract.
result Closed-form guarantees for single-signal recovery fidelity, continuous-noise robustness, and multi-signal capacity degradation are provided and validated.
Study shows label errors impact model disparity metrics, proposing mitigation methods.
problem Impact of label errors on model disparity metrics.
method Empirical study, characterizing label error effects; proposing estimation and relabeling methods.
result Label errors significantly affect model disparity metrics, particularly for minority groups.
Analyzes dropout in neural networks using differential equations.
problem Lack of theoretical understanding of dropout's effectiveness.
method Analytical study of dropout in two-layer neural networks with online stochastic gradient descent.
result Dropout reduces detrimental correlations and optimal probability increases with data noise.
A continuing mystery in understanding the empirical success of deep neural networks is their ability to achieve zero training error and generalize well, even when the training data is noisy and there are more parameters than data points. We investigate this overparameterized regime in linear regression, where all solut…
NAPP-ERM improves ERM with differential privacy guarantees by iteratively achieving target regularization and delivering strong convexity.
problem Over-regularization in privacy-preserving ERM approaches.
method Noise-Augmented Privacy-Preserving Empirical Risk Minimization (NAPP-ERM) with a dual-purpose l2 regularizer and privacy budget retrieval strategy.
result Mitigates over-regularization and achieves strong convexity through a single regularizer.
SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.
problem Misalignment during gradient descent in phase retrieval models with anisotropic inputs.
method Spectral gradient descent modifies gradient updates to preserve directional information and remove spike amplification.
result SpecGD removes spike amplification, leading to stable alignment and accelerated noise contraction.
Bayesian consensus improves accuracy of forecasts from miscalibrated sources.
problem Aggregating predictions from miscalibrated and noisy sources.
method Bayesian approach to adjust for bias and noise, using hierarchical models.
result Bayesian consensus estimator is unbiased and more efficient than alternatives.
Noise-resilient method improves Hurst exponent estimation accuracy in noisy data.
problem Noise degrades accuracy of Hurst exponent estimation methods.
method Noise-Controlled ALPHEE (NC-ALPHEE) using wavelet multi-scale analysis and neural network combination.
result NC-ALPHEE consistently outperforms existing techniques in noisy conditions.
SAP corrects model for label noise by identifying and removing noisy samples.
problem Label corruption degrades model performance; acquiring perfect labels is costly.
method SAP uses SVD to identify and project model weights onto a clean activation space.
result SAP improves model generalization by up to 6% on CIFAR dataset with 25% synthetic corruption.
Study explores loss design for decision trees to improve robustness against noisy labels.
problem Improving decision tree robustness to noisy labels.
method Investigated loss correction and symmetric losses, found ineffective.
result Other loss design directions need exploration for robust decision trees.
We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as possible. Similarly to active exploration in multi-armed bandit (MAB), states may have…
New method uses generative models to improve phase retrieval stability.
problem Improving stability of solutions in phase retrieval problems.
method Unified reconstruction approach using generative models to mitigate overfitting.
result Mitigates overfitting to generative model for varying noise levels.
Study improves parameter estimation for SDEs driven by Levy noise.
problem Challenges in estimating parameters of SDEs with non-Gaussian noises.
method Introduces PEnet, a CNN-LSTM model for efficient parameter estimation.
result PEnet offers superior accuracy and adaptability for various SDE scenarios.
A new diffusion model tackles brightness issues with a probabilistic approach.
problem Brightness-related limitations in diffusion models.
method Introduces a novel diffusion model with a probabilistic framework, modifying both forward and reverse diffusion processes.
result The model mitigates brightness-related limitations and improves performance in high-dimensional settings.
Improves diffusion models by controlling total variance and signal-to-noise-ratio.
problem Long sampling time in diffusion models.
method Total-Variance/Signal-to-Noise-Ratio (TV/SNR) disentangled framework.
result Improves generation performance by controlling TV and SNR independently.
Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance. To mitigate this issue, we provide a simple but effective baseline method that is robust to noisy labels, even with severe noise. Our objective involves a variance regulariz…
Improved self-distillation reduces label noise and enhances model accuracy.
problem Label noise in multi-class classification.
method Label averaging and refined partial labels.
result Single-round self-distillation achieves comparable performance to multi-round distillation.
TensorHyper-VQC improves VQC scalability and robustness.
problem Scalability and noise sensitivity in VQC.
method Tensor-train-guided hypernetwork framework.
result TensorHyper-VQC achieves superior performance and robust noise tolerance.
Deep models can fit noisy labels, but robustness and reliability are still issues.
problem Training deep models with noisy labels leads to unreliable uncertainty quantification.
method Analysis of conditional distribution over noisy labels and evaluation of robust loss functions.
result Strictly proper and robust loss functions preserve accuracy but do not guarantee reliability.
We propose self-adaptive training---a new training algorithm that dynamically corrects problematic training labels by model predictions without incurring extra computational cost---to improve generalization of deep learning for potentially corrupted training data. This problem is crucial towards robustly learning from …
A new method detects small holes in noisy data.
problem Detecting small holes in high-density regions from noise.
method Robust Density-Aware Distance (RDAD) filtration, incorporating distance-to-measure concept.
result The RDAD filtration prolongs the persistences of small holes, making them distinguishable from noise.
New method calibrates noise for attack risk, improving ML model accuracy.
problem Improving accuracy of privacy-preserving ML models while maintaining privacy.
method Directly calibrates noise scale to a desired attack risk level, bypassing the standard ε-calibration. result Significantly decreases noise scale, leading to increased utility at the same risk level.
Paper tackles noise-robust domain adaptation in noisy environments.
problem Learning machines struggle with domain adaptation in noisy environments.
method The paper proposes offline curriculum learning and proxy distribution based margin discrepancy to mitigate label and feature noise.
result The proposed algorithm significantly outperforms state-of-the-art methods in noisy environments.
New algorithm mitigates bias in subset selection with noisy protected attributes.
problem Mitigating bias in subset selection when protected attributes are noisy.
method Formulated a denoised selection problem and developed a linear-programming based approximation algorithm.
result The approach can produce fairer subsets despite noisy protected attributes.
Improved federated learning over-the-air reduces noise impact.
problem Noise in OTA FL degrades model accuracy.
method Developed COTAF algorithm with precoding and scaling.
result COTAF achieves similar convergence rate to error-free channels.
Generative models improve for multiscale scientific data with new noise and interpolation techniques.
problem Numerical challenges in generating high-fidelity samples for multiscale scientific data.
method Design of noise distributions and interpolation schedules in function space to ensure Lipschitz regularity and finite noise roughness.
result Scale-adaptive noise and interpolation schedules improve numerical efficiency and fidelity of generated samples.
Resetting from checkpoints improves DNN training with noisy labels.
problem Latent gradient bias induced by noisy labels causes overfitting.
method Stochastic resetting applied to SGD to mitigate latent gradient bias.
result Resetting significantly improves DNN generalization performance.
DRFLM improves federated learning by handling data heterogeneity and noise.
problem Data heterogeneity and noise in federated learning.
method Distributionally robust optimization and mixup techniques.
result Enhanced global model prediction accuracy through robust optimization and local mixup.
Double descent risk in L2-regularized models explained and mitigated.
problem Risk of overparameterized models in machine learning.
method Analysis of L2-regularized models, two-layer neural networks, and CNNs.
result Double descent risk in L2-regularized models can be explained and mitigated by adjusting regularization strengths.
Class2Simi reduces noise in noisy label learning by transforming noisy class labels into noisy similarity labels.
problem Learning with noisy labels in supervised and unsupervised settings.
method Transforming noisy class labels into noisy similarity labels, training DNNs from noisy data pairs.
result The noise rate reduction is theoretically guaranteed, making it easier to handle noisy similarity labels.
TSLANet improves time series models by capturing long-term and short-term interactions.
problem Noise sensitivity, computational efficiency, and overfitting in Transformer-based models for time series data.
method Adaptive Spectral Block and Interactive Convolution Block for robust feature representation and noise mitigation.
result TSLANet outperforms state-of-the-art models in various time series tasks.
Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
The ability for policies to generalize to new environments is key to the broad application of RL agents. A promising approach to prevent an agent's policy from overfitting to a limited set of training environments is to apply regularization techniques originally developed for supervised learning. However, there are sta…
We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle inequality which demonstrates that this procedure provably mitigates the variance in…
The paper analyzes how deep neural networks handle noisy labels and finds disparate impacts.
problem Disparate impacts of noisy labels on instances with different representation frequencies.
method Quantifying harms, analyzing solutions, and comparing their impacts on different frequency instances.
result Existing solutions lead to disparate treatments, benefiting higher-frequency instances more.