Synthetic noise training improves machine translation robustness to spelling mistakes.
problem Making machine translation robust to spelling mistakes and natural noise.
method Training on synthetic noise to improve robustness to natural noise.
result Training on synthetic noise improves robustness to natural noise without diminishing performance on clean text.
Quantum classification robustness improved via quantum hypothesis testing.
problem Vulnerability of quantum classification algorithms to input perturbations.
method Formalized link between quantum hypothesis testing and robustness, developed practical protocols.
result Tight robustness condition independent of noise source (natural or adversarial).
New method designs joint initial noises for diffusion models to improve diversity and alignment.
problem Independent initial noises limit diversity in generated images.
method Coupling of initial noises, maintaining Gaussian distribution while allowing dependence.
result Repulsive Gaussian coupling improves diversity without increasing sampling cost.
Noise makes learning linear thresholds hard, but algorithms can still learn near-optimal thresholds.
problem Learning linear thresholds in noisy data.
method Exploiting natural assumptions on data-generating process.
result Efficient learning of near-optimal linear thresholds is still possible with small data even in the presence of noise.
A successful class of image denoising methods is based on Bayesian approaches working in wavelet representations. However, analytical estimates can be obtained only for particular combinations of analytical models of signal and noise, thus precluding its straightforward extension to deal with other arbitrary noise sour…
It is important to learn various types of classifiers given training data with noisy labels. Noisy labels, in the most popular noise model hitherto, are corrupted from ground-truth labels by an unknown noise transition matrix. Thus, by estimating this matrix, classifiers can escape from overfitting those noisy labels. …
New methods for handling time-varying label noise in time series classification.
problem Temporal label noise in time series classification tasks.
method Proposed methods to estimate temporal label noise function directly from data.
result Our methods lead to state-of-the-art performance under diverse types of temporal label noise.
Generative models learn better with data-adaptive noise.
problem Learning heavy-tailed distributions in flow-based models.
method Data-adaptive latent noise using 1D quantile functions optimized via Wasserstein distance.
result Flexibility and effectiveness in learning heavy-tailed and compactly supported distributions.
A central challenge in neuroscience is to understand neural computations and circuit mechanisms that underlie the encoding of ethologically relevant, natural stimuli. In multilayered neural circuits, nonlinear processes such as synaptic transmission and spiking dynamics present a significant obstacle to the creation of…
Study shows how noise can ensure solutions to fluid dynamics equations.
problem Ensuring unique solutions to stochastic fluid dynamics equations.
method Extended existing results to linear advection of k-forms, proving existence and uniqueness of weak L^p-solutions.
result Proved existence and uniqueness of weak L^p-solutions to stochastic linear advection equation of k-forms.
One way to avoid overfitting in machine learning is to use model parameters distributed according to a Bayesian posterior given the data, rather than the maximum likelihood estimator. Stochastic gradient Langevin dynamics (SGLD) is one algorithm to approximate such Bayesian posteriors for large models and datasets. SGL…
Generalized R2R handles non-Gaussian noise for deep network training.
problem Training deep networks from noisy data alone.
method Extending R2R to handle various noise distributions.
result GR2R loss is an unbiased estimator of supervised loss.
Most real life systems have a random component: the multitude of endogenous and exogenous factors influencing them result in stochastic fluctuations of the parameters determining their dynamics. These empirical systems are in many cases subject to noise of multiplicative nature. The special properties of multiplicative…
Enhances network intrusion detection in noisy data.
problem Robustness against contaminated and noisy data inputs in network intrusion detection.
method Probabilistic Temporal Graph Network Support Vector Data Description (TGN-SVDD) model.
result Significant improvements in detection performance with synthetic noise.
Proposes a new k-NN algorithm to improve classification accuracy by removing noise and pseudo-neighbours.
problem Noise and pseudo-neighbours in large-scale databases affect k-NN performance.
method Introduces a weighted mutual k-Nearest Neighbour algorithm to detect and remove noise, and minimize distant neighbours' influence.
result The proposed algorithm provides comparative better results compared to standard k-NN.
A method for diffusion on probability simplex for generative models.
problem Tension between continuous and discrete data in diffusion models.
method Proposes using softmax function applied to Ornstein-Uhlenbeck Process on probability simplex.
result Method extends to bounded image generation.
Paper improves SLCB regret bound for bounded noise.
problem Stochastic linear contextual bandits with bounded noise.
method Set-membership estimation (SME) and optimism in the face of uncertainty (OFU).
result Improved regret bound of O(logT). Study examines noise sensitivity of DNNs for binary classification.
problem Understanding non-robustness of DNN classifiers under noise.
method Defined and extended noise sensitivity and stability concepts for Boolean functions, applied to DNN models.
result Sorted out the relation between definitions and properties of DNN architectures under noise.
Algorithm learns mixtures of rankings from noisy data.
problem Learning an unknown mixture of rankings from noisy samples.
method Algorithm for different noise models, including heat kernel and Mallows model.
result High accuracy learning of mixture to nO(logk) time. Study on the noise in SGD minibatches near local minima.
problem Understanding the noise in SGD minibatches near local minima.
method Detailed analysis of SGD noise in linear regression and derivation of a general formula for different types of minima.
result Provides insight into the stability of training neural networks and suggests large learning rates can help generalization.
Researchers improve NCE by addressing its flat loss landscape issues.
problem NCE's poor performance due to an ill-behaved loss landscape.
method Introduced eNCE with an exponential loss and normalized gradient descent.
result Proven that landscape issues arise from inappropriate noise distribution.
Proposes a method to improve classification robustness against label noise.
problem Improving classification robustness against label noise.
method Extends a simple regression approach to classification, modeling labels as samples from a logistic-normal distribution.
result The method improves robustness against label noise in classification.
Noise in imputed values corrects biases in machine learning models.
problem Systematic biases in imputed values affect downstream analyses.
method Introducing noise to imputed values to correct biases.
result Noise-corrected imputation methods produce unbiased estimates.
This paper proposes a technique to prioritize examples based on their noise sensitivity for generating adversarial examples.
problem Generating effective adversarial examples requires careful consideration of noise sensitivity.
method The paper introduces a noise-sensitivity-analysis-based test prioritization technique to identify examples sensitive to noise.
result The method effectively picks out examples by their noise sensitivity, improving the effectiveness of adversarial examples.
We revisit the Bayesian online inference problems for the linear dynamic systems (LDS) under non- Gaussian environment. The noises can naturally be non-Gaussian (skewed and/or heavy tailed) or to accommodate spurious observations, noises can be modeled as heavy tailed. However, at the cost of such noise robustness, the…
Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.
problem Denoising autoregressive signals corrupted by heavy-tailed noise.
method Self-supervised learning approach without requiring full noise distribution knowledge.
result Strong denoising performance compared to baseline methods, especially for impulsive noise.
Adaptive denoising models adjust the number of steps based on noise level.
problem Generating data with lower intrinsic dimensions.
method Adaptive diffusion models using Doob's h-transform to terminate at a random time.
result Adaptive models simplify termination to a first-hitting rule, enhancing adaptability.
Noise-resilient method improves Hurst exponent estimation accuracy in noisy data.
problem Noise degrades accuracy of Hurst exponent estimation methods.
method Noise-Controlled ALPHEE (NC-ALPHEE) using wavelet multi-scale analysis and neural network combination.
result NC-ALPHEE consistently outperforms existing techniques in noisy conditions.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
Proposes Neural SDE for better model robustness and generalization.
problem Missing regularization mechanisms in Neural ODE networks.
method Integrates various regularization mechanisms via stochastic noise injection.
result Improves robustness and generalization compared to Neural ODE.
Noise stability improves understanding of Transformer models.
problem Lack of robustness metrics for real-valued domains and junta-like input dependence in modern LLMs.
method Proposed noise stability as a new metric and developed a practical regularization method.
result Noise stability regularization method accelerates training by 35-75%.
Proposes a new model for noisy labels considering multiple labelers and adversarial attacks.
problem Real-world noisy label models with multiple labelers and adversarial attacks.
method Labeler-dependent noise model with adversarial attack vectors.
result State-of-the-art approaches for learning from noisy labels are defeated by adversarial label attacks.
SAP corrects model for label noise by identifying and removing noisy samples.
problem Label corruption degrades model performance; acquiring perfect labels is costly.
method SAP uses SVD to identify and project model weights onto a clean activation space.
result SAP improves model generalization by up to 6% on CIFAR dataset with 25% synthetic corruption.
Noise-resilient optimization on noisy quantum computers.
problem Noise's impact on hybrid quantum-classical optimization.
method Iterative quantum circuit with noise consideration, using Quantum Fisher Information bound.
result Algorithm robustness against different noise strengths.
The learning with privileged information setting has recently attracted a lot of attention within the machine learning community, as it allows the integration of additional knowledge into the training process of a classifier, even when this comes in the form of a data modality that is not available at test time. Here, …
Adaptively preconditions SGLD for faster convergence and better generalization.
problem Pathological curvature in deep network loss landscapes.
method Adaptive estimation of noise parameters to precondition isotropic gradient noise.
result Adaptively preconditioned SGLD achieves faster convergence and generalization equivalent of SGD.
Many data-driven approaches exist to extract neural representations of functional magnetic resonance imaging (fMRI) data, but most of them lack a proper probabilistic formulation. We propose a group level scalable probabilistic sparse factor analysis (psFA) allowing spatially sparse maps, component pruning using automa…
In high-dimensional data, structured noise caused by observed and unobserved factors affecting multiple target variables simultaneously, imposes a serious challenge for modeling, by masking the often weak signal. Therefore, (1) explaining away the structured noise in multiple-output regression is of paramount importanc…
Improved SGD bounds for machine learning models with Markovian noise.
problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.
method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.
result Matching 1/k decay rate for expected suboptimality. New method learns SDEs with structured noise from data.
problem Learning SDEs with structured noise from data.
method Nonparametric framework for drift and diffusion terms.
result Accurately infers low-dimensional interaction kernels.
Study on ReLU regression with Massart noise, achieving exact parameter recovery.
problem Efficiently fitting ReLUs to data in the presence of Massart noise.
method Developed an efficient algorithm for exact parameter recovery under mild assumptions.
result Achieved exact parameter recovery in ReLU regression with Massart noise.
Simple method improves deep classifier accuracy under noisy labels.
problem Training deep classifiers with noisy labels.
method Probabilistic approach using temperature parameterized softmax.
result Improves accuracy, log-likelihood and calibration on noisy datasets.
New model shows neural networks can use noise to improve long-tailed data classification.
problem Understanding overfitting in neural networks with long-tailed data.
method Refined feature-noise data model incorporating class-dependent heterogeneous noise.
result Neural networks can leverage data noise to learn implicit features improving long-tailed data classification.
Policy gradient methods find Nash equilibrium in noisy games.
problem Finding Nash equilibrium in noisy games.
method Policy gradient methods with noise added.
result Policy gradient methods converge to Nash equilibrium in noisy games.
SGLRW improves robustness of stochastic gradient MCMC methods.
problem Sensitivity to minibatch size and gradient noise in stochastic-gradient MCMC methods.
method Proposes Stochastic Gradient Lattice Random Walk (SGLRW) with lattice-based discretization.
result SGLRW remains stable in regimes where SGLD fails, including heavy-tailed gradient noise.
Interpolating noisy data in linear regression leads to zero training error.
problem Understanding why deep neural networks generalize well with noisy data.
method Investigated overparameterized linear regression, analyzing generalization error and proposing a hybrid scheme.
result Interpolating solutions in noisy data can generalize well, with error decaying to zero with more features.
GDiff tackles blind denoising with Gibbs sampling and Monte Carlo inference.
problem Blind denoising of signals with unknown noise parameters.
method Gibbs Diffusion (GDiff) method that alternates sampling steps from a conditional diffusion model and a Monte Carlo sampler.
result GDiff achieves blind denoising of natural images and cosmic microwave background data.
The paper explores selecting the parameter α for Fermat distance to balance geometry and noise.
problem Choosing the optimal parameter α for Fermat distance to navigate geometry and noise.
method Theoretical and simulation studies to determine the best α value.
result An optimal α value is identified to balance geometry and noise.