Wave maps with noise can lead to self-similar blowup from arbitrary initial data.
problem Analyzing self-similar blowup in wave maps with additive noise.
method Stochastic perturbation of wave maps in supercritical dimensions.
result Self-similar blowup with positive probability for arbitrary corotational initial data.
Accurate noise modelling is important for training of deep learning reconstruction algorithms. While noise models are well known for traditional imaging techniques, the noise distribution of a novel sensor may be difficult to determine a priori. Therefore, we propose learning arbitrary noise distributions. To do so, th…
New bounds for SA with arbitrary norm contractions and Markovian noise.
problem Finite-time analysis of two-time-scale stochastic approximation with arbitrary norm contractions and Markovian noise.
method Use of generalized Moreau envelope for arbitrary norm contractions and solutions of Poisson equation for Markovian noise.
result Mean square error decays at rates of O(1/n2/3) and O(1/n) under different conditions. We consider the problem of learning from distributed data in the agnostic setting, i.e., in the presence of arbitrary forms of noise. Our main contribution is a general distributed boosting-based procedure for learning an arbitrary concept space, that is simultaneously noise tolerant, communication efficient, and compu…
Unified framework for discrete diffusion modeling with flexible noising processes.
problem Efficient modeling of large discrete state spaces with arbitrary corruption dynamics.
method Generalized Discrete Diffusion from Snapshots (GDDS) framework that supports uniformization for fast noising and snapshot-based ELBO for reverse process.
result GDDS outperforms existing discrete diffusion methods in training efficiency and generation quality.
New approach to neural networks by incorporating observation noise and arbitrary prior means.
problem Misspecification on noisy data and limitations of NTK-GP equivalence.
method Introducing a regularizer for observation noise and proposing a shifted network for arbitrary prior means.
result Removes key obstacles to practical Gaussian process modeling in neural networks.
Paper tackles matrix estimation under arbitrary noise, achieving minimax optimality.
problem Noisy low-rank-plus-sparse matrix recovery under arbitrary dependence.
method Incoherent-constrained least-square estimator, novel energy spreading result.
result Achieves minimax optimality in estimating structured Markov transition kernels.
SGD-trained neural networks generalize well even with adversarial label noise.
problem Generalization of neural networks trained on adversarial label noise.
method Training a one-hidden-layer neural network with SGD on arbitrary width networks.
result SGD-trained networks achieve classification accuracy competitive with the best halfspace over adversarial label noise.
Developed LQ MFG theory with common noise, proving existence and uniqueness.
problem Linear-quadratic mean field games with common noise.
method Coupled forward-backward stochastic evolution equations (FBSEEs) in Hilbert spaces.
result Existence and uniqueness of solutions for small and arbitrary finite time horizons.
Study nonparametric factor analysis with arbitrary noise.
problem Identify latent variables in noisy, non-invertible settings.
method Developed a general framework and estimation methods.
result Identify latent variables up to certain indeterminacies.
We introduce a novel method to combat label noise when training deep neural networks for classification. We propose a loss function that permits abstention during training thereby allowing the DNN to abstain on confusing samples while continuing to learn and improve classification performance on the non-abstained sampl…
WS diffusion models handle anisotropic Gaussian noise better than conventional methods.
problem Handling anisotropic Gaussian noise in imaging inverse problems.
method Whitened Score (WS) diffusion models based on stochastic differential equations.
result WS DMs outperform conventional DMs on anisotropic Gaussian noise.
We consider the non-parametric regression problem under Huber's ε-contamination model, in which an ε fraction of observations are subject to arbitrary adversarial noise. We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up t…
We study the robustness of classifiers to various kinds of random noise models. In particular, we consider noise drawn uniformly from the ℓ_p ball for p∈[1,∞] and Gaussian noise with an arbitrary covariance matrix. We characterize this robustness to random noise in terms of the distance to the decisio…
Noise in SGD affects overparameterized models, favoring sparse solutions.
problem Understanding and mitigating implicit bias in SGD with parameter-dependent noise.
method Theoretical analysis of a quadratically-parameterized model with label noise and Gaussian noise.
result SGD with label noise recovers sparse ground-truth solutions, while SGD with Gaussian noise overfits dense solutions.
We present a noise-injected version of the Expectation-Maximization (EM) algorithm: the Noisy Expectation Maximization (NEM) algorithm. The NEM algorithm uses noise to speed up the convergence of the EM algorithm. The NEM theorem shows that injected noise speeds up the average convergence of the EM algorithm to a local…
We solve image inverse problems using a flow-based noise model.
problem Image inverse problems with complex noise patterns.
method Normalizing flow prior for maximum a posteriori estimation.
result Empirical validation on various inverse problems.
We consider the robust phase retrieval problem of recovering the unknown signal from the magnitude-only measurements, where the measurements can be contaminated by both sparse arbitrary corruption and bounded random noise. We propose a new nonconvex algorithm for robust phase retrieval, namely Robust Wirtinger Flow to …
We consider the mixed regression problem with two components, under adversarial and stochastic noise. We give a convex optimization formulation that provably recovers the true solution, and provide upper bounds on the recovery errors for both arbitrary noise and stochastic noise settings. We also give matching minimax …
New error bounds for noisy phase retrieval problems using empirical risk minimization.
problem Estimating signals in noisy phase retrieval problems.
method Empirical ℓ2 risk minimization (ERM) with new error bounds for different noise patterns. result Established new error bounds for NPR and NGPR, showing improved performance under various noise conditions.
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
Algorithm learns binary function efficiently under arbitrary covariate shift.
problem Learning binary function under arbitrary distributions P and Q.
method PQ-learning algorithm using reliable learner with selective classification.
result Polynomial-time algorithm for covariate shift learning.
Compression is at the heart of effective representation learning. However, lossy compression is typically achieved through simple parametric models like Gaussian noise to preserve analytic tractability, and the limitations this imposes on learning are largely unexplored. Further, the Gaussian prior assumptions in model…
Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
problem Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
method A perturbed form of gradient descent with arbitrary initialization.
result Gradient descent with noise converges to a unique optimum.
Numerous empirical evidence has corroborated that the noise plays a crucial rule in effective and efficient training of neural networks. The theory behind, however, is still largely unknown. This paper studies this fundamental problem through training a simple two-layer convolutional neural network model. Although trai…
Normalizing flows improve density estimation from noisy data.
problem Estimating underlying density from noisy samples.
method Use normalizing flows for density estimation with arbitrary noise distributions, using amortized variational inference.
result Normalizing flows can outperform Gaussian mixtures for density deconvolution.
Proposes DISCO, the first CVI for density-based clustering with noise.
problem Evaluation of noise assignments in density-based clustering.
method Density-connectivity and Silhouette Coefficient adaptation for noise evaluation.
result DISCO is the first CVI to explicitly assess noise assignments.
The paper analyzes conditions for solving low-rank matrix recovery problems with noisy measurements.
problem Low-rank matrix recovery with corrupted measurements.
method Analysis of the restricted isometry property (RIP) and local search methods.
result Sharp bounds on the maximum distance between local minimizers and the ground truth.
Method constructs confidence regions for linear models with arbitrary predictors.
problem Constructing confidence regions for linear models with non-linear predictors.
method Mixed Integer Linear Programming for constraints.
result Empty confidence regions for hypothesis testing.
We study active learning of homogeneous s-sparse halfspaces in Rd under the setting where the unlabeled data distribution is isotropic log-concave and each label is flipped with probability at most η for a parameter η∈[0,21), known as the bounded noise. Even in the presence of mild la…
CMRM improves robustness in noisy label settings without requiring privileged knowledge.
problem Learning with noisy labels without privileged knowledge.
method Conformal Margin Risk Minimization (CMRM) framework.
result CMRM consistently improves accuracy and reduces mislabeling under various noise conditions.
GDiff tackles blind denoising with Gibbs sampling and Monte Carlo inference.
problem Blind denoising of signals with unknown noise parameters.
method Gibbs Diffusion (GDiff) method that alternates sampling steps from a conditional diffusion model and a Monte Carlo sampler.
result GDiff achieves blind denoising of natural images and cosmic microwave background data.
A low rank matrix X has been contaminated by uniformly distributed noise, missing values, outliers and corrupt entries. Reconstruction of X from the singular values and singular vectors of the contaminated matrix Y is a key problem in machine learning, computer vision and data science. In this paper we show that common…
A new diffusion model tackles brightness issues with a probabilistic approach.
problem Brightness-related limitations in diffusion models.
method Introduces a novel diffusion model with a probabilistic framework, modifying both forward and reverse diffusion processes.
result The model mitigates brightness-related limitations and improves performance in high-dimensional settings.
Noise balance theory explains SGD's behavior in neural networks.
problem Understanding SGD's navigation in neural network loss landscapes.
method Analyzes minibatch noise and loss function symmetries.
result Derives the stationary distribution of SGD for deep networks.
A new method inflates and deflates data manifolds to estimate densities without losing universality.
problem Density estimation on low-dimensional manifolds with non-Euclidean support.
method Inflation-deflation approach using Normalizing Flows with added noise.
result Exact estimation of densities on manifolds with sufficient conditions and Gaussian noise approximation.
The paper analyzes the implicit bias of SGD near loss manifold and provides new insights.
problem Understanding the implicit bias of SGD near loss manifolds in overparametrized models.
method Adapting ideas from Katzenberger (1991) to analyze SGD dynamics using a stochastic differential equation (SDE).
result SGD with label noise locally decreases the sharpness of loss, leading to a global analysis of implicit bias.
The study sets limits on how well halfspaces can be learned when labels are corrupted.
problem Learning halfspaces in the presence of Massart noise.
method Statistical query (SQ) lower bounds.
result No SQ algorithm can achieve misclassification error better than the corruption rate η with superpolynomial accuracy or a superpolynomial number of queries. This article considers algorithmic and statistical aspects of linear regression when the correspondence between the covariates and the responses is unknown. First, a fully polynomial-time approximation scheme is given for the natural least squares optimization problem in any constant dimension. Next, in an average-case…
In this paper we propose new techniques to sample arbitrary third-order tensors, with an objective of speeding up tensor algorithms that have recently gained popularity in machine learning. Our main contribution is a new way to select, in a biased random way, only O(n1.5/ε2) of the possible n3 elements while s…
Proposes incorporating noise sources in machine learning evaluation for more reliable conclusions.
problem Inadequate handling of nondeterminism in machine learning research leads to unreliable results.
method Uses linear mixed effects models (LMEMs) and generalized likelihood ratio tests (GLRT) to analyze performance evaluation scores and assess performance differences.
result Demonstrates how to incorporate various sources of noise and data properties into statistical significance testing and reliability analysis.
Statistic dynamics of financial systems is investigated, basing on a model of randomly coupled equation system driven by stochastic Langevin force. It is found that in stable regime the noise power spectrum of the system is of 1/f^alpha form, with the exponent alpha=3/2 in case of Hermitian coupling matrices, or slight…
Max-linear regression problem solved with convex programming.
problem Estimating parameters in max-linear regression models.
method Formulated and analyzed a scalable convex program called anchored regression (AR).
result AR provides high probability recovery of parameters with a sample complexity of k4p. New method certifies neural network robustness under random input noise.
problem Certifying neural network robustness against random input noise.
method Chance-constrained optimization problem reformulated with input-output samples, convex conditions developed.
result Proposed method certifies robustness against various input noise regimes over larger uncertainty regions.
The paper extends Pearson correlation to multi-variables, useful for noise measurement and feature selection.
problem The standard Pearson correlation coefficient is limited to two variables and doesn't meet the needs for multi-variable analysis.
method The authors use random matrix theory to extend Pearson's correlation coefficient to an arbitrary number of variables.
result The extended correlation coefficient is useful for gauging noise and selecting features, particularly in classification.
While deep learning is remarkably successful on perceptual tasks, it was also shown to be vulnerable to adversarial perturbations of the input. These perturbations denote noise added to the input that was generated specifically to fool the system while being quasi-imperceptible for humans. More severely, there even exi…
SkewD robustly discovers causal relationships in skewed noise models.
problem Distinguishing cause from effect in skewed noise models.
method SkewD extends normal-distribution framework to skew-normal setting for reliable inference.
result SkewD remains robust under high skewness, improving reliability.
This paper improves convergence guarantees for gradient clipping in deep learning.
problem Improving convergence guarantees for gradient clipping in deep learning models.
method Analyzes and provides precise convergence guarantees for arbitrary clipping thresholds.
result Shows tight convergence guarantees for clipped stochastic gradient descent.