arXiv research

Nonlinear SGD achieves high-probability rates in non-convex optimization with heavy-tailed noise.

S-SGD adds symmetrical noise to weights to avoid sharp minima in deep learning.

Deep networks can handle noisy labels up to a certain threshold.

In this article, we first propose the modified Hannan-Rissanen Method for estimating the parameters of the autoregressive moving average (ARMA) process with symmetric stable noise and symmetric stable generalized autoregressive conditional heteroskedastic (GARCH) noise. Next, we propose the modified empirical character…

Kernel embeddings of distributions and the Maximum Mean Discrepancy (MMD), the resulting distance between distributions, are useful tools for fully nonparametric two-sample testing and learning on distributions. However, it is rarely that all possible differences between samples are of interest -- discovered difference…

The paper analyzes deflation for estimating a low-rank spike in large tensors with noise.

SymNoise improves language model fine-tuning by 6.7% over NEFTune, using symmetric noise.

In recent years, correntropy and its applications in machine learning have been drawing continuous attention owing to its merits in dealing with non-Gaussian noise and outliers. However, theoretical understanding of correntropy, especially in the statistical learning context, is still limited. In this study, within the…

We study the robustness to symmetric label noise of GNNs training procedures. By combining the nonlinear neural message-passing models (e.g. Graph Isomorphism Networks, GraphSAGE, etc.) with loss correction methods, we present a noise-tolerant approach for the graph classification task. Our experiments show that test a…

TrustNet robustly learns noise patterns from trusted data to improve weakly-supervised classification.

New algorithms recover signals robustly against outliers and heavy-tailed noise.

New privacy mechanism reduces error in query results.

Paper tackles small eigen-gap estimation and inference for noisy symmetric matrices.

Novel criterion identifies heteroscedastic noise in causal discovery.

New insights on robust learning under strong noise models.

Symmetrizes loss functions to improve neural network robustness against noisy labels.

A new method, Residual-Permuted Sums, improves confidence region construction for linear regression models.

Doubly-stochastic normalization improves robustness to heteroskedastic noise.

SkewD robustly discovers causal relationships in skewed noise models.

Low-rank matrix factorization (LRMF) has received much popularity owing to its successful applications in both computer vision and data mining. By assuming noise to come from a Gaussian, Laplace or mixture of Gaussian distributions, significant efforts have been made on optimizing the (weighted) $L_1$ or $L_2$-norm los…

New UCB-type algorithms reduce regret bounds for stochastic bandits with heavy and super heavy noise.

Optimal Gaussian noise mechanisms achieve nearly optimal error in unbiased mean estimation.

Training accurate deep neural networks (DNNs) in the presence of noisy labels is an important and challenging task. Though a number of approaches have been proposed for learning with noisy labels, many open issues remain. In this paper, we show that DNN learning with Cross Entropy (CE) exhibits overfitting to noisy lab…

We propose a data-driven approach to quantify the uncertainty of models constructed by kernel methods. Our approach minimizes the needed distributional assumptions, hence, instead of working with, for example, Gaussian processes or exponential families, it only requires knowledge about some mild regularity of the measu…

Using a Bayesian approach, we consider the problem of recovering sparse signals under additive sparse and dense noise. Typically, sparse noise models outliers, impulse bursts or data loss. To handle sparse noise, existing methods simultaneously estimate the sparse signal of interest and the sparse noise of no interest.…

Study recovers tree structure in noisy MRFs with support size 3 or more.

New model accounts for scale variation and noise in pairwise comparisons.

In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that $0$-$1$ Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; un…

Paper proposes a new covariance estimator ensuring positive semi-definite matrices.

Label smoothing improves model performance even with noisy labels.

We present a theoretical analysis of Maximum a Posteriori (MAP) sequence estimation for binary symmetric hidden Markov processes. We reduce the MAP estimation to the energy minimization of an appropriately defined Ising spin model, and focus on the performance of MAP as characterized by its accuracy and the number of s…

Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) as DNNs usually have the high capacity to memorize the noisy labels. In this paper, we find that the test accuracy can be quantitatively characterized in terms of the noise ratio in datasets. In…

SVDD and Deep SVDD improve radar target detection in clutter.

Study explores loss design for decision trees to improve robustness against noisy labels.

Estimates rank-one spikes from heavy-tailed noise using self-avoiding walks.

Efficiently estimates mean of symmetric distributions without moments.

Efficiently refits black box predictions with wild refitting method.

This paper is concerned with the interplay between statistical asymmetry and spectral methods. Suppose we are interested in estimating a rank-1 and symmetric matrix $\mathbf{M}^{\star}\in \mathbb{R}^{n\times n}$, yet only a randomly perturbed version $\mathbf{M}$ is observed. The noise matrix $\mathbf{M}-\mathbf{M}^{\s…

Develops a new method to discover stochastic systems with non-Gaussian noise.

AniDS improves molecular force field modeling by learning anisotropic noise.

We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network $y = A σ(Wx) + ξ,$ where $A,W$ are weight matrices, $ξ$ represents noise, and the number of neurons in the hidden layer is no larger than the input or outp…

Recent reinforcement learning algorithms, though achieving impressive results in various fields, suffer from brittle training effects such as regression in results and high sensitivity to initialization and parameters. We claim that some of the brittleness stems from variance differences, i.e. when different environmen…

We assume data sampled from a mixture of d-dimensional linear subspaces with spherically symmetric distributions within each subspace and an additional outlier component with spherically symmetric distribution within the ambient space (for simplicity we may assume that all distributions are uniform on their correspondi…

Noise titration benchmarks time series forecasting models rigorously.

The paper improves confidence ellipsoids for ridge regression with PAC bounds.

Rotation invariant algorithms fail on sparse problems even with noise.

Tensor decompositions have rich applications in statistics and machine learning, and developing efficient, accurate algorithms for the problem has received much attention recently. Here, we present a new method built on Kruskal's uniqueness theorem to decompose symmetric, nearly orthogonally decomposable tensors. Unlik…

Study detects signals in spiked Wigner models using log likelihood ratio.