arXiv research

Interpolating label noise makes models vulnerable to adversarial attacks.

Paper shows how to integrate quantization into neural compression models.

Adding noise controls capacity of function compositions.

PCA++ improves robustness to background noise in contrastive learning.

New bounds for kernel regression under non-Gaussian noise.

In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that $0$-$1$ Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; un…

In active learning, the user sequentially chooses values for feature $X$ and an oracle returns the corresponding label $Y$. In this paper, we consider the effect of feature noise in active learning, which could arise either because $X$ itself is being measured, or it is corrupted in transmission to the oracle, or the o…

There has been significant study on the sample complexity of testing properties of distributions over large domains. For many properties, it is known that the sample complexity can be substantially smaller than the domain size. For example, over a domain of size $n$, distinguishing the uniform distribution from distrib…

Uniform TD(0) bound derived for function approximation with Markov noise.

Unified framework for discrete diffusion modeling with flexible noising processes.

Noise-Aware Conformal Prediction (NACP) calibrates CP for noisy labels.

We present a novel method for neural network quantization that emulates a non-uniform $k$-quantile quantizer, which adapts to the distribution of the quantized parameters. Our approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We suggest to compare the results as …

This letter presents an improved version of diffusion least mean ppower (LMP) algorithm for distributed estimation. Instead of sum of mean square errors, a weighted sum of mean square error is defined as the cost function for global and local cost functions of a network of sensors. The weight coefficients are updated b…

This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extracting maximum likelihood estimates from corrupted or incomplete data. The convergence speed-up is an e…

We study the robustness of classifiers to various kinds of random noise models. In particular, we consider noise drawn uniformly from the $\ell\_p$ ball for $p \in [1, \infty]$ and Gaussian noise with an arbitrary covariance matrix. We characterize this robustness to random noise in terms of the distance to the decisio…

New rigidity theorem for product of lattices.

SymNoise improves language model fine-tuning by 6.7% over NEFTune, using symmetric noise.

Unified stability bounds for noisy SGD across convex and non-convex losses.

Spectral algorithm recovers community structure in sparse hypergraphs.

We propose a general framework for solving the group synchronization problem, where we focus on the setting of adversarial or uniform corruption and sufficiently small noise. Specifically, we apply a novel message passing procedure that uses cycle consistency information in order to estimate the corruption levels of gr…

New bounds for learning polynomial surrogates with $L_\infty$ guarantees.

SGD handles label noise with bounds improving over SGLD.

A bandit algorithm reduces regret in noisy, communication-constrained feedback.

Based on the stochastic model proposed by Patriarca-Kaski-Chakraborti that describes the exchange of wealth between $n$ economic agents, we analyze the evolution of the corresponding economies under the assumption of a Gaussian background, modeling the exchange parameter $ε$. We demonstrate, that within Gaussian noise,…

Label smoothing improves model performance even with noisy labels.

We study classification problems where features are corrupted by noise and where the magnitude of the noise in each feature is influenced by the resources allocated to its acquisition. This is the case, for example, when multiple sensors share a common resource (power, bandwidth, attention, etc.). We develop a method f…

Gaussian multiplicative noise is commonly used as a stochastic regularisation technique in training of deterministic neural networks. A recent paper reinterpreted the technique as a specific algorithm for approximate inference in Bayesian neural networks; several extensions ensued. We show that the log-uniform prior us…

This paper investigates the ability of generative networks to convert their input noise distributions into other distributions. Firstly, we demonstrate a construction that allows ReLU networks to increase the dimensionality of their noise distribution by implementing a "space-filling" function based on iterated tent ma…

Develops statistical confidence sets for multidimensional scaling.

We assume data independently sampled from a mixture distribution on the unit ball of the D-dimensional Euclidean space with K+1 components: the first component is a uniform distribution on that ball representing outliers and the other K components are uniform distributions along K d-dimensional linear subspaces restric…

We present a simple noise-robust margin-based active learning algorithm to find homogeneous (passing the origin) linear separators and analyze its error convergence when labels are corrupted by noise. We show that when the imposed noise satisfies the Tsybakov low noise condition (Mammen, Tsybakov, and others 1999; Tsyb…

The paper analyzes stability of random matrix products with Markovian noise.

Accurate noise modelling is important for training of deep learning reconstruction algorithms. While noise models are well known for traditional imaging techniques, the noise distribution of a novel sensor may be difficult to determine a priori. Therefore, we propose learning arbitrary noise distributions. To do so, th…

Study recovers Riemannian quantities from noisy data densities.

Generalizes smoothness conditions for optimization methods.

Paper tackles SMPC for linear systems with unknown noise distribution.

Improved SGD bounds for machine learning models with Markovian noise.

Study generalizes matrix completion with side info in low noise settings.

Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer a lot from label noise. This paper targets at the challenge of robust training at high label noise regimes. The key insight to …

New method detects communities in complex hypergraphs, matching theoretical limits.

Algorithm learns decision trees from noisy data.

A low rank matrix X has been contaminated by uniformly distributed noise, missing values, outliers and corrupt entries. Reconstruction of X from the singular values and singular vectors of the contaminated matrix Y is a key problem in machine learning, computer vision and data science. In this paper we show that common…

Principal Component Analysis (PCA) is a method for estimating a subspace given noisy samples. It is useful in a variety of problems ranging from dimensionality reduction to anomaly detection and the visualization of high dimensional data. PCA performs well in the presence of moderate noise and even with missing data, b…

Enhanced stability improves privacy in machine learning.

Efficiently learns halfspaces with malicious noise, near-optimal label complexity.

In Deep Learning, Stochastic Gradient Descent (SGD) is usually selected as a training method because of its efficiency; however, recently, a problem in SGD gains research interest: sharp minima in Deep Neural Networks (DNNs) have poor generalization; especially, large-batch SGD tends to converge to sharp minima. It bec…

PS-IG improves feature attribution by reducing noise and variance.

Paper presents a deep learning method for estimating asset return precision matrices in noisy financial markets.