arXiv research

Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.

Study revisits AdaGrad convergence with relaxed noise assumptions.

We investigate the problem of classification in the presence of unknown class-conditional label noise in which the labels observed by the learner have been corrupted with some unknown class dependent probability. In order to obtain finite sample rates, previous approaches to classification with unknown class-conditiona…

Inserting label noise can improve model accuracy and fairness.

SGD handles label noise with bounds improving over SGLD.

Nonlinear SGD achieves high-probability rates in non-convex optimization with heavy-tailed noise.

Paper derives convergence rates and confidence intervals for LSA with Markovian noise.

Compression is at the heart of effective representation learning. However, lossy compression is typically achieved through simple parametric models like Gaussian noise to preserve analytic tractability, and the limitations this imposes on learning are largely unexplored. Further, the Gaussian prior assumptions in model…

We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback. Specifically, we consider binary losses xored with the noise, which is a Bernoulli random variable. We consider both a constant noise r…

SGD's escape rate depends on log loss barrier, not linear loss barrier.

Paper reconciles minimax rates and optimal recovery rates for noisy observations.

We consider classification in the presence of class-dependent asymmetric label noise with unknown noise probabilities. In this setting, identifiability conditions are known, but additional assumptions were shown to be required for finite sample rates, and so far only the parametric rate has been obtained. Assuming thes…

New convergence bounds for online learning with heavy-tailed noise.

Study on the noise in SGD minibatches near local minima.

For binary classification we establish learning rates up to the order of $n^{-1}$ for support vector machines (SVMs) with hinge loss and Gaussian RBF kernels. These rates are in terms of two assumptions on the considered distributions: Tsybakov's noise assumption to establish a small estimation error, and a new geometr…

Classical scaling is shown to be optimal under various noisy conditions.

This work addresses various open questions in the theory of active learning for nonparametric classification. Our contributions are both statistical and algorithmic: -We establish new minimax-rates for active learning under common \textit{noise conditions}. These rates display interesting transitions -- due to the inte…

Unified framework for isotropic SG noise in posterior sampling.

The paper analyzes convergence rates for stochastic approximation and reinforcement learning.

New bandit algorithm for non-i.i.d. noise, improving standard rates.

We consider the problem of Probably Approximate Correct (PAC) learning of a binary classifier from noisy labeled examples acquired from multiple annotators (each characterized by a respective classification noise rate). First, we consider the complete information scenario, where the learner knows the noise rates of all…

We formulate and study a general family of (continuous-time) stochastic dynamics for accelerated first-order minimization of smooth convex functions. Building on an averaging formulation of accelerated mirror descent, we propose a stochastic variant in which the gradient is contaminated by noise, and study the resultin…

Peer loss functions learn from noisy labels without noise rate specification.

Study shows multi-distribution learning has slower rates than single-task learning.

Bayes classifier cannot be learned from noisy labels without knowing noise distribution.

Paper introduces $ρ$-Perfect to estimate model-human correlation in subjective datasets.

New method improves false-/true-positive-rate estimation in fraud detection with noisy labels.

Noise enhancement improves generalization in training.

Momentum methods such as Polyak's heavy ball (HB) method, Nesterov's accelerated gradient (AG) as well as accelerated projected gradient (APG) method have been commonly used in machine learning practice, but their performance is quite sensitive to noise in the gradients. We study these methods under a first-order stoch…

Study on sensor fusion algorithms under high dimensional noise.

Noise affects the effectiveness of interpolating models, especially those with strong inductive biases.

In this paper, we study a classification problem in which sample labels are randomly corrupted. In this scenario, there is an unobservable sample with noise-free labels. However, before being observed, the true labels are independently flipped with a probability $ρ\in[0,0.5)$, and the random label noise can be class-co…

Study large deviations rates for SGD with strongly convex functions.

Study efficient learning of halfspaces with constant noise tolerance.

Ridge regression performs optimally in noisy environments with heavy-tailed distributions.

We studied non-dynamical stochastic resonance for the number of trades in the stock market. The trade arrival rate presents a deterministic pattern that can be modeled by a cosine function perturbed by noise. Due to the nonlinear relationship between the rate and the observed number of trades, the noise can either enha…

Poly-GNNs achieve similar performance regardless of depth, highlighting graph noise's dominance.

We introduce a new approach for designing computationally efficient learning algorithms that are tolerant to noise, and demonstrate its effectiveness by designing algorithms with improved noise tolerance guarantees for learning linear separators. We consider both the malicious noise model and the adversarial label nois…

New algorithm learns halfspaces with adversarial noise efficiently.

GD-trained shallow ReLU nets learn Lipschitz functions with noise.

This article studies the achievable guarantees on the error rates of certain learning algorithms, with particular focus on refining logarithmic factors. Many of the results are based on a general technique for obtaining bounds on the error rates of sample-consistent classifiers with monotonic error regions, in the real…

Paper develops Euler scheme for fractional delay diff. eqs with additive noise.

AdaGrad converges under heavy-tailed noise without extra operations.

Adapts SGD to noise and problem specifics for faster convergence.

Paper studies Adam's convergence under relaxed assumptions, proving a rate of O(poly(log T)/sqrt(T)).

New bounds for SA with arbitrary norm contractions and Markovian noise.

The Sampled Gaussian Mechanism's noise level decreases with larger subsampling rates, improving privacy-utility trade-offs.

Label noise in SGD helps converge to flatter minima.