Interpolating label noise makes models vulnerable to adversarial attacks.
problem Adversarial vulnerability of models trained on noisy labels.
method Theoretical analysis of label noise and adversarial risk relationship.
result Uniform label noise induces adversarial risk similar to worst-case poisoning.
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
Adding noise controls capacity of function compositions.
problem Large capacity of function compositions with bounded capacity classes.
method Adding Gaussian noise to the output of F before composing with H. result Noise effectively controls the capacity of H∘F, offering a general recipe for modular design. PCA++ improves robustness to background noise in contrastive learning.
problem Recovering shared signal subspaces from positive pairs in high-dimensional data with structured background noise.
method PCA++ uses hard uniformity-constrained contrastive learning to enforce identity covariance on projected features.
result PCA++ outperforms standard PCA and alignment-only PCA+ in simulations and real-world datasets.
New bounds for kernel regression under non-Gaussian noise.
problem Uncertainty quantification for function estimates from noisy observations.
method Novel non-asymptotic probabilistic uniform error bounds for kernel-based regression.
result Proposed bounds apply to a broad class of non-Gaussian noise distributions.
In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that 0-1 Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; un…
In active learning, the user sequentially chooses values for feature X and an oracle returns the corresponding label Y. In this paper, we consider the effect of feature noise in active learning, which could arise either because X itself is being measured, or it is corrupted in transmission to the oracle, or the o…
There has been significant study on the sample complexity of testing properties of distributions over large domains. For many properties, it is known that the sample complexity can be substantially smaller than the domain size. For example, over a domain of size n, distinguishing the uniform distribution from distrib…
Uniform TD(0) bound derived for function approximation with Markov noise.
problem Uniform concentration bound for TD(0) with function approximation.
method Contractive stochastic approximation, martingale and Markov noises, Poisson equation, relaxed concentration inequalities.
result Uniform all-time concentration bound for TD(0) with linear function approximation.
Unified framework for discrete diffusion modeling with flexible noising processes.
problem Efficient modeling of large discrete state spaces with arbitrary corruption dynamics.
method Generalized Discrete Diffusion from Snapshots (GDDS) framework that supports uniformization for fast noising and snapshot-based ELBO for reverse process.
result GDDS outperforms existing discrete diffusion methods in training efficiency and generation quality.
Noise-Aware Conformal Prediction (NACP) calibrates CP for noisy labels.
problem Calibrating Conformal Prediction with noisy labels.
method Estimate conformal threshold from noisy labels using uniform noise coverage guarantee.
result Finite sample coverage guarantee for uniform noise remains effective in high-class tasks.
We present a novel method for neural network quantization that emulates a non-uniform k-quantile quantizer, which adapts to the distribution of the quantized parameters. Our approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We suggest to compare the results as …
This letter presents an improved version of diffusion least mean ppower (LMP) algorithm for distributed estimation. Instead of sum of mean square errors, a weighted sum of mean square error is defined as the cost function for global and local cost functions of a network of sensors. The weight coefficients are updated b…
This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extracting maximum likelihood estimates from corrupted or incomplete data. The convergence speed-up is an e…
We study the robustness of classifiers to various kinds of random noise models. In particular, we consider noise drawn uniformly from the ℓ_p ball for p∈[1,∞] and Gaussian noise with an arbitrary covariance matrix. We characterize this robustness to random noise in terms of the distance to the decisio…
SymNoise improves language model fine-tuning by 6.7% over NEFTune, using symmetric noise.
problem Improving performance of language models through noise-based fine-tuning.
method Introducing SymNoise, a new fine-tuning method using symmetric noise in embeddings.
result SymNoise increases model performance by 69.04% on AlpacaEval compared to NEFTune's 64.69%.
New rigidity theorem for product of lattices.
problem Understanding quasi-isometry of product lattices.
method Demonstrated rigidity for product of non-uniform rank one lattice and nilpotent lattice.
result Any quasi-isometric group is an extension of a non-uniform rank one lattice by a nilpotent lattice.
New method solves group synchronization with cycle-edge message passing.
problem Solving group synchronization with adversarial or uniform corruption and small noise.
method Cycle-edge message passing procedure using cycle consistency information.
result Exact recovery and linear convergence guarantees under adversarial corruption.
Unified stability bounds for noisy SGD across convex and non-convex losses.
problem Deriving generalization bounds for noisy stochastic gradient descent.
method Unified approach using Lyapunov functions and applied probability.
result Time-uniform stability bounds for SGD on various loss functions.
Spectral algorithm recovers community structure in sparse hypergraphs.
problem Community detection in sparse random hypergraphs with community structure and higher-order interactions.
method Spectral algorithm with three steps: hyperedge selection, spectral partition, and correction/merging.
result Weak consistency achieved for weak signal-to-noise ratio.
New bounds for learning polynomial surrogates with L∞ guarantees.
problem Learning polynomial surrogates for bounded binary functions with L∞ error guarantees. method Characterized minimax sample complexity for two classes of polynomials under subgaussian noise.
result Sample complexity rates differ from noiseless case, scaling as nd+1 for degree d polynomials and ns2 for sparse polynomials. SGD handles label noise with bounds improving over SGLD.
problem Label noise in non-convex optimization.
method Stochastic gradient descent with uniform dissipativity and smoothness conditions, using Wasserstein distance and algorithmic stability.
result Generalization error bounds with a rate of n−2/3, better than SGLD's n−1/2. A bandit algorithm reduces regret in noisy, communication-constrained feedback.
problem Distributed stochastic multi-armed bandit with noisy, communication-constrained feedback.
method Proposes a multi-phase bandit algorithm, UE-UCB++, that matches an information-theoretic lower bound.
result Matches an information-theoretic lower bound of Ω(√(KT/σ²)) on the minimax regret.
Method generates random numbers from sensor noise, improving accuracy and speed.
problem Improving accuracy and speed of Monte Carlo integration.
method Sampling a physical process in a controlled environment.
result Reduces error of Monte Carlo integration by 10^68 times while doubling speed.
Based on the stochastic model proposed by Patriarca-Kaski-Chakraborti that describes the exchange of wealth between n economic agents, we analyze the evolution of the corresponding economies under the assumption of a Gaussian background, modeling the exchange parameter ε. We demonstrate, that within Gaussian noise,…
Label smoothing improves model performance even with noisy labels.
problem Mitigating label noise in deep learning models.
method Examined label smoothing as a technique to cope with label noise and compared it to loss-correction methods.
result Label smoothing is competitive with loss-correction techniques under label noise and beneficial for distillation from noisy data.
New method learns nonlinear systems from single finite trajectory samples.
problem Learning stabilizable nonlinear systems from single finite trajectory samples.
method Gradient-based algorithms with noise-sensitive uniform convergence guarantees.
result Efficient learning of general nonlinear systems with high accuracy and small sample complexity.
Gaussian multiplicative noise is commonly used as a stochastic regularisation technique in training of deterministic neural networks. A recent paper reinterpreted the technique as a specific algorithm for approximate inference in Bayesian neural networks; several extensions ensued. We show that the log-uniform prior us…
We study classification problems where features are corrupted by noise and where the magnitude of the noise in each feature is influenced by the resources allocated to its acquisition. This is the case, for example, when multiple sensors share a common resource (power, bandwidth, attention, etc.). We develop a method f…
This paper investigates the ability of generative networks to convert their input noise distributions into other distributions. Firstly, we demonstrate a construction that allows ReLU networks to increase the dimensionality of their noise distribution by implementing a "space-filling" function based on iterated tent ma…
Develops statistical confidence sets for multidimensional scaling.
problem Statistical uncertainty in multidimensional scaling of noisy data.
method Formal statistical framework, distributional convergence results, uniform confidence sets, bootstrap procedures.
result Construction of reliable confidence sets for latent configurations in multidimensional scaling.
We assume data independently sampled from a mixture distribution on the unit ball of the D-dimensional Euclidean space with K+1 components: the first component is a uniform distribution on that ball representing outliers and the other K components are uniform distributions along K d-dimensional linear subspaces restric…
We present a simple noise-robust margin-based active learning algorithm to find homogeneous (passing the origin) linear separators and analyze its error convergence when labels are corrupted by noise. We show that when the imposed noise satisfies the Tsybakov low noise condition (Mammen, Tsybakov, and others 1999; Tsyb…
The paper analyzes stability of random matrix products with Markovian noise.
problem Analyzing stability of random matrix products with Markovian noise.
method Using a super-Lyapunov drift condition and controlled growth of matrix-valued functions, the paper provides an exponential stability result for the p-th moment of random matrix product.
result Finite-time p-th moment bounds for linear stochastic approximation and TD learning algorithms.
Accurate noise modelling is important for training of deep learning reconstruction algorithms. While noise models are well known for traditional imaging techniques, the noise distribution of a novel sensor may be difficult to determine a priori. Therefore, we propose learning arbitrary noise distributions. To do so, th…
Study recovers Riemannian quantities from noisy data densities.
problem Recovering geometric structure from noisy data on submanifolds.
method Derive uniform small-noise expansions of noisy density and its derivatives; construct estimators for tangent spaces, intrinsic dimension, and second fundamental form.
result Fundamental Riemannian quantities identifiable from density derivatives.
Generalizes smoothness conditions for optimization methods.
problem Optimization under non-uniform smoothness conditions.
method Develops a new analysis technique for bounding gradients.
result Obtains convergence rates for gradient descent and Nesterov's method.
Paper tackles SMPC for linear systems with unknown noise distribution.
problem Stochastic MPC for linear systems with chance state constraints and unknown noise distribution.
method Reformulate chance constraints, design robust benchmark SMPC, and develop adaptive SMPC with online noise statistics learning.
result Adaptive SMPC guarantees time-uniform satisfaction of unknown reformulated state constraints with high probability.
Improved SGD bounds for machine learning models with Markovian noise.
problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.
method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.
result Matching 1/k decay rate for expected suboptimality. Study generalizes matrix completion with side info in low noise settings.
problem Matrix completion with side information in low noise conditions.
method Inductive matrix completion with i.i.d. subgaussian noise, uniform sampling, and side information.
result Generalization bounds with noise scaling, convergence to zero, and logarithmic dependence on matrix size.
Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer a lot from label noise. This paper targets at the challenge of robust training at high label noise regimes. The key insight to …
New method detects communities in complex hypergraphs, matching theoretical limits.
problem Detecting communities in non-uniform hypergraphs with varying hyperedge sizes.
method Developed a spectral theory for weighted non-backtracking operators on non-uniform hypergraphs.
result Achieved the Kesten-Stigum bound for weak recovery in a general class of non-uniform HSBMs.
Algorithm learns decision trees from noisy data.
problem Learning stochastic decision trees from corrupted samples.
method Quasipolynomial-time algorithm for adversarial noise.
result Returns a hypothesis with error within 2η+ε of optimal. A low rank matrix X has been contaminated by uniformly distributed noise, missing values, outliers and corrupt entries. Reconstruction of X from the singular values and singular vectors of the contaminated matrix Y is a key problem in machine learning, computer vision and data science. In this paper we show that common…
Principal Component Analysis (PCA) is a method for estimating a subspace given noisy samples. It is useful in a variety of problems ranging from dimensionality reduction to anomaly detection and the visualization of high dimensional data. PCA performs well in the presence of moderate noise and even with missing data, b…
Enhanced stability improves privacy in machine learning.
problem Improving privacy in machine learning training while maintaining accuracy.
method Study of stability in private empirical risk minimization, focusing on strongly-convex loss functions and uniform stability.
result An algorithm with uniform stability of β implies a bound of O(√β) on the scale of noise required for differential privacy.
Efficiently learns halfspaces with malicious noise, near-optimal label complexity.
problem Learning s-sparse halfspaces under malicious label noise. method Active learning algorithm with instance reweighting and empirical risk minimization.
result Near-optimal label complexity of O(slog4d/ε) and noise tolerance Ω(ε). In Deep Learning, Stochastic Gradient Descent (SGD) is usually selected as a training method because of its efficiency; however, recently, a problem in SGD gains research interest: sharp minima in Deep Neural Networks (DNNs) have poor generalization; especially, large-batch SGD tends to converge to sharp minima. It bec…