This paper identifies and estimates the label noise transition matrix without ground truth labels.
problem Learning with noisy labels and identifying the noise transition matrix.
method Building on Kruskal's identifiability results, the paper characterizes the identifiability of the label noise transition matrix for the generic case at the instance level.
result The necessity of multiple noisy labels in identifying the noise transition matrix for the generic case at the instance level.
Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.
problem Estimating noise transition matrix from noisy data.
method Total variation regularization to encourage distinguishable predicted probabilities.
result Consistent estimator of the noise transition matrix under mild assumptions.
Dual-T method improves transition matrix estimation in noisy label learning.
problem Large estimation error in noisy class posterior leads to poor transition matrix estimation.
method Introducing an intermediate class to avoid direct estimation of noisy class posterior, factorizing the transition matrix into two easier-to-estimate matrices.
result The dual-T estimator leads to better classification performances.
In label-noise learning, \textit{noise transition matrix}, denoting the probabilities that clean labels flip into noisy labels, plays a central role in building \textit{statistically consistent classifiers}. Existing theories have shown that the transition matrix can be learned by exploiting \textit{anchor points} (i.e…
New method improves robustness of deep learning with noisy labels.
problem Robust deep learning on corrupted labels with noisy samples.
method Meta-transition adaptation through clean meta data guidance.
result More accurate estimation of noise transition matrix and classifier parameters.
Improves detection of low-rank signals from noisy data matrices.
problem Statistical detection of low-rank signals in noisy data matrices.
method Entrywise pre-transforming data matrix for non-Gaussian noise, sharp phase transition thresholds, central limit theorem for linear spectral statistics, hypothesis test.
result Improves detection of low-rank signals from noisy data matrices, generalizing known results.
It is important to learn various types of classifiers given training data with noisy labels. Noisy labels, in the most popular noise model hitherto, are corrupted from ground-truth labels by an unknown noise transition matrix. Thus, by estimating this matrix, classifiers can escape from overfitting those noisy labels. …
Study on signal-plus-noise decomposition in nonlinear spiked random matrices.
problem Nonlinear spiked random matrix models with rank-one signal and noise.
method Signal-plus-noise decomposition and phase transition analysis.
result Identified precise phase transitions in signal components at critical thresholds.
Study on signal recovery from low-rank matrix with sparse noise.
problem Inference of a rank-one signal in the presence of sparse noise.
method Replica method from statistical physics, recursive distributional equations, population dynamics algorithm.
result Critical signal strength for recovery via top eigenvector identified.
Paper tackles instance-dependent label noise by approximating it with part-dependent noise.
problem Learning with instance-dependent label noise is challenging.
method Approximate instance-dependent label noise with part-dependent noise. Use transition matrices for parts to model noise.
result Method outperforms state-of-the-art approaches for instance-dependent label noise.
Study on Langevin dynamics for recovering planted signals in spiked matrix models.
problem Recovering a planted signal in spiked matrix models.
method Path-wise characterization of overlap using integro-differential equations and explicit formula derivation.
result Sharp phase transition in limiting overlap: positive in one regime, zero in another due to injected noise.
Gradient descent solves rank-one matrix estimation problem with detailed time evolution analysis.
problem Estimating a rank-one symmetric matrix corrupted by noise.
method Gradient descent on a sphere, using local versions of the semi-circle law.
result Explicit formulas for the time evolution of the estimator and cost function, revealing phase transitions.
Paper tackles matrix estimation under arbitrary noise, achieving minimax optimality.
problem Noisy low-rank-plus-sparse matrix recovery under arbitrary dependence.
method Incoherent-constrained least-square estimator, novel energy spreading result.
result Achieves minimax optimality in estimating structured Markov transition kernels.
Improves signal detection in non-Gaussian noise using transformed data.
problem Signal detection in rank-one signal-plus-noise data matrices.
method Pre-transforming matrix entries and using linear spectral statistics for hypothesis testing.
result Sharp phase transition of largest eigenvalues in spiked rectangular matrices.
Paper tackles noisy similarity labels for multi-class classification.
problem Learning multi-class classifiers from noisy similarity-labeled data.
method Proposes a method using a noise transition matrix to learn from noisy data.
result Demonstrates superior performance compared to state-of-the-art methods.
The paper studies phase transitions in random matrices and tensor unfolding for detecting signals.
problem Phase transitions in singular values and vectors of large random matrices.
method Analysis of singular values and vectors of long rectangular random matrices, and tensor unfolding algorithm for asymmetric rank-one spiked tensor models.
result An exact threshold for tensor unfolding to detect signals, independent of unfolding procedure.
Optimal spectral method found for inhomogeneous spiked Wigner model.
problem Structured noise in learning scenarios.
method Random matrix theory and spectral analysis.
result Optimal threshold for phase transition in block-structured Wigner model.
Randomized SVD shows phase transitions in noisy data.
problem Noise sensitivity of randomized SVD in large rank matrices.
method Analyzed R-SVD under low-rank signal plus noise model.
result R-SVD exhibits BBP-like phase transition with outliers above detectability threshold.
Estimates rank-one spikes from heavy-tailed noise using self-avoiding walks.
problem Estimating rank-one spikes from heavy-tailed noise.
method Self-avoiding walks to count and estimate the spikes.
result Optimal estimation up to the BBP threshold for heavy-tailed noise.
High-dimensional time series data exist in numerous areas such as finance, genomics, healthcare, and neuroscience. An unavoidable aspect of all such datasets is missing data, and dealing with this issue has been an important focus in statistics, control, and machine learning. In this work, we consider a high-dimensiona…
Paper proposes an alternative to anchor points for learning with noisy labels.
problem Learning with noisy labels is challenging due to inaccurate labels.
method Estimates transition matrix using clusterability condition and noisy labels.
result Estimation of transition matrix is more accurate and efficient than anchor points.
We consider the problem of reconstructing a low rank matrix from noisy observations of a subset of its entries. This task has applications in statistical learning, computer vision, and signal processing. In these contexts, "noise" generically refers to any contribution to the data that is not captured by the low-rank m…
Study on estimating sparse transition matrix of partially-observed VAR with noisy and sparse data.
problem Estimating sparse transition matrix of partially-observed VAR with noisy and sparse data.
method Yule-Walker equation, Dantzig selector, minimax lower bound.
result Near-optimality of the proposed estimator with convergence rate analysis.
New insights show stochastic initialization prevents token clustering in deep Transformers.
problem Understanding token dynamics in deep stochastic Transformers.
method Analysis of deep Transformers with random initialization noise, proving convergence to an interacting-particle system on the sphere.
result Initialization noise prevents token clustering, leading to antipodal formations.
Improved market state classification for risk assessment.
problem Classifying financial market states for better risk assessment.
method Modified selection criteria for market states, clustering optimization, and visualization of correlation matrices.
result Statistically significant results in SP 500 and Nikkei 225 markets.
Study reveals efficient recovery of multi-modal signals via Bayesian methods and sequential learning.
problem Recovering multiple high-dimensional signals from correlated modalities.
method Bayesian Approximate Message Passing and Sequential Curriculum Learning.
result Sequential learning strategy optimally recovers weak signals in multi-modal settings.
The paper explores how low-degree polynomials can detect shuffled linear regression models.
problem Detecting multivariate shuffled linear regression models from independent Gaussian random matrices.
method Investigates the effectiveness of low-degree polynomial algorithms for distinguishing the model from independent Gaussian random matrices.
result Establishes a phase transition phenomenon in the performance of low-degree polynomial algorithms for distinguishing the model.
Study non-asymptotic bounds on correlation in high-dimensional linear systems, revealing invariant subspaces and bottlenecks.
problem Understanding correlation and mixing in high-dimensional linear systems with Gaussian noise.
method Sampling from sub-trajectories, using Talagrand's inequality, and analyzing invariant subspaces.
result Large discrepancy between algebraic and geometric multiplicity leads to bottlenecks between invariant subspaces.
Detect changes in noisy dynamical systems using empirical approximations and finite-sample bounds.
problem Change detection in noisy dynamical systems
method Partition-based empirical approximations and finite-state stationary distribution stability
result Finite-sample bound for empirical stationary density
A method to reduce bias in model-based policy evaluation by shifting operators.
problem Bias in value function computation from noisy estimated models.
method Operator shifting method to reduce the residual norm error.
result The shifting factor is always positive and upper bounded by $1+O\left(1/n
ight)$.
Study improves model robustness in noisy datasets.
problem Instance-specific label noise in robust classification tasks.
method Coordinated Sparse Recovery (CSR) method introduces a collaboration matrix and confidence weights to reduce generalization error.
result CSR and CSR+ significantly reduce generalization error compared to existing methods.
PLS-SVD struggles with missing data in multimodal datasets, showing a phase transition in performance.
problem Missing data in PLS-SVD for multimodal datasets.
method Replica-symmetric analysis of spiked rectangular random matrices with missing entries.
result PLS-SVD performance transitions from uninformative to informative singular vectors at a critical signal-to-noise threshold.
New approach uses random matrix theory to understand tensor estimation performance.
problem Understanding the performance of estimators for low-rank signals in noisy tensors.
method Developed a new approach using random matrix theory to study random tensors.
result Discovered a fixed-point equation that matches the performance of the maximum likelihood estimator.
This paper studies large-scale dynamical networks where the current state of the system is a linear transformation of the previous state, contaminated by a multivariate Gaussian noise. Examples include stock markets, human brains and gene regulatory networks. We introduce a transition matrix to describe the evolution, …
In this paper we study the matrix completion problem: Suppose X∈Rnr×nc is unknown except for a known upper bound r on its rank. By measuring a small number m≪nrnc of elements of X, is it possible to recover X exactly with noise-free measurements, or to construct a good approxi…
The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.
problem Structured matrix estimation under growing ambient dimensions and latent representations.
method Proposes a general transfer framework decomposing target parameters into embedded source components, low-rank innovations, and sparse edits. Develops an anchored alternating projection estimator.
result Establishes deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding improved rates.
A new method handles label noise by leveraging causal information.
problem Label noise degrades deep learning performance.
method Proposes a novel generative approach using a structural causal model.
result Improves classifier performance on label-noise datasets.
Random matrix theory explains transient signal detectability in early-stopped gradient flow.
problem Transient signal detectability in early-stopped gradient flow.
method Random matrix theory applied to gradient flow in a linear teacher-student setting.
result Transient Baik-Ben Arous-Péché (BBP) transition in learning dynamics due to anisotropy and noise.
Study optimal algorithms for recovering signals through inhomogeneous low-rank channels.
problem Recovering signals through an inhomogeneous low-rank matrix channel.
method Derive and analyze an approximate message-passing algorithm (AMP) and a spectral method.
result The AMP iteration matches the conjectured optimal computational phase transition.
Learning with noisy labels, which aims to reduce expensive labors on accurate annotations, has become imperative in the Big Data era. Previous noise transition based method has achieved promising results and presented a theoretical guarantee on performance in the case of class-conditional noise. However, this type of a…
An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive the result. The analysis is conducted through evaluation of the entropy, an expo…
Increasing variance of losses improves learning with noisy labels.
problem Learning with noisy labels and the need to penalize variance of losses.
method Designing regularizers based on the label noise transition matrix to increase variance of losses.
result Increasing variance of losses significantly improves generalization ability.
A method to approximate instance-dependent label noise using instance-confidence embedding.
problem Real-world label noise that depends on individual instances.
method Variational approximation with instance embedding to capture instance-specific label corruption.
result ICE method effectively approximates instance-dependent noise and detects ambiguous instances.
A method uses confidence scores to handle noisy labels for each instance.
problem Learning with noisy labels where each instance's label can randomly change.
method Introduces confidence-scored instance-dependent noise (CSIDN) to estimate transition distributions for each instance.
result Demonstrates the utility and effectiveness of CSIDN through experiments with synthetic and real-world noise.
New method handles complex systems with discontinuous, heavy-tailed noise.
problem Handling discontinuous, heavy-tailed Lévy noise in stochastic systems.
method Developed nonlocal Kramers-Moyal formulas for SDEs with multiplicative Lévy noise.
result Validated framework for discovering interpretable SDE models from data.
There have been several spectral bounds for the percolation transition in networks, using spectrum of matrices associated with the network such as the adjacency matrix and the non-backtracking matrix. However they are far from being tight when the network is sparse and displays clustering or transitivity, which is repr…
New approach to analyze matrix denoising using gradient flow and fixed point equations.
problem Positive semi-definite matrix denoising in extensive-rank and high-dimensional settings.
method Gradient flow and fixed point equations derived from linear pencil techniques of random matrix theory.
result Continuous phase transitions in the extensive-rank and high-dimensional regime.
Study optimizes shared singular subspace estimation from noisy matrices.
problem Estimating shared singular subspaces across multiple noisy matrices.
method Low-rank matrix denoising framework with Stack-SVD and novel estimators.
result Stack-SVD achieves minimax rate-optimality for identical shared subspaces, and novel estimators for partial sharing.