Characterizes uncertainty in low-rank matrix completion with noisy data.
problem Uncertainty quantification in low-rank matrix completion with heterogeneous sub-exponential noise.
method Characterizes the distribution of estimated matrix entries under low-rank estimators with heterogeneous sub-exponential noise.
result Explicit formulas for the distribution of estimated matrix entries under Poisson and Binary noise.
Proposes isotonic recalibration for insurance pricing to ensure auto-calibration under low signal-to-noise ratio.
problem Ensuring auto-calibration in insurance pricing models to prevent cross-financing.
method Applies isotonic recalibration to regression models to achieve auto-calibration.
result Isotonically recalibrated regression functions have low complexity under low signal-to-noise ratio.
Study generalizes matrix completion with side info in low noise settings.
problem Matrix completion with side information in low noise conditions.
method Inductive matrix completion with i.i.d. subgaussian noise, uniform sampling, and side information.
result Generalization bounds with noise scaling, convergence to zero, and logarithmic dependence on matrix size.
We develop a flexible framework for low-rank matrix estimation that allows us to transform noise models into regularization schemes via a simple bootstrap algorithm. Effectively, our procedure seeks an autoencoding basis for the observed matrix that is stable with respect to the specified noise model; we call the resul…
Physen-Noise2Noise tackles defocus deblurring in low-light conditions with physics-guided self-supervised learning.
problem Defocus deblurring in low-light conditions with complex biased noise.
method Physics-guided self-supervised deblurring framework that leverages noisy multi-frame observations and a learnable noise bias parameter.
result Physen-Noise2Noise consistently outperforms state-of-the-art methods in defocus deblurring with complex biased noise.
The paper analyzes deflation for estimating a low-rank spike in large tensors with noise.
problem Estimating a low-rank symmetric spike in large tensors with additive Gaussian noise.
method Characterization of deflation performance in terms of vector alignments and weights.
result Understanding deflation mechanism in noisy conditions and designing more efficient methods.
Paper improves privacy in SGD with low noise, achieving optimal risk rates.
problem Privacy-preserving machine learning with good performance.
method Differentially private SGD with low-noise analysis.
result Achieves optimal excess risk rates for non-smooth losses.
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
This paper studies the classification of high-dimensional Gaussian signals from low-dimensional noisy, linear measurements. In particular, it provides upper bounds (sufficient conditions) on the number of measurements required to drive the probability of misclassification to zero in the low-noise regime, both for rando…
Principal Component Analysis (PCA) has wide applications in machine learning, text mining and computer vision. Classical PCA based on a Gaussian noise model is fragile to noise of large magnitude. Laplace noise assumption based PCA methods cannot deal with dense noise effectively. In this paper, we propose Cauchy Princ…
Manually labeled corpora are expensive to create and often not available for low-resource languages or domains. Automatic labeling approaches are an alternative way to obtain labeled data in a quicker and cheaper way. However, these labels often contain more errors which can deteriorate a classifier's performance when …
DOPPLER optimizes DP training with low-pass filtering, improving model accuracy.
problem Privacy concerns in deep learning models and performance degradation of DP optimizers.
method Developed DOPPLER, a low-pass filter for DP optimizers, to reduce privacy noise and enhance model quality.
result DOPPLER optimizers outperform non-DOPPLER counterparts by 3%-10% in test accuracy.
Robust method learns nonlinear structures robustly to noise.
problem Learning nonlinear structures in noisy data.
method Robust Non-Linear Matrix Factorization (RNLMF).
result RNLMF achieves noticeable improvements in denoising and clustering.
Image denoising techniques are essential to reducing noise levels and enhancing diagnosis reliability in low-dose computed tomography (CT). Machine learning based denoising methods have shown great potential in removing the complex and spatial-variant noises in CT images. However, some residue artifacts would appear in…
This paper studies least-square regression penalized with partly smooth convex regularizers. This class of functions is very large and versatile allowing to promote solutions conforming to some notion of low-complexity. Indeed, they force solutions of variational problems to belong to a low-dimensional manifold (the so…
The paper analyzes side effects of learning from low-dimensional data embedded in a Euclidean space.
problem Learning from data distributed in a linear subspace of high-dimensional space.
method Derives estimates on the variation of the learning function and studies regularization effects.
result Potential regularization effects associated with network depth and noise in codimension of data manifold.
The paper develops a method to model high-dimensional data with many variables and weak signals.
problem Modeling high-dimensional dependent data with many explanatory variables and low signal-to-noise ratio.
method Penalized regression for high-dimensional data, factor modeling of residuals, high-dimensional white noise testing, projected Principal Component Analysis.
result Established asymptotic properties of the proposed method for high-dimensional data.
Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.
problem PAC learning γ-margin halfspaces with Random Classification Noise.
method Established an information-computation tradeoff and provided a simple efficient algorithm with sample complexity O(1/(γ^2 ε^2)). Also, proved lower bounds for SQ algorithms and low-degree polynomial tests.
result Inherent gap between sample complexity and computational efficiency for learning halfspaces with random noise.
Study on signal recovery from low-rank matrix with sparse noise.
problem Inference of a rank-one signal in the presence of sparse noise.
method Replica method from statistical physics, recursive distributional equations, population dynamics algorithm.
result Critical signal strength for recovery via top eigenvector identified.
This paper describes a fast algorithm for recovering low-rank matrices from their linear measurements contaminated with Poisson noise: the Poisson noise Maximum Likelihood Singular Value thresholding (PMLSV) algorithm. We propose a convex optimization formulation with a cost function consisting of the sum of a likeliho…
A new algorithm improves both computational efficiency and statistical optimality for robust low-rank matrix and tensor estimation.
problem Challenges in low-rank matrix estimation under heavy-tailed noise, both computationally and statistically.
method Riemannian sub-gradient (RsGrad) algorithm, which is computationally efficient and statistically optimal.
result RsGrad achieves linear convergence and statistical optimality for robust loss functions under Gaussian and heavy-tailed noise.
Principal component analysis (PCA) is one of the most widely used dimension reduction and multivariate statistical techniques. From a probabilistic perspective, PCA seeks a low-dimensional representation of data in the presence of independent identical Gaussian noise. Probabilistic PCA (PPCA) and its variants have been…
Paper tackles robust matrix completion with heavy-tailed noise.
problem Estimating a low-rank matrix from noisy incomplete data.
method Adaptive Huber loss for robustness, nonconvex algorithm with spectral initialization.
result Achieves minimax-optimal statistical estimation error under bounded second moment condition.
Latent FxLMS accelerates ANC by adapting along low-dimensional filter weights.
problem Improving active noise control with neural adaptive filters.
method Training an auto-encoder on filter coefficients, constraining weights to latent variables, and updating in latent space.
result Latent FxLMS converges in fewer steps with comparable error to standard FxLMS.
New method reduces variance in stochastic optimization with high confidence.
problem Achieving high-probability guarantees in stochastic optimization with weaker noise assumptions.
method Stochastic proximal point method combining proximal subproblem solver and probability booster.
result Demonstrates convergence with low sample complexity under bounded variance assumptions.
Study assesses CNN model robustness to noise in low-cost CT scans.
problem Evaluate CNN model performance on noisy, artifact-prone low-cost CT images.
method Developed and tested a CNN model for head CT triage, varying tube current and projections.
result Model remains robust to reduced tube current and fewer projections, maintaining AUROC close to original.
Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …
Improves detection of low-rank signals from noisy data matrices.
problem Statistical detection of low-rank signals in noisy data matrices.
method Entrywise pre-transforming data matrix for non-Gaussian noise, sharp phase transition thresholds, central limit theorem for linear spectral statistics, hypothesis test.
result Improves detection of low-rank signals from noisy data matrices, generalizing known results.
Study disproves conjecture about low-degree polynomials in hypothesis testing.
problem Conjecture about limitations of polynomial-time algorithms in hypothesis testing.
method Used counterexamples to refute the conjecture and modified the conjecture to rule out the counterexample.
result Disproved conjecture about limitations of low-degree polynomials in hypothesis testing.
We consider the problem of reconstructing a low rank matrix from noisy observations of a subset of its entries. This task has applications in statistical learning, computer vision, and signal processing. In these contexts, "noise" generically refers to any contribution to the data that is not captured by the low-rank m…
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the num…
Paper tackles matrix estimation under arbitrary noise, achieving minimax optimality.
problem Noisy low-rank-plus-sparse matrix recovery under arbitrary dependence.
method Incoherent-constrained least-square estimator, novel energy spreading result.
result Achieves minimax optimality in estimating structured Markov transition kernels.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.
problem Robust matrix completion with sparse noise.
method Alternates between projected gradient step for low-rank and thresholding step for sparse noise.
result Achieves linear convergence for general thresholding functions.
BDDMs eliminate noise conditioning in diffusion models, simplifying training and sampling.
problem Noise conditioning in diffusion models is ad hoc and requires unprincipled noise embeddings.
method Introduce blind denoising diffusion models (BDDMs) that do not require noise conditioning.
result BDDMs simplify training and sampling by eliminating noise conditioning.
The problem of Poisson denoising appears in various imaging applications, such as low-light photography, medical imaging and microscopy. In cases of high SNR, several transformations exist so as to convert the Poisson noise into an additive i.i.d. Gaussian noise, for which many effective algorithms are available. Howev…
Robust principal component analysis (RPCA) can recover low-rank matrices when they are corrupted by sparse noises. In practice, many matrices are, however, of high-rank and hence cannot be recovered by RPCA. We propose a novel method called robust kernel principal component analysis (RKPCA) to decompose a partially cor…
LAAT detects multiple low-density manifolds in noisy data.
problem Detecting multiple low-density manifolds in noisy data.
method Locally Aligned Ant Technique (LAAT) based on Ant Colony Optimization.
result LAAT recovers multiple manifolds in extremely noisy data.
A new method inflates and deflates data manifolds to estimate densities without losing universality.
problem Density estimation on low-dimensional manifolds with non-Euclidean support.
method Inflation-deflation approach using Normalizing Flows with added noise.
result Exact estimation of densities on manifolds with sufficient conditions and Gaussian noise approximation.
Classical convolutional neural networks (cCNNs) are very good at categorizing objects in images. But, unlike human vision which is relatively robust to noise in images, the performance of cCNNs declines quickly as image quality worsens. Here we propose to use recurrent connections within the convolutional layers to mak…
New findings on computational limits for estimating hidden structures.
problem Estimating hidden structures in noisy data.
method Use of low-degree polynomials as a restricted model of computation.
result Established low-degree hardness of recovery problems for easy detection problems.
Many models for sparse regression typically assume that the covariates are known completely, and without noise. Particularly in high-dimensional applications, this is often not the case. This paper develops efficient OMP-like algorithms to deal with precisely this setting. Our algorithms are as efficient as OMP, and im…
We tackle anomaly detection in sparse time series data.
problem Sparse time series with low signal-to-noise ratios and non-uniform performance.
method We introduce a novel generative procedure for benchmark datasets and demonstrate how anomaly score smoothing improves performance.
result Anomaly score smoothing consistently improves performance in low-count time series anomaly detection.
A method merges two pretrained diffusion experts to improve image quality and likelihood.
problem Trade-off between image quality and data likelihood in diffusion models.
method Combining two pretrained diffusion experts by switching between them along the denoising trajectory.
result The merged model consistently matches or outperforms its base components, improving or preserving both likelihood and sample quality.
The local linear embedding algorithm (LLE) is a non-linear dimension-reducing technique, widely used due to its computational simplicity and intuitive approach. LLE first linearly reconstructs each input point from its nearest neighbors and then preserves these neighborhood relations in the low-dimensional embedding. W…
New method reduces neural network memorization of noisy labels.
problem Neural networks memorize label noise, leading to poor generalization.
method Proposes an auxiliary network that predicts gradients without labels.
result Reduces memorization of label-noise, improving generalization.
Low precision operations can provide scalability, memory savings, portability, and energy efficiency. This paper proposes SWALP, an approach to low precision training that averages low-precision SGD iterates with a modified learning rate schedule. SWALP is easy to implement and can match the performance of full-precisi…