arXiv research

New algorithms improve robust PCA for vision tasks with heavy-tailed distributions.

Efficiently regularizes deep learning models using Jacobian nuclear norm.

The Schatten-p quasi-norm $(0<p<1)$ is usually used to replace the standard nuclear norm in order to approximate the rank function more accurately. However, existing Schatten-p quasi-norm minimization algorithms involve singular value decomposition (SVD) or eigenvalue decomposition (EVD) in each iteration, and thus may…

The matrix completion problem consists in reconstructing a matrix from a sample of entries, possibly observed with noise. A popular class of estimator, known as nuclear norm penalized estimators, are based on minimizing the sum of a data fitting term and a nuclear norm penalization. Here, we investigate the case where …

Gradient flow on softmax attention minimizes nuclear norm of weight matrices.

Extended Gauss-Markov theorem for linear estimation with bounded bias.

This work simplifies proximal mapping for low-rank norms.

Advances robust principal component analysis with transformed ℓ1 regularization.

The problem of low-rank approximation with convex constraints, which appears in data analysis, system identification, model order reduction, low-order controller design and low-complexity modelling is considered. Given a matrix, the objective is to find a low-rank approximation that meets rank and convex constraints, w…

The paper tackles matrix estimation from noisy data, focusing on low-rank matrices.

Efficiently learns matching rewards in two-sided markets with matrix completion.

Mirror descent algorithm recovers low-rank matrices in matrix sensing.

New nonconvex regularizer speeds up low-rank matrix completion.

Study connects covariance cleaning theory to information theory for heavy-tailed distributions.

Spectral gradient methods outperform Euclidean in certain deep learning scenarios.

Unified framework for coupled tensor completion improves recovery accuracy.

Study on tensor nuclear norm's decomposability and subdifferential.

Low-rank matrix recovery has found many applications in science and engineering such as machine learning, signal processing, collaborative filtering, system identification, and Euclidean embedding. But the low-rank matrix recovery problem is an NP hard problem and thus challenging. A commonly used heuristic approach is…

Minimizing the nuclear norm of a matrix has been shown to be very efficient in reconstructing a low-rank sampled matrix. Furthermore, minimizing the sum of nuclear norms of matricizations of a tensor has been shown to be very efficient in recovering a low-Tucker-rank sampled tensor. In this paper, we propose to recover…

GL-LowPopArt improves minimax-optimal estimation for trace regression.

A new tensor p-shrinkage nuclear norm improves low-rank tensor completion.

Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations. Nuclear norm regularization is the prevailing convexifying technique for dealing …

Recently theoretical guarantees have been obtained for matrix completion in the non-uniform sampling regime. In particular, if the sampling distribution aligns with the underlying matrix's leverage scores, then with high probability nuclear norm minimization will exactly recover the low rank matrix. In this article, we…

New methods for estimating panel regression models with interactive fixed effects.

New method speeds up nuclear-norm constrained learning over multiple machines.

In this paper, we consider the Tensor Robust Principal Component Analysis (TRPCA) problem, which aims to exactly recover the low-rank and sparse components from their sum. Our model is based on the recently proposed tensor-tensor product (or t-product). Induced by the t-product, we first rigorously deduce the tensor sp…

Recovering a large matrix from limited measurements is a challenging task arising in many real applications, such as image inpainting, compressive sensing and medical imaging, and this kind of problems are mostly formulated as low-rank matrix approximation problems. Due to the rank operator being non-convex and discont…

Estimation of low-rank matrices is of significant interest in a range of contemporary applications. In this paper, we introduce a rank-one projection model for low-rank matrix recovery and propose a constrained nuclear norm minimization method for stable recovery of low-rank matrices in the noisy case. The procedure is…

New method for factor analysis using nuclear and $\ell_0$ norms.

Characterizes dropout's regularizer in deep linear networks.

Numerous applications in data mining and machine learning require recovering a matrix of minimal rank. Robust principal component analysis (RPCA) is a general framework for handling this kind of problems. Nuclear norm based convex surrogate of the rank function in RPCA is widely investigated. Under certain assumptions,…

Rank minimization has attracted a lot of attention due to its robustness in data recovery. To overcome the computational difficulty, rank is often replaced with nuclear norm. For several rank minimization problems, such a replacement has been theoretically proven to be valid, i.e., the solution to nuclear norm minimiza…

New algorithms solve tensor problems with random components using SDP.

We analyze a class of estimators based on convex relaxation for solving high-dimensional matrix decomposition problems. The observations are noisy realizations of a linear transformation $\mathfrak{X}$ of the sum of an approximately) low rank matrix $Θ^\star$ with a second matrix $Γ^\star$ endowed with a complementary …

Paper optimizes private PCA for covariance estimation in statistics.

In this paper, we investigate the sample size requirement for a general class of nuclear norm minimization methods for higher order tensor completion. We introduce a class of tensor norms by allowing for different levels of coherence, which allows us to leverage the incoherence of a tensor. In particular, we show that …

Method provides bounds for sparse PCA and nuclear norm problems.

Let $A:[0,1]\rightarrow\mathbb{H}_m$ (the space of Hermitian matrices) be a matrix valued function which is low rank with entries in Hölder class $Σ(β,L)$. The goal of this paper is to study statistical estimation of $A$ based on the regression model $\mathbb{E}(Y_j|τ_j,X_j) = \langle A(τ_j), X_j \rangle,$ where $τ_j$ …

New ONMF model minimizes KL divergence for better sparse data modeling.

Recovering a low-rank tensor from incomplete information is a recurring problem in signal processing and machine learning. The most popular convex relaxation of this problem minimizes the sum of the nuclear norms of the unfoldings of the tensor. We show that this approach can be substantially suboptimal: reliably recov…

In recent years, the nuclear norm minimization (NNM) problem has been attracting much attention in computer vision and machine learning. The NNM problem is capitalized on its convexity and it can be solved efficiently. The standard nuclear norm regularizes all singular values equally, which is however not flexible enou…

New method uses nuclear and ℓ1 penalties for matrix regression, improving brain disorder detection.

Paper tackles low-rank matrix recovery with column $\ell_{2,0}$-norm regularization.

Exact partitioning of high-order planted models achieved through convex optimization.

SpINNEr uses matrix regression to analyze brain connectivity, improving accuracy over other methods.

We use convex relaxation techniques to provide a sequence of solutions to the matrix completion problem. Using the nuclear norm as a regularizer, we provide simple and very efficient algorithms for minimizing the reconstruction error subject to a bound on the nuclear norm. Our algorithm iteratively replaces the missing…

Matrix completion works well for smooth non-linear structures, even without low-rank assumptions.

In this paper, we study the problem of approximately computing the product of two real matrices. In particular, we analyze a dimensionality-reduction-based approximation algorithm due to Sarlos [1], introducing the notion of nuclear rank as the ratio of the nuclear norm over the spectral norm. The presented bound has i…