arXiv research

AIR-Net adapts low-rank regularization dynamically for better image completion.

This paper develops a new class of nonconvex regularizers for low-rank matrix recovery. Many regularizers are motivated as convex relaxations of the matrix rank function. Our new factor group-sparse regularizers are motivated as a relaxation of the number of nonzero columns in a factorization of the matrix. These nonco…

Reduced-rank method improves least-squares regression under output regularity.

New nonconvex regularizer speeds up low-rank matrix completion.

We prove the $C^{1}$ regularity for a class of abnormal length-minimizers in rank $2$ sub-Riemannian structures. As a consequence of our result, all length-minimizers for rank $2$ sub-Riemannian structures of step up to $4$ are of class $C^{1}$.

Paper tackles low-rank matrix recovery with column $\ell_{2,0}$-norm regularization.

Ranked data appear in many different applications, including voting and consumer surveys. There often exhibits a situation in which data are partially ranked. Partially ranked data is thought of as missing data. This paper addresses parameter estimation for partially ranked data under a (possibly) non-ignorable missing…

This article announces the completion of the classification of rank 4 locally projective polytopes and their quotients. There are seventeen universal locally projective polytopes (nine nondegenerate). Amongst their 441 quotients are a further four (nonuniversal) regular polytopes, and 152 nonregular but section regular…

SVD training reduces DNN rank and computation load without SVD per step.

The paper analyzes implicit regularization in tensor factorization using neural networks.

Paper uses optimal transport-based statistics for change point detection.

FedLoRU improves FL efficiency by using low-rank updates.

New framework explains why nonconvex methods work well in low-rank matrix estimation.

New method improves calibration in multi-output probabilistic models.

There has been an increased interest in multimodal language processing including multimodal dialog, question answering, sentiment analysis, and speech recognition. However, naturally occurring multimodal data is often imperfect as a result of imperfect modalities, missing entries or noise corruption. To address these c…

This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an $\ell_{2,0}$-norm regularized term is added to the factored loss function to reduce the rank adaptively; and…

SGD can jump from high rank minima to low rank minima in DLNs, but not back.

We consider whether algorithmic choices in over-parameterized linear matrix factorization introduce implicit regularization. We focus on noiseless matrix sensing over rank-$r$ positive semi-definite (PSD) matrices in $\mathbb{R}^{n \times n}$, with a sensing mechanism that satisfies restricted isometry properties (RIP)…

Solves weakly supervised regression using low-rank approximations and manifold regularization.

In this paper, we develop a relative error bound for nuclear norm regularized matrix completion, with the focus on the completion of full-rank matrices. Under the assumption that the top eigenspaces of the target matrix are incoherent, we derive a relative upper bound for recovering the best low-rank approximation of t…

ReLU networks implicitly favor low-rank solutions, but not as strongly as linear networks.

A self-supervised debiasing method using rank regularization mitigates spurious correlations in neural networks.

The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.

Develops new oracle inequalities for Gaussian ranking estimators.

We consider the problem of constructing a reduced-rank regression model whose coefficient parameter is represented as a singular value decomposition with sparse singular vectors. The traditional estimation procedure for the coefficient parameter often fails when the true rank of the parameter is high. To overcome this …

We introduce a new framework for optimal transport using Schatten-p regularization to recover low-rank structures.

Algorithm ensures fair ranking by minority groups alongside majority groups.

Rank-one measurements limit feasible sets for low-rank PSD matrices.

Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better recovery performance. However, the resultant optimization pro…

Low-rank modeling has many important applications in computer vision and machine learning. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better empirical performance. However, the resulting optimization problem is much more challengin…

In this paper, we propose a low-rank approximation method based on discrete least-squares for the approximation of a multivariate function from random, noisy-free observations. Sparsity inducing regularization techniques are used within classical algorithms for low-rank approximation in order to exploit the possible sp…

Paper develops methods for non-quadratic loss low-rank matrix recovery.

Improves robustness of high-dimensional regression with rank objective and group lasso regularization.

Unified approach tackles high-dimensional tensor bandits with convex optimization and weakly decomposable regularizers.

Recent developments in linear system identification have proposed the use of non-parameteric methods, relying on regularization strategies, to handle the so-called bias/variance trade-off. This paper introduces an impulse response estimator which relies on an $\ell_2$-type regularization including a rank-penalty derive…

Efficient solver for nonconvex tensor regularization reduces computational cost.

Gradient descent in tensor factorization favors low-rank solutions.

The paper connects neural collapse and low-rank bias in networks with L2 regularization.

Recently, there has been an abundance of works on designing Deep Neural Networks (DNNs) that are robust to adversarial examples. In particular, a central question is which features of DNNs influence adversarial robustness and, therefore, can be to used to design robust DNNs. In this work, this problem is studied throug…

Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…

Gradient flow with infinitesimal initialization converges to Greedy Low-Rank Learning for matrix factorization.

A novel regularizer of the PARAFAC decomposition factors capturing the tensor's rank is proposed in this paper, as the key enabler for completion of three-way data arrays with missing entries. Set in a Bayesian framework, the tensor completion method incorporates prior information to enhance its smoothing and predictio…

Rank minimization (RM) is a wildly investigated task of finding solutions by exploiting low-rank structure of parameter matrices. Recently, solving RM problem by leveraging non-convex relaxations has received significant attention. It has been demonstrated by some theoretical and experimental work that non-convex relax…

Gradient descent promotes low-rank solutions in tensor completion.

Paper proposes a new method to separate low rank and sparse matrices without bias.

In the past decade, sparse and low-rank recovery have drawn much attention in many areas such as signal/image processing, statistics, bioinformatics and machine learning. To achieve sparsity and/or low-rankness inducing, the $\ell_1$ norm and nuclear norm are of the most popular regularization penalties due to their co…

Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations. Nuclear norm regularization is the prevailing convexifying technique for dealing …

GCL-LRR improves node classification in noisy graphs.