arXiv research

New algorithm improves deep learning models' robustness without sacrificing accuracy.

Over the past years Robust PCA has been established as a standard tool for reliable low-rank approximation of matrices in the presence of outliers. Recently, the Robust PCA approach via nuclear norm minimization has been extended to matrices with linear structures which appear in applications such as system identificat…

In this paper, we propose a low-rank approximation method based on discrete least-squares for the approximation of a multivariate function from random, noisy-free observations. Sparsity inducing regularization techniques are used within classical algorithms for low-rank approximation in order to exploit the possible sp…

Develops PRPCA for smooth image recovery combining low-rank and smoothness.

New method solves robust matrix completion using nonlinear equations.

We address the problem of minimizing a convex function over the space of large matrices with low rank. While this optimization problem is hard in general, we propose an efficient greedy algorithm and derive its formal approximation guarantees. Each iteration of the algorithm involves (approximately) finding the left an…

New method normalizes matrix features for robust low-rank approximation.

Sign-RIP improves robust low-rank matrix recovery by preserving norms even with corrupted measurements.

We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building …

IRCUR accelerates RPCA by using CUR decomposition for efficient low rank estimation.

The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of …

LaLoRA prevents forgetting in LoRA fine-tuning.

Efficiently completes low-rank matrices with nearly linear time complexity.

This paper improves neural network compression by using robust low-rank approximations.

Advances robust principal component analysis with transformed ℓ1 regularization.

Proposes coreset method for robust training of neural networks with noisy labels.

DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.

Principal components analysis (PCA) is a well-known technique for approximating a tabular data set by a low rank matrix. Here, we extend the idea of PCA to handle arbitrary data sets consisting of numerical, Boolean, categorical, ordinal, and other data types. This framework encompasses many well known techniques in da…

Paper tackles fair low-rank approximation and column subset selection.

Low-rank approximation is an effective model compression technique to not only reduce parameter storage requirements, but to also reduce computations. For convolutional neural networks (CNNs), however, well-known low-rank approximation methods, such as Tucker or CP decomposition, result in degraded model accuracy becau…

Simplifies solving noisy SDPs for low rank matrix recovery problems.

RCaGP improves robustness and computational efficiency in Gaussian processes.

We study a data model in which the data matrix D can be expressed as D = L + S + C, where L is a low rank matrix, S an element-wise sparse matrix and C a matrix whose non-zero columns are outlying data points. To date, robust PCA algorithms have solely considered models with either S or C, but not both. As such, existi…

Low-rank matrix approximations are often used to help scale standard machine learning algorithms to large-scale problems. Recently, matrix coherence has been used to characterize the ability to extract global information from a subset of matrix entries in the context of these low-rank approximations and other sampling-…

This work improves robustness guarantees for neural networks using low rank representations.

Many learning tasks, such as cross-validation, parameter search, or leave-one-out analysis, involve multiple instances of similar problems, each instance sharing a large part of learning data with the others. We introduce a robust framework for solving multiple square-root LASSO problems, based on a sketch of the learn…

Proposes a low-rank PGD attack for more efficient adversarial training.

Robust PCA is a widely used statistical procedure to recover a underlying low-rank matrix with grossly corrupted observations. This work considers the problem of robust PCA as a nonconvex optimization problem on the manifold of low-rank matrices, and proposes two algorithms (for two versions of retractions) based on ma…

FedLoRU improves FL efficiency by using low-rank updates.

We describe several algorithms for matrix completion and matrix approximation when only some of its entries are known. The approximation constraint can be any whose approximated solution is known for the full matrix. For low rank approximations, similar algorithms appears recently in the literature under different name…

Matrix approximation is a common tool in machine learning for building accurate prediction models for recommendation systems, text mining, and computer vision. A prevalent assumption in constructing matrix approximations is that the partially observed matrix is of low-rank. We propose a new matrix approximation model w…

Efficient SVD algorithm robust to outliers.

The study assesses low-rank approximations in Gaussian Process regression.

Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better recovery performance. However, the resultant optimization pro…

New algorithm for weighted low rank approximation with provable guarantees.

Low-rank modeling has many important applications in computer vision and machine learning. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better empirical performance. However, the resulting optimization problem is much more challengin…

The study assesses low-rank approximations in Gaussian Process regression.

New method improves robust low-rank matrix completion for computer vision.

Paper proposes fast, robust methods for low-rank matrix recovery.

We analyze a class of estimators based on convex relaxation for solving high-dimensional matrix decomposition problems. The observations are noisy realizations of a linear transformation $\mathfrak{X}$ of the sum of an approximately) low rank matrix $Θ^\star$ with a second matrix $Γ^\star$ endowed with a complementary …

Unified approach for robust low rank matrix estimation with adversaries.

Recently, there has been an abundance of works on designing Deep Neural Networks (DNNs) that are robust to adversarial examples. In particular, a central question is which features of DNNs influence adversarial robustness and, therefore, can be to used to design robust DNNs. In this work, this problem is studied throug…

We accelerate the power method for strong low-rank approximation using fast sketching.

A low-rank transformation learning framework for subspace clustering and classification is here proposed. Many high-dimensional data, such as face images and motion sequences, approximately lie in a union of low-dimensional subspaces. The corresponding subspace clustering problem has been extensively studied in the lit…

We develop an efficient algorithm for low-rank approximation with improved approximation guarantees.

Stochastic gradient descent on manifolds improves low-rank approximation.

Paper develops a new weighted low-rank matrix approximation technique.

Unified error analysis for low-rank approximation improves data assimilation performance.