A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper develops methods for non-quadratic loss low-rank matrix recovery.
New framework solves low-rank optimization problems to certifiable optimality.
New method reduces computational cost for nonnegative low rank matrix approximation.
Flora uses random projections to achieve high-rank updates with low memory usage.
Low-rank structure have been profoundly studied in data mining and machine learning. In this paper, we show a dense matrix 's low-rank approximation can be rapidly built from its left and right random projections and , or bilateral random projection (BRP). We then show power scheme can further…
Estimates joint probability distribution from 1-way marginals using low-rank tensors and random projections.
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
Tensorized random projections reduce high-dimensional tensor size efficiently.
Convex optimization method recovers low-rank matrices from rank-one projections efficiently.
Rank-one measurements limit feasible sets for low-rank PSD matrices.
We consider the problem of estimation of a low-rank matrix from a limited number of noisy rank-one projections. In particular, we propose two fast, non-convex \emph{proper} algorithms for matrix recovery and support them with rigorous theoretical analysis. We show that the proposed algorithms enjoy linear convergence a…
Estimation of low-rank matrices is of significant interest in a range of contemporary applications. In this paper, we introduce a rank-one projection model for low-rank matrix recovery and propose a constrained nuclear norm minimization method for stable recovery of low-rank matrices in the noisy case. The procedure is…
In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…
Seq2Tens uses tensors to efficiently represent sequences, improving performance on time series and video tasks.
We propose a generic framework based on a new stochastic variance-reduced gradient descent algorithm for accelerating nonconvex low-rank matrix recovery. Starting from an appropriate initial estimator, our proposed algorithm performs projected gradient descent based on a novel semi-stochastic gradient specifically desi…
Nonnegative low-rank matrix recovery can have spurious local minima.
Develops precise expressions for random projections for better machine learning tasks.
Low-rank matrix regression refers to the instances of recovering a low-rank matrix based on specially designed measurements and the corresponding noisy outcomes. In the last decade, numerous statistical methodologies have been developed for efficiently recovering the unknown low-rank matrices. However, in some applicat…
Tensorized Rademacher projections outperform Gaussian projections in reducing tensor dimensions.
High-dimensional representations often have a lower dimensional underlying structure. This is particularly the case in many decision making settings. For example, when the representation of actions is generated from a deep neural network, it is reasonable to expect a low-rank structure whereas conventional structures l…
Projection-cost preservation is a low-rank approximation guarantee which ensures that the cost of any rank- projection can be preserved using a smaller sketch of the original data matrix. We present a general structural result outlining four sufficient conditions to achieve projection-cost preservation. These condit…
The paper establishes theoretical foundations for low-rank knowledge distillation in LLMs.
A new method for Bayesian inference tackles high-dimensional problems.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
We develop a new compressive sensing (CS) inversion algorithm by utilizing the Gaussian mixture model (GMM). While the compressive sensing is performed globally on the entire image as implemented in our lensless camera, a low-rank GMM is imposed on the local image patches. This low-rank GMM is derived via eigenvalue th…
We study the projected gradient descent method on low-rank matrix problems with a strongly convex objective. We use the Burer-Monteiro factorization approach to implicitly enforce low-rankness; such factorization introduces non-convexity in the objective. We focus on constraint sets that include both positive semi-defi…
Paper proposes an optimal framework for tensor estimation across various applications.
Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.
LORENZA improves LLM fine-tuning efficiency and generalization.
New approach to convex hulls for low-rank problems.
We revisit the use of Stochastic Gradient Descent (SGD) for solving convex optimization problems that serve as highly popular convex relaxations for many important low-rank matrix recovery problems such as \textit{matrix completion}, \textit{phase retrieval}, and more. The computational limitation of applying SGD to so…
There has recently been considerable interest in completing a low-rank matrix or tensor given only a small fraction (or few linear combinations) of its entries. Related approaches have found considerable success in the area of recommender systems, under machine learning. From a statistical estimation point of view, the…
Proposes a low-rank PGD attack for more efficient adversarial training.
Low-rank inducing unitarily invariant norms have been introduced to convexify problems with low-rank/sparsity constraint. They are the convex envelope of a unitary invariant norm and the indicator function of an upper bounding rank constraint. The most well-known member of this family is the so-called nuclear norm. To …
PLUMAGE improves large model training efficiency and stability.
Proposes a faster Isomap algorithm by reducing eigenvalue decomposition complexity.
Unified theory and debiasing framework for random oblique projections in high dimensions.
We propose a unified framework to solve general low-rank plus sparse matrix recovery problems based on matrix factorization, which covers a broad family of objective functions satisfying the restricted strong convexity and smoothness conditions. Based on projected gradient descent and the double thresholding operator, …
Optimization problems with rank constraints arise in many applications, including matrix regression, structured PCA, matrix completion and matrix decomposition problems. An attractive heuristic for solving such problems is to factorize the low-rank matrix, and to run projected gradient descent on the nonconvex factoriz…
Tensor completion estimates missing components by exploiting the low-rank structure of multi-way data. The recently proposed methods based on tensor train (TT) and tensor ring (TR) show better performance in image recovery than classical ones. Compared with TT and TR, the projected entangled pair state (PEPS), which is…
We propose a new method for robust PCA -- the task of recovering a low-rank matrix from sparse corruptions that are of unknown value and support. Our method involves alternating between projecting appropriate residuals onto the set of low-rank matrices, and the set of sparse matrices; each projection is {\em non-convex…
This paper surveys various methods for dimensionality reduction and nearest neighbor search.
Density matrices are positively semi-definite Hermitian matrices with unit trace that describe the states of quantum systems. Many quantum systems of physical interest can be represented as high-dimensional low rank density matrices. A popular problem in {\it quantum state tomography} (QST) is to estimate the unknown l…
LoRA and privacy: Random projections help but not always.
Low-rank modeling plays a pivotal role in signal processing and machine learning, with applications ranging from collaborative filtering, video surveillance, medical imaging, to dimensionality reduction and adaptive filtering. Many modern high-dimensional data and interactions thereof can be modeled as lying approximat…
The report analyzes Legendre decomposition for tensor data.
Paper projects GP basis functions using tensor networks to reduce complexity.
New tensor recovery method improves efficiency under strict complementarity.