Rank-one measurements limit feasible sets for low-rank PSD matrices.
problem Feasibility of PSD matrices under rank-one measurements.
method Characterization of feasible sets for PSD matrices given rank-one projections.
result Radius of feasible sets determines singleton solution sets for low-rank matrices.
This paper tackles fitting multilevel low rank matrices by addressing three problems.
problem Fitting a given matrix by an MLR matrix in the Frobenius norm.
method Factor fitting, rank allocation, and hierarchical partitioning.
result The proposed methods can fit a given matrix by an MLR matrix in the Frobenius norm.
Low-rank matrices explain data science patterns.
problem Why do data matrices often have low rank?
method A generative model with latent variables and piecewise functions.
result Approximating large matrices with low rank is feasible.
Paper solves low-rank Boolean matrix approximation using integer programming.
problem Finding low-rank approximations to Boolean matrices.
method Integer programming formulation with polynomial variables and constraints.
result First computationally tractable integer programming approach.
New algorithm learns low-rank matrices with linear number of samples.
problem Learning low-rank matrices efficiently in latent-variable applications.
method Proposed algorithm that uses linear number of samples in high dimension.
result Learning kimesk, rank-r, matrices requires $Ω( rac{kr}{ε^2})$ samples. Novel factorization for low-rank matrices in subspaces, improving efficiency.
problem Learning low-rank matrices constrained to subspaces.
method Riemannian manifold optimization with conjugate gradient and trust-region algorithms.
result Efficient algorithms for structured low-rank matrix learning.
Robust PCA method optimizes low-rank matrices with corrupted data.
problem Recover a low-rank matrix from grossly corrupted observations.
method Nonconvex optimization on the manifold of low-rank matrices, using manifold optimization algorithms.
result Proposed algorithms converge to the underlying low-rank matrix linearly with proper initialization.
Algorithm recovers multiple low-rank matrices from unlabeled data.
problem Learning mixtures of low-rank models from unlabelled data.
method Three-stage meta-algorithm that copes with non-convexity and noise.
result Near-optimal sample and computational complexities under Gaussian designs.
New framework solves low-rank optimization problems to certifiable optimality.
problem Low-rank optimization problems with certifiable solutions.
method Mixed-Projection Conic Optimization framework using symmetric projection matrices and outer-approximation algorithms.
result Solves low-rank problems to certifiable optimality, outperforming existing methods.
New method reduces computational cost for nonnegative low rank matrix approximation.
problem Efficiently compute nonnegative low rank matrix approximation for nonnegative matrices.
method Alternating projections onto tangent spaces of fixed rank matrices manifold and nonnegative matrix manifold.
result Sequence converges linearly to optimal solutions, showing better performance in terms of computational time and accuracy.
New method learns compressed transforms with flexible displacement operators.
problem Efficiently representing and learning shift-invariant patterns in neural networks.
method Explicitly learns over displacement operators and low-rank components in LDR matrices.
result Reduces sample complexity and improves model accuracy with fewer parameters.
Paper proposes a method for estimating complex low-rank matrices from phase-only measurements.
problem Estimating complex low-rank matrices from magnitude-only measurements.
method A hierarchical prior model with a Gaussian-Wishart distribution is used to promote low-rankness. A variational EM algorithm is developed to solve the problem.
result The proposed method is less sensitive to initialization and performs well with random initialization.
New algorithms improve RPCA for large matrices with upper rank bounds.
problem Efficiently decompose large matrices into low-rank and sparse parts.
method Combine regularization and matrix multiplication approaches with upper rank bounds.
result Proposed algorithms are faster and more robust than existing methods.
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
RES-PCA efficiently recovers low-rank matrices without precise rank knowledge.
problem Inefficient and computationally expensive RPCA methods.
method RES-PCA, a scalable and linearly efficient RPCA method.
result RES-PCA is faster and more robust than existing scalable methods.
Proposes a new method for high-dimensional data analysis.
problem Sparse PCA limitations in high-dimensional data analysis.
method Low-rank principal eigenmatrix analysis, matricized rank-truncated power method.
result Competitive empirical performance in synthetic data sets.
We address some theoretical guarantees for Schatten-p quasi-norm minimization (p∈(0,1]) in recovering low-rank matrices from compressed linear measurements. Firstly, using null space properties of the measurement operator, we provide a sufficient condition for exact recovery of low-rank matrices. This condition…
Weight Decay induces low-rank weight matrices in neural networks, improving generalization.
problem Improving generalization in neural networks.
method Training ReLU NN with Weight Decay and Stochastic Gradient Descent.
result The weight matrix of a trained NN is approximately rank-two.
A new method for matrix completion identifies low-rank submatrices.
problem Matrix completion for non-low-rank matrices.
method Targeted framework: extract low-rank submatrices, complete separately.
result Significantly smaller reconstruction errors than classical methods.
New algorithms recover low-rank matrices from few noisy projections.
problem Estimating low-rank matrices from rank-one projections with noise.
method Two fast, non-convex algorithms for matrix recovery.
result Proposed algorithms achieve linear convergence and independent sample complexity of condition number.
We develop latent variable models for Bayesian learning based low-rank matrix completion and reconstruction from linear measurements. For under-determined systems, the developed methods are shown to reconstruct low-rank matrices when neither the rank nor the noise power is known a-priori. We derive relations between th…
New algorithm recovers matrices that are both low rank and sparse in rows and columns.
problem Recovering matrices that are simultaneously low rank and row/column sparse.
method Gradient Descent with hard Thresholding (GDT) algorithm to minimize a bi-convex function over a nonconvex set of constraints.
result GDT achieves linear convergence to near optimal solutions with statistical error.
Algorithm recovers sparse and low rank matrix components efficiently.
problem Recovery of sparse and low rank components of matrices.
method Iterative method with adaptive thresholding.
result Algorithm performs well with low run-time and suitable for non-sparse noise.
LDR neural networks reduce space and complexity with high accuracy.
problem Reducing space and computational complexity in large-scale neural networks.
method Formal study of LDR matrices, proving approximation property, error bounds, and proposing training algorithm.
result LDR neural networks achieve high accuracy with significant reduction in space and computational complexity.
Paper tackles matrix completion with a mixture of low-rank matrices.
problem Matrix completion with a mixture of low-rank matrices.
method Generalized matrix completion model (MMC) with theoretical and practical contributions.
result MMC provides a more accurate model for recommender systems and clustering.
New method corrects quantization errors in LLMs using low-rank matrices.
problem Correcting quantization errors in large language models.
method Introducing low-rank weight matrices to correct quantized activations in LLMs.
result Reduces accuracy gap with original model by more than 50% using low-rank matrices.
A distributed algorithm for learning low-rank matrices from large datasets.
problem Learning high-dimensional low-rank matrices from distributed data with trace norm constraint.
method DFW-Trace, a distributed Frank-Wolfe algorithm using power method approximations.
result DFW-Trace achieves sublinear convergence to optimal solutions with few power iterations.
New method improves matrix factorization accuracy and speed.
problem Improving matrix factorization for large, noisy data.
method Introducing generalized round-rank (GRR) for ordinal-valued matrices.
result GRR-based matrices cannot be well approximated by low-rank linear factorization.
LOCUS separates brain network connectivity matrices efficiently.
problem High dimensionality, latent sources, and spurious findings in analyzing brain connectivity matrices.
method LOCUS: low-rank structure with uniform sparsity, iterative Node-Rotation algorithm.
result LOCUS achieves more efficient and accurate source separation for connectivity matrices.
Efficient algorithm estimates low-rank matrices from noisy measurements.
problem Estimating low-rank matrices from noisy linear measurements.
method Stochastic variance-reduced gradient descent algorithm.
result Algorithm converges to the unknown low-rank matrix at a linear rate up to the minimax optimal statistical error.
New bound for neural networks with full-rank weights, independent of network width.
problem Understanding generalization of neural networks with full-rank weight matrices.
method Using Koopman operators to derive a tighter generalization bound for full-rank weight matrices.
result The bound is tighter than existing norm-based bounds when condition numbers are small.
Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability issue arises with tra…
TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.
problem Limited expressivity and generalization of standard LoRA.
method TensorGuide uses a unified tensor-train structure with controlled Gaussian noise to generate correlated low-rank matrices.
result TensorGuide achieves superior accuracy and scalability with fewer parameters compared to standard LoRA and TT-LoRA.
Algorithm compresses large matrices by approximating them as low rank and low precision factors.
problem Efficiently storing and processing large matrices with billions of elements.
method Randomized sketching and quantization of matrix columns to achieve low rank and low precision factorization.
result Achieves compression ratios as low as one bit per matrix coordinate while maintaining or improving performance.
Extends MMF to nonsymmetric matrices for hierarchical structure.
problem Capturing hierarchical structure in nonsymmetric matrices.
method Multiresolution Matrix Factorization (MMF) extended to nonsymmetric matrices.
result Effective for matrix compression tasks, outperforming low-rank methods.
Flora uses random projections to achieve high-rank updates with low memory usage.
problem Excessive memory usage in large neural networks during training.
method Flora approximates LoRA using random projections to enable high-rank updates with sublinear space complexity.
result Flora achieves high-rank updates with significantly reduced memory usage compared to LoRA.
Mirror descent algorithm recovers low-rank matrices in matrix sensing.
problem Matrix sensing with low-rank matrices under certain conditions.
method Discrete-time mirror descent applied to empirical risk with Bregman divergence analysis.
result Mirror descent converges to a matrix minimizing a specific nuclear norm-related quantity.
Algorithm learns a better sketch matrix for low-rank approximations.
problem Efficiently compute low-rank approximations of large matrices.
method Uses a learned sketch matrix instead of random matrix for optimization.
result Learned sketch matrix reduces approximation loss significantly compared to random matrix.
Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.
problem Spurious critical points on the boundary of low-rank matrix manifold.
method Riemannian gradient descent with dynamical low-rank approximation and rescaled gradient flow.
result Riemannian gradient descent escapes some spurious critical points on the boundary of the manifold.
Estimation of low-rank matrices is of significant interest in a range of contemporary applications. In this paper, we introduce a rank-one projection model for low-rank matrix recovery and propose a constrained nuclear norm minimization method for stable recovery of low-rank matrices in the noisy case. The procedure is…
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
New method for tensor recovery with fewer samples.
problem Recovering low-TT-rank tensors from few samples.
method Minimizing a weighted sum of nuclear norms of unfoldings.
result Significantly fewer samples required for recovery.
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
This paper uncovers the low-rank structure of neural network Hessians.
problem Understanding the structure of Hessians in neural networks.
method Proposes a decoupling conjecture to decompose layer-wise Hessians into Kronecker products of smaller matrices.
result Proves the structure of top eigenspaces in 2-layer networks and shows high overlap in top eigenvectors across different models.
New algorithm improves low-rank matrix estimation accuracy.
problem Estimating low-rank matrices with noisy entries.
method Approximate Message Passing (AMP) combined with spectral initialization.
result Achieves Bayes-optimal accuracy above the spectral threshold.
Efficient algorithm for Hadamard decomposition of matrices.
problem Decomposing matrices into low-rank factors efficiently.
method Alternating optimization with SVD-inspired initialization and momentum.
result Significantly improved performance compared to existing methods.
An algorithm finds the maximum entry of a stochastic low-rank matrix from noisy observations.
problem Finding the maximum entry of a stochastic low-rank matrix from sequential observations.
method LowRankElim algorithm, which is a statistical approach to find the maximum entry of a non-negative matrix.
result An upper bound on the regret of $O((K + L) \poly(d) Δ^{-1} \log n)$, where K and L are the number of rows and columns, d is the rank of the matrix, and Δ is the minimum gap. New method solves nonsmooth low-rank matrix optimization problems efficiently.
problem Nonsmooth and low-rank matrix optimization problems in statistics and machine learning.
method Low-rank Extragradient Method with warm-start initialization.
result The extragradient method converges to an optimal solution with rate O(1/t) and requires only two low-rank SVDs per iteration.