Low-rank matrix recovery has found many applications in science and engineering such as machine learning, signal processing, collaborative filtering, system identification, and Euclidean embedding. But the low-rank matrix recovery problem is an NP hard problem and thus challenging. A commonly used heuristic approach is…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “matrix norm minimization”
New proof shows norms can't explain deep learning's implicit regularization.
problem Understanding the implicit regularization in deep learning.
method Mathematical proof on matrix factorization problems.
result Implicit regularization drives norms towards infinity, suggesting rank minimization is key.
Recently theoretical guarantees have been obtained for matrix completion in the non-uniform sampling regime. In particular, if the sampling distribution aligns with the underlying matrix's leverage scores, then with high probability nuclear norm minimization will exactly recover the low rank matrix. In this article, we…
The paper analyzes error bounds and KL properties for noisy matrix recovery problems.
problem Noisy low-rank matrix recovery problems.
method Squared F-norm regularization, accelerated alternating minimization method.
result Established error bounds and KL properties for critical points and global minimizers.
Recovering a large matrix from limited measurements is a challenging task arising in many real applications, such as image inpainting, compressive sensing and medical imaging, and this kind of problems are mostly formulated as low-rank matrix approximation problems. Due to the rank operator being non-convex and discont…
Minimizing the nuclear norm of a matrix has been shown to be very efficient in reconstructing a low-rank sampled matrix. Furthermore, minimizing the sum of nuclear norms of matricizations of a tensor has been shown to be very efficient in recovering a low-Tucker-rank sampled tensor. In this paper, we propose to recover…
Paper tackles low-rank matrix recovery with column -norm regularization.
problem Low-rank matrix recovery problems with column sparsity constraints.
method Developed alternating majorization-minimization (AMM) methods with extrapolation and hybrid AMM.
result Global convergence analysis and superior performance in matrix completion problems.
The Schatten quasi-norm can be used to bridge the gap between the nuclear norm and rank function, and is the tighter approximation to matrix rank. However, most existing Schatten quasi-norm minimization (SQNM) algorithms, as well as for nuclear norm minimization, are too slow or even impractical for large-scale problem…
Selective sampling improves matrix completion with known structure.
problem Reconstructing a low-rank matrix with incomplete data.
method Designing observation sets based on matrix structure and selective sampling.
result Improved reconstruction accuracy with selective sampling.
factor analysismath.OC
New method for factor analysis using nuclear and norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, norm, and KL divergence. Used alternating minimization algorithm.
result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
New algorithms improve robust PCA for vision tasks with heavy-tailed distributions.
problem Challenging non-convex, non-smooth, non-Lipschitz problems in robust PCA.
method Bilinear factor matrix norm minimization models with double nuclear and hybrid norms.
result Our methods yield more accurate solutions than original Schatten quasi-norm minimization.
Mirror descent algorithm recovers low-rank matrices in matrix sensing.
problem Matrix sensing with low-rank matrices under certain conditions.
method Discrete-time mirror descent applied to empirical risk with Bregman divergence analysis.
result Mirror descent converges to a matrix minimizing a specific nuclear norm-related quantity.
Study connects covariance cleaning theory to information theory for heavy-tailed distributions.
problem Optimizing covariance matrices for heavy-tailed distributions using information theory.
method Minimizing Frobenius norm and information loss between true and estimated covariance matrices.
result Asymptotic regime of large matrices minimizes information loss for Student's t distributions.
Gradient flow with infinitesimal initialization converges to Greedy Low-Rank Learning for matrix factorization.
problem Understanding implicit regularization in gradient descent for matrix factorization.
method Theoretical and empirical analysis of gradient flow with infinitesimal initialization and Greedy Low-Rank Learning.
result Gradient flow with infinitesimal initialization is mathematically equivalent to Greedy Low-Rank Learning for depth-2 matrix factorization under reasonable assumptions.
Recently, matrix norm has been widely applied to many areas such as computer vision, pattern recognition, biological study and etc. As an extension of vector norm, the mixed matrix norm is often used to find jointly sparse solutions. Moreover, an efficient iterative algorithm has been designed…
For interpolating kernel machines, minimizing the norm of the ERM solution minimizes stabilitystat.ML
Study shows minimizing the norm of the ERM solution stabilizes kernel ridge-less regression.
problem Stability of kernel ridge-less regression.
method Minimizing the norm of the ERM solution to minimize CV stability.
result Interpolating solution with minimum norm minimizes CV stability.
Numerous applications in data mining and machine learning require recovering a matrix of minimal rank. Robust principal component analysis (RPCA) is a general framework for handling this kind of problems. Nuclear norm based convex surrogate of the rank function in RPCA is widely investigated. Under certain assumptions,…
Matrix completion has been well studied under the uniform sampling model and the trace-norm regularized methods perform well both theoretically and numerically in such a setting. However, the uniform sampling model is unrealistic for a range of applications and the standard trace-norm relaxation can behave very poorly …
New ONMF model minimizes KL divergence for better sparse data modeling.
problem Clustering and data modeling with sparse vectors.
method Developed KL-ONMF algorithm based on alternating optimization.
result KL-ONMF outperforms Frobenius-norm ONMF for document classification and hyperspectral image unmixing.
On the Loss Landscape Geometry of Regularized Deep Matrix Factorization: Uniqueness and Sharpnessstat.ML
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of -regularized deep matrix factorization/deep linear network training problems with squared-error loss.
result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
Smoothed Low Rank and Sparse Matrix Recovery by Iteratively Reweighted Least Squares Minimizationcs.LG
This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…
We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…
Paper finds exact Hessian sharpness in deep matrix factorization.
problem Understanding the geometry of loss landscapes in deep matrix factorization.
method Presented the first exact expression for Hessian maximum eigenvalue.
result Spectral-norm balance is a sufficient condition for flatness in deep matrix factorization.
KL property of exponent of -norm and DC regularized factorizations for low-rank matrix recoverymath.OC
Paper tackles low-rank matrix recovery with KL property and DC reformulation.
problem Low-rank matrix recovery with coarse rank estimation.
method Adds -norm and balanced terms to factorized loss function; establishes KL property and DC reformulations.
result Establishes KL property of exponent for the composite function and its global minimizers.
The matrix completion problem consists in reconstructing a matrix from a sample of entries, possibly observed with noise. A popular class of estimator, known as nuclear norm penalized estimators, are based on minimizing the sum of a data fitting term and a nuclear norm penalization. Here, we investigate the case where …
The paper tackles matrix estimation from noisy data, focusing on low-rank matrices.
problem Estimating a low-rank matrix from noisy observations.
method The paper analyzes several estimators, including constrained nuclear-norm minimization, nuclear-norm regularized least squares, and a nonconvex constrained low-rank optimization problem.
result The estimators provide upper error bounds that depend on matrix rank, observed fraction, and matrix sums, and are minimax optimal.
Given a limited number of entries from the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, recovery of the low-rank and sparse components is a fundamental task subsuming compressed sensing, matrix completion, and principal components pursuit. This paper devel…
This paper presents a deterministic method for matrix completion using Ramanujan graphs.
problem Exact and stable recovery of unknown matrices from a small number of measurements.
method Deterministic sampling using asymmetric Ramanujan graphs and constrained nuclear norm minimization.
result The method achieves exact recovery with noise-free measurements and stable approximation with noisy measurements.
Paper develops efficient AltMin algorithm for SRPCP robust matrix recovery.
problem SRPCP model robust matrix recovery with universal penalty parameter.
method Tuning-free alternating minimization (AltMin) algorithm with closed-form subproblems.
result Efficient AltMin algorithm confirms robustness and efficiency.
Gradient flow on softmax attention minimizes nuclear norm of weight matrices.
problem Classification with separate key and query weight matrices.
method Gradient flow on exponential loss, separability assumption, reparameterization, approximate KKT conditions.
result Gradient flow implicitly minimizes nuclear norm of weight matrices, contrasting with Frobenius norm minimization.
In this paper we study general Schatten- quasi-norm (SPQN) regularized matrix minimization problems. In particular, we first introduce a class of first-order stationary points for them, and show that the first-order stationary points introduced in [11] for an SPQN regularized minimization problem are equiva…
New methods improve portfolio risk minimization by estimating covariance matrix more accurately.
problem Uncertainty in estimating covariance matrix leads to unreliable hedge trades.
method Proposes two new estimators of the inverse covariance matrix using l2 and l1 norms.
result Portfolio formed using proposed estimators achieves substantial risk reduction and improved returns.
New SAM method improves model robustness with spectral inner perturbation and Muon optimizer.
problem Improving model robustness to small parameter perturbations.
method Introducing a spectral inner perturbation step in SAM combined with Muon optimizer.
result Spectral inner perturbation combined with Muon optimizer achieves best validation accuracy on ImageNet-1K.
New guarantees for matrix completion from any deterministic sampling patterns.
problem Proving guarantees for low-rank matrix completion from non-random sampling schemes.
method Introduced a graph with observed entries as edges to analyze the performance of constrained nuclear norm minimization algorithm.
result The algorithm can successfully complete the matrix if the observation graph is well-connected and has similar node degrees.
Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.
problem Understanding learnability in overparameterized quadratic neural networks.
method Mapping ERM to convex matrix sensing with nuclear norm penalization.
result Characterization of global minima and precise generalization thresholds.
New algorithm recovers matrices with unknown correspondences.
problem Recovering matrices from observations with unknown correspondences.
method Solves a nuclear norm minimization problem via proximal gradient with a Max-Oracle.
result Achieves state-of-the-art performance and high accuracy in recovering ground-truth correspondences.
In this paper, we investigate the sample size requirement for a general class of nuclear norm minimization methods for higher order tensor completion. We introduce a class of tensor norms by allowing for different levels of coherence, which allows us to leverage the incoherence of a tensor. In particular, we show that …
The Schatten- norm () has been widely used to replace the nuclear norm for better approximating the rank function. However, existing methods are either 1) not scalable for large scale problems due to relying on singular value decomposition (SVD) in every iteration, or 2) specific to some values, e.g., $1/…
Paper shows no spurious local minima in a specific matrix factorization problem.
problem Optimization of -norm rank-one symmetric matrix factorization.
method Second-order variational analysis to study the landscape of the problem.
result Any second-order stationary point is globally optimal.
As surrogate functions of -norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…
Suppose a given observation matrix can be decomposed as the sum of a low-rank matrix and a sparse matrix (outliers), and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent var…
The recent proposed Tensor Nuclear Norm (TNN) [Lu et al., 2016; 2018a] is an interesting convex penalty induced by the tensor SVD [Kilmer and Martin, 2011]. It plays a similar role as the matrix nuclear norm which is the convex surrogate of the matrix rank. Considering that the TNN based Tensor Robust PCA [Lu et al., 2…
The Schatten-p quasi-norm is usually used to replace the standard nuclear norm in order to approximate the rank function more accurately. However, existing Schatten-p quasi-norm minimization algorithms involve singular value decomposition (SVD) or eigenvalue decomposition (EVD) in each iteration, and thus may…
New method improves efficiency of non-convex matrix reconstruction.
problem Reconstructing a low-rank matrix from few linear measurements.
method Factorized gradient descent with spectral initialization.
result Non-convex approaches can match the sample complexity of convex methods.
Beurling-Selberg Extremization for Dual-Blind Deconvolution Recovery in Joint Radar-Communicationscs.IT
Optimal joint separation condition for radar and communications channels in dual-blind deconvolution.
problem Recovering information from overlaid radar and communications signals with unknown channels.
method Extremal functions from Beurling-Selberg interpolation theory for joint separation, nuclear norm minimization for matrix retrieval, and MUSIC for parameter estimation.
result Guaranteed well-conditioned Vandermonde matrix for MUSIC, validating theoretical findings.
We study the column subset selection problem with respect to the entrywise -norm loss. It is known that in the worst case, to obtain a good rank- approximation to a matrix, one needs an arbitrarily large number of columns to obtain a -approximation to the best entrywise -norm low ra…
Low-rank matrix estimation from incomplete measurements recently received increased attention due to the emergence of several challenging applications, such as recommender systems; see in particular the famous Netflix challenge. While the behaviour of algorithms based on nuclear norm minimization is now well understood…
In many applications that require matrix solutions of minimal rank, the underlying cost function is non-convex leading to an intractable, NP-hard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios th…