New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
problem Efficiently recover low-rank matrices from incomplete data.
method Factor group-sparse regularization, related to Schatten-p norms.
result Improved generalization error bounds for Schatten-p norms as p decreases.
Gradient descent in deep matrix factorization favors low-rank solutions, improving recovery accuracy.
problem Understanding the generalization in deep learning models.
method Study of gradient descent over deep linear neural networks for matrix completion and sensing.
result Adding depth enhances an implicit tendency towards low-rank solutions, leading to more accurate recovery.
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of ℓ2-regularized deep matrix factorization/deep linear network training problems with squared-error loss. result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
New insights into how deep models generalize, focusing on matrix factorization.
problem Understanding how deep models generalize and why they work well.
method Using Morse functions and dynamical systems to study implicit regularization.
result Solved a conjecture on implicit regularization in matrix factorization.
Proposes a new method for selecting regularization parameters in sparse precision matrix estimation.
problem Selecting an appropriate regularization parameter for sparse precision matrix estimation.
method Developed a closed-form matrix-valued regularization parameter based on the sampling distribution of optimality conditions.
result The proposed method achieves comparable estimation accuracy and superior support recovery to cross-validation, with significant runtime improvements.
Adaptive regularization methods pre-multiply a descent direction by a preconditioning matrix. Due to the large number of parameters of machine learning problems, full-matrix preconditioning methods are prohibitively expensive. We show how to modify full-matrix adaptive regularization in order to make it practical and e…
In this paper, we develop a relative error bound for nuclear norm regularized matrix completion, with the focus on the completion of full-rank matrices. Under the assumption that the top eigenspaces of the target matrix are incoherent, we derive a relative upper bound for recovering the best low-rank approximation of t…
New nonconvex regularizer speeds up low-rank matrix completion.
problem Low-rank matrix completion with good theoretical and empirical performance.
method Proposes a new nonconvex regularizer with adaptive shrinkage, scalable, and fast optimization.
result Proposed method achieves state-of-the-art recovery performance and is the fastest.
New proof shows norms can't explain deep learning's implicit regularization.
problem Understanding the implicit regularization in deep learning.
method Mathematical proof on matrix factorization problems.
result Implicit regularization drives norms towards infinity, suggesting rank minimization is key.
Solutions to a quadratic matrix equation are linked to strongly regular graphs and multiplicative characters.
problem Solving a specific quadratic matrix equation in Riemannian geometry.
method Constructing nonzero solutions using group rings and multiplicative characters of finite fields.
result Solutions relate to strongly regular graphs and multiplicative characters of finite fields.
Improved covariance matrix estimation for multiple classes with limited data.
problem Estimating covariance matrices for multiple classes with scarce data.
method Coupled regularized sample covariance matrix estimator (RSCM) that combines pooled SCM and scaled identity matrix for regularization.
result The coupled RSCM estimators outperform cross-validation in classification tasks with comparable accuracy but faster computation.
SNN architecture shows gradient descent converges to regularized solution in matrix sensing problems.
problem Understanding implicit regularization in neural networks for matrix sensing.
method Developed Spectral Neural Networks (SNN) for matrix learning problems, rigorously demonstrating implicit regularization.
result Gradient descent converges to the solution of a regularized learning problem in matrix sensing problems.
The paper analyzes error bounds and KL properties for noisy matrix recovery problems.
problem Noisy low-rank matrix recovery problems.
method Squared F-norm regularization, accelerated alternating minimization method.
result Established error bounds and KL properties for critical points and global minimizers.
This paper proposes robust matrix variate regression models with rank constraints and vector regularization.
problem High dimensional and noisy matrix-valued predictors in regression models.
method Rank constraint, vector regularization, alternating projected gradient descent algorithm.
result The proposed method achieves the minimax rate of estimation errors.
Paper tackles low-rank matrix recovery with column ℓ2,0-norm regularization.
problem Low-rank matrix recovery problems with column sparsity constraints.
method Developed alternating majorization-minimization (AMM) methods with extrapolation and hybrid AMM.
result Global convergence analysis and superior performance in matrix completion problems.
Method improves clarity in forecasting spatio-temporal data.
problem Forecasting spatio-temporal data with clarity and interpretability.
method Supervised semi-nonnegative matrix factorization with frequency regularization.
result Method offers clearer interpretability in forecasting spatio-temporal data.
Spectral methods are popular in detecting global structures in the given data that can be represented as a matrix. However when the data matrix is sparse or noisy, classic spectral methods usually fail to work, due to localization of eigenvectors (or singular vectors) induced by the sparsity or noise. In this work, we …
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
DeepVir uses deep matrix factorization to predict antivirals for COVID-19.
problem Predicting effective antivirals for COVID-19 using known drug-virus associations.
method Graphical deep matrix factorization with HyPALM optimization.
result DeepVir outperforms state-of-the-art techniques in predicting antivirals for COVID-19.
AIR-Net adapts low-rank regularization dynamically for better image completion.
problem Fixed low-rank regularization limits adaptability to different images.
method AIR-Net uses adaptive and implicit regularization parameterized by a dynamic Laplacian matrix.
result AIR-Net enhances implicit regularization and outperforms fixed methods in non-uniform missing data scenarios.
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing. Estimation of the underlying matrix from regularization-based approaches promoting low-rankedness, notably nuclear norm regularization, have enjoye…
Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.
problem Estimating noise transition matrix from noisy data.
method Total variation regularization to encourage distinguishable predicted probabilities.
result Consistent estimator of the noise transition matrix under mild assumptions.
Derives adjoint formulas for matrix operations and applies them to specific cases.
problem Computing adjoints for matrix operations and specific matrix types.
method Derives adjoint formulas for matrix operations and applies them to specific cases.
result Closed-form expressions for adjoints in specific matrix types.
Solves weakly supervised regression using low-rank approximations and manifold regularization.
problem Weakly supervised regression with known, unknown, and uncertain labels.
method Combines manifold regularization and low-rank matrix decomposition for optimization.
result Improves solution quality and stability for large datasets.
We consider the problem of matrix completion with side information (\textit{inductive matrix completion}). In real-world applications many side-channel features are typically non-informative making feature selection an important part of the problem. We incorporate feature selection into inductive matrix completion by p…
Paper tackles low-rank matrix recovery with KL property and DC reformulation.
problem Low-rank matrix recovery with coarse rank estimation.
method Adds ℓ2,0-norm and balanced terms to factorized loss function; establishes KL property and DC reformulations. result Establishes KL property of exponent 1/2 for the composite function and its global minimizers. New framework explains why nonconvex methods work well in low-rank matrix estimation.
problem Nonconvex low-rank matrix estimation problems in machine learning.
method Developed a theoretical framework revealing a benign regularizer.
result Nonconvex procedures can behave well due to a disguised convexity.
The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods whi…
The paper tackles system identification via Hankel nuclear norm regularization, improving estimation rates and singular value gaps.
problem Identifying low-order linear systems from limited data.
method Hankel nuclear norm regularization to encourage low-rankness of the Hankel matrix.
result Hankel regularization enables optimal system recovery with fewer observations and better estimation rates.
Unweighted matrix factorization can match or outperform weighted methods in recommender systems.
problem Improving recommendation performance with matrix factorization on implicit feedback data.
method Systematic study of various weighting schemes and matrix factorization algorithms.
result Training with unweighted data can perform comparably to, and sometimes outperform, training with weighted data.
Gradient descent recovers principal components of overparametrized asymmetric matrices without explicit regularization.
problem Asymmetric matrix factorization under overparametrization with minimal rank assumptions.
method Vanilla gradient descent with small random initialization and proper early stopping.
result Gradient descent produces the best low-rank approximation without explicit regularization.
Proposes a new model for image restoration combining deep learning and total variation.
problem Restoring images from limited data with low-rank constraints insufficient.
method Regularized Deep Matrix Factorized (RDMF) model using deep neural network's low-rank bias and total variation.
result Outperforms state-of-the-art models in image restoration from few observations.
In this paper we study general Schatten-p quasi-norm (SPQN) regularized matrix minimization problems. In particular, we first introduce a class of first-order stationary points for them, and show that the first-order stationary points introduced in [11] for an SPQN regularized vector minimization problem are equiva…
Deep tensor factorization benefits from implicit regularization with polynomial growth.
problem Tensor factorization's implicit regularization effect in deep networks is not well understood.
method Investigated the implicit regularization in deep tensor factorization, showing polynomial growth.
result Implicit regularization in deep tensor factorization grows polynomially with depth, improving estimation accuracy and convergence.
Abstract proposes a new categorical approach to quantization of Poisson algebras.
problem Quantization of Poisson algebras.
method Defining quantization categories as subcategories of R-module categories with classical limits.
result Categories of strict deformation quantization, prequantization, and matrix regularization are equivalent, while Poisson enveloping algebra is not.
A new method for community detection in networks is presented.
problem Community detection in network analysis.
method Mixed regularized spectral clustering (Mixed-RSC) based on the regularized Laplacian matrix.
result The method is asymptotically consistent under mild conditions.
The paper studies implicit regularization in over-parameterized models for high-dimensional data.
problem Understanding implicit regularization in over-parameterized models for high-dimensional data.
method The paper designs regularization-free algorithms for the high-dimensional single index model and provides theoretical guarantees for the induced implicit regularization phenomenon.
result The proposed methods achieve minimax optimal statistical rates of convergence and outperform classical methods with explicit regularization.
Regularized EM algorithm improves clustering performance with small sample sizes.
problem Performance reduction in EM algorithm due to small sample size and poorly conditioned covariance matrices.
method Regularized EM algorithm that uses prior knowledge to ensure positive definiteness of covariance matrices.
result The regularized EM algorithm outperforms standard EM in clustering tasks with small sample sizes.
We consider whether algorithmic choices in over-parameterized linear matrix factorization introduce implicit regularization. We focus on noiseless matrix sensing over rank-r positive semi-definite (PSD) matrices in Rn×n, with a sensing mechanism that satisfies restricted isometry properties (RIP)…
We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix X with gradient descent on a factorization of X. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a f…
In this paper, we solve a semi-supervised regression problem. Due to the lack of knowledge about the data structure and the presence of random noise, the considered data model is uncertain. We propose a method which combines graph Laplacian regularization and cluster ensemble methodologies. The co-association matrix of…
Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better recovery performance. However, the resultant optimization pro…
The paper examines how ESG constraints affect portfolio optimization in large datasets.
problem Investment optimization with ESG constraints in large portfolios.
method Asymptotic analysis of out-of-sample Sharpe ratio, regularization matrix estimation, and adaptive portfolio selection.
result The proposed adaptive ESG-constrained portfolio yields a high out-of-sample Sharpe ratio while meeting ESG requirements.
New method uses spectral geometry to improve matrix completion with geometric relations.
problem Matrix completion problems with underlying geometric or topological relations.
method Interprets DMF through spectral geometry to incorporate explicit regularization.
result DMF models can exploit geometric relations, improving performance on real benchmarks.
Proposes a new regularizer for semi-supervised learning on multilayer graphs.
problem Semi-supervised learning on multilayer graphs with labeled and unlabeled data.
method Generalized matrix mean regularizer and matrix-free numerical scheme.
result The regularizer outperforms state-of-the-art methods numerically.
Autoencoders are popular among neural-network-based matrix completion models due to their ability to retrieve potential latent factors from the partially observed matrices. Nevertheless, when training data is scarce their performance is significantly degraded due to overfitting. In this paper, we mit- igate overfitting…
Combining explicit and implicit regularization improves deep learning performance without needing depth.
problem Improving deep learning performance without increasing model complexity.
method Proposes an explicit penalty to mirror implicit regularization bias in adaptive gradient optimizers.
result Single-layer networks can achieve low-rank approximations with similar performance to deep linear networks.