Improved prediction accuracy in matrix factorization using graph-based priors.
problem Graph side-information may not align with latent-feature relations in matrix completion.
method Identify and remove 'contested' edges using graphical lasso approximation, maintaining linear scalability.
result Improved prediction accuracy with fewer graph edges, demonstrating the often inaccurate nature of graph side-information.
DeepVir uses deep matrix factorization to predict antivirals for COVID-19.
problem Predicting effective antivirals for COVID-19 using known drug-virus associations.
method Graphical deep matrix factorization with HyPALM optimization.
result DeepVir outperforms state-of-the-art techniques in predicting antivirals for COVID-19.
Improves scalability and robustness of dynamic graph clustering.
problem Scalability and robustness issues in matrix factorization methods for dynamic graphs.
method Temporal separated matrix factorization, bi-clustering regularization, selective embedding updating.
result Demonstrated scalability, robustness, and effectiveness on synthetic and real-world benchmarks.
Nonnegative Matrix Factorization (NMF) has been continuously evolving in several areas like pattern recognition and information retrieval methods. It factorizes a matrix into a product of 2 low-rank non-negative matrices that will define parts-based, and linear representation of nonnegative data. Recently, Graph regula…
Enhances matrix completion with pairwise penalties for latent features.
problem Improving prediction performance in matrix completion.
method Proposes a general optimization framework with non-/convex pairwise penalty functions and develops an efficient algorithm.
result The proposed framework outperforms standard matrix completion methods, especially in scenarios with latent subgroup structures.
High-dimensional time series prediction is needed in applications as diverse as demand forecasting and climatology. Often, such applications require methods that are both highly scalable, and deal with noisy data in terms of corruptions or missing values. Classical time series methods usually fall short of handling bot…
Graph neural networks speed up nonnegative matrix factorization.
problem Efficiently factorize nonnegative matrices for various applications.
method Developed a graph neural network that combines bipartite self-attention with ADMM updates.
result Significant acceleration achieved in nonnegative matrix factorization.
New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
problem Efficiently recover low-rank matrices from incomplete data.
method Factor group-sparse regularization, related to Schatten-p norms.
result Improved generalization error bounds for Schatten-p norms as p decreases.
Gradient descent in deep matrix factorization favors low-rank solutions, improving recovery accuracy.
problem Understanding the generalization in deep learning models.
method Study of gradient descent over deep linear neural networks for matrix completion and sensing.
result Adding depth enhances an implicit tendency towards low-rank solutions, leading to more accurate recovery.
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of ℓ 2 \ell^2 ℓ 2 -regularized deep matrix factorization/deep linear network training problems with squared-error loss. result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
MISC finds multiple independent clusterings in different subspaces.
problem Difficulties in understanding diverse clusterings.
method Two-stage approach using independent subspace analysis and graph regularized semi-nonnegative matrix factorization.
result MISC discovers different clusterings from independent subspaces.
New insights into how deep models generalize, focusing on matrix factorization.
problem Understanding how deep models generalize and why they work well.
method Using Morse functions and dynamical systems to study implicit regularization.
result Solved a conjecture on implicit regularization in matrix factorization.
Graph diffusion processes approximate manifold heat semigroups using graph transition matrices.
problem Approximating manifold heat semigroups from graph data under low regularity conditions.
method Iterating graph transition matrix P P P to approximate Q t = e t Δ Q_t = e^{tΔ} Q t = e t Δ , bounding error in ∞ \infty ∞ -norm. result Convergence rates O ( N − 2 / ( d + 6 ) ) O(N^{-2/(d+6)}) O ( N − 2/ ( d + 6 ) ) for manifold heat semigroup approximation, valid for in-sample and out-of-sample. Dropout improves matrix factorization by controlling factor size.
problem Understanding regularization properties of dropout for matrix factorization.
method Theoretical analysis of dropout's equivalence to a deterministic model with adaptive dropout rates.
result Dropout's regularization effect is limited by the fixed dropout rate, suggesting adaptive rates.
Proposes a deep Auto-Encoder-like framework for visual-tactile fusion object clustering.
problem Combining visual and tactile information for better object clustering.
method Deep Auto-Encoder-like Non-negative Matrix Factorization framework, graph regularizer, modality-level consensus regularizer, alternating minimization strategy.
result Improves object clustering performance by leveraging both visual and tactile modalities.
Dropout improves matrix factorization by acting as a low-rank regularizer.
problem Improving matrix factorization performance through regularization.
method Using Bernoulli random variables to drop columns of factors, demonstrating equivalence to a deterministic model with sum of squared Euclidean norms.
result Dropout achieves the global minimum of a convex approximation problem with squared nuclear norm regularization.
A new method learns DAGs from Gaussian data without verifying acyclicity.
problem Learning DAGs from Gaussian data without verifying acyclicity.
method Relaxation technique for permutation matrix estimation and cyclic coordinatewise descent for sparse Cholesky factor estimation.
result The method recovers DAGs without verifying acyclicity constraints.
Proposes a method to infer complex network topologies from multiple graphs.
problem Learning multiple graph Laplacian matrices from heterogeneous graph signals with intricate topological patterns.
method Structured fusion regularization and ADMM algorithm for efficient computation.
result Establishes a non-asymptotic bound of the estimation error and reflects the effect of key factors on convergence rate.
Sparse Tucker decomposition with graph regularization improves time series forecasting accuracy.
problem High-dimensional time series forecasting with over-parameterization issue.
method Sparse Tucker decomposition and graph regularization for tensor-based model.
result Non-asymptotic error bound and superior performance in numerical experiments.
Paper tackles low-rank matrix recovery with column ℓ 2 , 0 \ell_{2,0} ℓ 2 , 0 -norm regularization.
problem Low-rank matrix recovery problems with column sparsity constraints.
method Developed alternating majorization-minimization (AMM) methods with extrapolation and hybrid AMM.
result Global convergence analysis and superior performance in matrix completion problems.
The paper introduces a pooling mechanism for graph CNNs using NMF.
problem Pooling in graph structured data for efficient computation.
method Non-negative matrix factorization for node pooling.
result The pooling mechanism improves graph classification performance.
Deep tensor factorization benefits from implicit regularization with polynomial growth.
problem Tensor factorization's implicit regularization effect in deep networks is not well understood.
method Investigated the implicit regularization in deep tensor factorization, showing polynomial growth.
result Implicit regularization in deep tensor factorization grows polynomially with depth, improving estimation accuracy and convergence.
Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix factorization approximates the entries of the matrix by a simple, fixed function-…
Factored gradient descent finds unique rank-r solution in PSD matrix sensing.
problem Finding a unique rank-r solution in over-parameterized matrix sensing.
method Factored gradient descent with PSD constraints.
result PSD constraint alone leads to a unique rank-r matrix recovery.
New proof shows norms can't explain deep learning's implicit regularization.
problem Understanding the implicit regularization in deep learning.
method Mathematical proof on matrix factorization problems.
result Implicit regularization drives norms towards infinity, suggesting rank minimization is key.
Solutions to a quadratic matrix equation are linked to strongly regular graphs and multiplicative characters.
problem Solving a specific quadratic matrix equation in Riemannian geometry.
method Constructing nonzero solutions using group rings and multiplicative characters of finite fields.
result Solutions relate to strongly regular graphs and multiplicative characters of finite fields.
Matrix factorization generates investment recommendations for investors.
problem Generating accurate investment recommendations for investors.
method Used matrix factorization and an iterative conjugate gradient method to optimize investment recommendations.
result Achieved highest average prediction accuracy of 13.3% for investors.
Paper tackles low-rank matrix recovery with KL property and DC reformulation.
problem Low-rank matrix recovery with coarse rank estimation.
method Adds ℓ 2 , 0 \ell_{2,0} ℓ 2 , 0 -norm and balanced terms to factorized loss function; establishes KL property and DC reformulations. result Establishes KL property of exponent 1 / 2 1/2 1/2 for the composite function and its global minimizers. Method improves clarity in forecasting spatio-temporal data.
problem Forecasting spatio-temporal data with clarity and interpretability.
method Supervised semi-nonnegative matrix factorization with frequency regularization.
result Method offers clearer interpretability in forecasting spatio-temporal data.
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
Unweighted matrix factorization can match or outperform weighted methods in recommender systems.
problem Improving recommendation performance with matrix factorization on implicit feedback data.
method Systematic study of various weighting schemes and matrix factorization algorithms.
result Training with unweighted data can perform comparably to, and sometimes outperform, training with weighted data.
The paper analyzes error bounds and KL properties for noisy matrix recovery problems.
problem Noisy low-rank matrix recovery problems.
method Squared F-norm regularization, accelerated alternating minimization method.
result Established error bounds and KL properties for critical points and global minimizers.
We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix X X X with gradient descent on a factorization of X X X . We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a f…
A new framework approximates covariance matrices using tree decompositions.
problem Approximating covariance matrices for Gaussian distributions.
method Cascade of tree decompositions with Cholesky factorization.
result The proposed framework guarantees convergence and outperforms KL divergence.
New distribution simplifies covariance matrix inference.
problem Efficient inference for covariance matrices in large models.
method Incorporates Inverse G-Wishart distribution for variational message passing.
result Elegant and succinct expression of variational message passing fragments.
Observational data usually comes with a multimodal nature, which means that it can be naturally represented by a multi-layer graph whose layers share the same set of vertices (users) with different edges (pairwise relationships). In this paper, we address the problem of combining different layers of the multi-layer gra…
Improves Graph Convolutional Network performance on citation datasets.
problem Improving Graph Convolutional Network performance on citation datasets.
method Exploring graph regularization and alternative graph convolution approaches.
result Explicit graph regularization was incorrectly rejected by Kipf & Welling (2016).
We present a method based on the orthogonal symmetric non-negative matrix tri-factorization of the normalized Laplacian matrix for community detection in complex networks. While the exact factorization of a given order may not exist and is NP hard to compute, we obtain an approximate factorization by solving an optimiz…
Proposes a new regularizer for semi-supervised learning on multilayer graphs.
problem Semi-supervised learning on multilayer graphs with labeled and unlabeled data.
method Generalized matrix mean regularizer and matrix-free numerical scheme.
result The regularizer outperforms state-of-the-art methods numerically.
GLFA improves latent factor analysis by incorporating graph structures for HiDS matrices.
problem Accurate representation learning on high-dimensional and sparse matrices.
method GLFA incorporates a graph to identify hidden high-order interactions and uses a recurrent LFA structure to improve representation learning.
result GLFA outperforms state-of-the-art models in predicting missing data of HiDS matrices.
Time series of graphs are increasingly prevalent in modern data and pose unique challenges to visual exploration and pattern extraction. This paper describes the development and application of matrix factorizations for exploration and time-varying community detection in time-evolving graph sequences. The matrix factori…
SON-NMF estimates nonnegative rank on-the-fly for NMF.
problem Estimating the nonnegative rank of data in NMF.
method Sum-of-norms (SON) regularization to reduce rank, combined with a first-order BCD algorithm.
result SON-NMF can automatically estimate the rank from data without prior knowledge.
L21 SNF compresses mixed-sign data robustly.
problem Compression of mixed-sign data with high fidelity.
method Regularized L21 Semi-NonNegative Matrix Factorization (L21 SNF).
result Rigorous proof of convergence and use-case advantages demonstrated.
Gradient descent implicitly regularizes over-parameterized matrix factorization and neural networks with quadratic activations.
problem Implicit regularization in over-parameterized models with quadratic activations.
method Gradient descent applied to parameterizing U U o p UU^ op U U o p with U ∈ R d i m e s d U\in \mathbb R^{d imes d} U ∈ R d im es d to recover a rank r r r positive semidefinite matrix X ⋆ X^{\star} X ⋆ . result Gradient descent recovers X ⋆ X^{\star} X ⋆ in i l d e O ( r ) ilde{O}(\sqrt{r}) i l d e O ( r ) iterations starting from a small initialization. This work formulates a novel song recommender system as a matrix completion problem that benefits from collaborative filtering through Non-negative Matrix Factorization (NMF) and content-based filtering via total variation (TV) on graphs. The graphs encode both playlist proximity information and song similarity, using …
Gradient descent proves global convergence for 4-layer matrix factorization.
problem Global convergence of gradient descent on four-layer matrix factorization under random initialization.
method New techniques to show saddle-avoidance properties and extend eigenvalue theories.
result Polynomial-time global convergence guarantee for randomly initialized gradient descent on four-layer matrix factorization.
Proposes a new model for image restoration combining deep learning and total variation.
problem Restoring images from limited data with low-rank constraints insufficient.
method Regularized Deep Matrix Factorized (RDMF) model using deep neural network's low-rank bias and total variation.
result Outperforms state-of-the-art models in image restoration from few observations.
Paper shows LDA and SMF have similar generalization errors.
problem LDA and SMF's generalization performance is unknown.
method Algebraic and geometric method to show equivalence of LDA and SMF.
result LDA and SMF have asymptotically same Bayesian generalization error.