Article presents QR and LQ decomposition algorithms for various matrix sizes and ranks.
problem Solving least squares problems in machine learning and computer vision.
method Developed novel matrix backpropagation algorithms for QR and LQ decompositions of different matrix sizes and ranks.
result Numerical stability and computational efficiency of the proposed methods.
A fast algorithm for generalized matrix regression improves machine learning performance.
problem Efficiently solving generalized matrix regression problems in machine learning.
method Utilizes sketching technique to achieve (1+ε) relative error with sketching sizes of order $\cO(ε^{-1/2})$. result The Fast GMR algorithm achieves better performance in symmetric positive definite matrix approximation and single pass singular value decomposition.
New method models matrix time series using tensor CP-decomposition.
problem Modeling matrix time series with reduced complexity.
method One-pass estimation via generalized eigenanalysis and refined projection.
result Component coefficient vectors estimated consistently with certain rates.
BJMD integrates multi-source data with heterogeneous noise using Bayesian inference.
problem Integrating data from multiple sources with different noise levels.
method BJMD uses a Bayesian framework to model noise heterogeneity and develops scalable algorithms for joint matrix decomposition.
result BJMD outperforms state-of-the-art methods in integrating multi-source data with heterogeneous noise.
New matrix approximation method using RBF components for better memory efficiency.
problem Efficiently approximate any real matrix without being symmetric or positive definite.
method Formulate as an optimization problem with gradient descent methods.
result Significantly reduces memory usage for various matrix types.
Proposes D-CDLF for multi-view data decomposition.
problem Uncorrelatedness between common and distinctive latent factors.
method Decomposes data into common, distinctive, and noise components.
result Effective uncorrelatedness between distinctive latent factors from different views.
Efficient algorithm for Hadamard decomposition of matrices.
problem Decomposing matrices into low-rank factors efficiently.
method Alternating optimization with SVD-inspired initialization and momentum.
result Significantly improved performance compared to existing methods.
The paper connects matrix decompositions to Riemannian geometry and optimal transport.
problem Understanding matrix decompositions through geometric perspectives.
method Riemannian geometry, optimal transport, and information geometry.
result New geometric interpretations and gradient flows for matrix decompositions.
Study on signal-plus-noise decomposition in nonlinear spiked random matrices.
problem Nonlinear spiked random matrix models with rank-one signal and noise.
method Signal-plus-noise decomposition and phase transition analysis.
result Identified precise phase transitions in signal components at critical thresholds.
A novel algorithm converges for solving a specific matrix decomposition problem.
problem Nonlinear matrix decomposition with ReLU function for sparse data.
method Introduced a reparametrization of the Latent-RMD model and developed eBCD for convergence proof.
result eBCD converges and outperforms state-of-the-art methods on various data sets.
Improved method for estimating precision matrices without knowing variable order.
problem Estimating precision matrices without knowing the order of variables.
method Combining multiple permutations and thresholding for sparse structure.
result Consistent property established under weak conditions, superior performance in simulations and real data.
Suppose a given observation matrix can be decomposed as the sum of a low-rank matrix and a sparse matrix (outliers), and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent var…
New algorithms accelerate solving nonlinear matrix decomposition with ReLU.
problem Nonlinear matrix decomposition with ReLU function.
method Two new algorithms: A-NMD and 3B-NMD, with adaptive extrapolation and block parametrization.
result Effective algorithms accelerate solving ReLU-NMD problems.
The article develops a method to learn sparse and low rank PARAFAC decomposition robust to noise.
problem Learning sparse and low rank PARAFAC decomposition for tensors with missing values.
method Bayesian model with elastic net regularization, efficient algorithms for large scale problems.
result The method finds true rank and sparse factor matrix robust to noise.
A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
New insights into when NMF decompositions are not unique.
problem Non-identifiability of NMF decompositions.
method Characterization of non-identifiability conditions.
result Characterized when and how non-uniqueness can occur in NMF.
New tensor-based method for estimating stock correlation matrices.
problem Choosing a proper sample period for estimating correlation matrices.
method Slice-Diagonal Tensor (SDT) factorization technique.
result The new method produces a stable correlation matrix unaffected by the sample period.
Derives adjoint formulas for matrix operations and applies them to specific cases.
problem Computing adjoints for matrix operations and specific matrix types.
method Derives adjoint formulas for matrix operations and applies them to specific cases.
result Closed-form expressions for adjoints in specific matrix types.
A method for semi-supervised regression under uncertainty combines graph Laplacian and cluster ensemble.
problem Semi-supervised regression with uncertain data structure and noise.
method Combines graph Laplacian regularization and cluster ensemble methodologies; uses low-rank decomposition of co-association matrix.
result Robust, efficient, and scalable method demonstrated through numerical experiments.
Paper studies nonnegative Tucker decomposition identifiability with sparsity conditions.
problem Identify nonnegative Tucker decomposition factors uniquely.
method Adapting NMF identifiability results, derive procedures using tensor unfoldings or slices.
result Nonnegative Tucker decomposition factors are identifiable under certain sparsity conditions.
Improved SVD for shifted matrices without explicit matrix construction.
problem Efficiently estimating SVD of shifted matrices.
method Shifted Randomized SVD algorithm.
result More efficient matrix factorization and low-rank approximation.
The CUR matrix decomposition is an important extension of Nyström approximation to a general matrix. It approximates any data matrix in terms of a small number of its columns and rows. In this paper we propose a novel randomized CUR algorithm with an expected relative-error bound. The proposed algorithm has the advanta…
Paper introduces MPS for efficient tensor compression and classification.
problem Efficiently compressing and classifying higher-order tensors.
method Matrix Product State (MPS) using successive SVD.
result MPS achieves better classification performance with lower computation cost.
Extends matrix factorization for deviance-based losses with GLM theory.
problem Improving data loss models beyond squared error.
method Adapts GLM theory to matrix factorization for deviance losses.
result Strong consistency and robustness of the proposed decomposition.
ADMM algorithm solves nonlinear matrix decompositions efficiently.
problem Nonlinear matrix decompositions for various applications.
method Alternating Direction Method of Multipliers (ADMM) for nonlinear matrix factorization.
result The method efficiently solves diverse nonlinear matrix decompositions.
A new framework approximates covariance matrices using tree decompositions.
problem Approximating covariance matrices for Gaussian distributions.
method Cascade of tree decompositions with Cholesky factorization.
result The proposed framework guarantees convergence and outperforms KL divergence.
Efficient spectral clustering using autoencoders and landmarks.
problem High computational complexity of spectral clustering.
method Build adjacency matrix using landmarks, define efficient Laplacian matrix, perform eigen decomposition using autoencoder.
result Overall complexity of O(np), where n is data points and p is landmarks. New algorithms for distributed, differentially-private matrix and tensor factorization.
problem Private data distributed across different locations requires privacy-preserving algorithms.
method Distributed and differentially-private algorithms for PCA and OTD using correlated noise.
result Achieves utility matching centralized scenario while maintaining differential privacy.
Variables in many massive high-dimensional data sets are structured, arising for example from measurements on a regular grid as in imaging and time series or from spatial-temporal measurements as in climate studies. Classical multivariate techniques ignore these structural relationships often resulting in poor performa…
A new multi-view clustering method using deep matrix decomposition and partition alignment.
problem Improving multi-view clustering methods to better utilize data representations and view-specific structures.
method Deep matrix decomposition for partition representations, joint use of partition representations, and alternating optimization.
result Demonstrated effectiveness on six benchmark datasets compared to state-of-the-art methods.
Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…
Randomized HALS for efficient NMF on big data.
problem Challenges in computing nonnegative matrix factorization for big data.
method Randomized hierarchical alternating least squares (HALS) algorithm.
result Efficient nonnegative decomposition for big data applications.
This paper reviews methods for discovering patient subgroups from EHR data.
problem Discovering subgroups of patients and co-occurring medical conditions from EHR data.
method Low-rank data approximation methods like matrix and tensor decompositions.
result These methods provide transparent and interpretable insights into patient phenotypes.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
Graphical notation simplifies tensor operations and decompositions.
problem Complex tensor operations are difficult to understand and represent.
method Introduces graphical notation to represent tensor operations.
result Simplified representation of tensor operations and decompositions.
Algorithm learns a better sketch matrix for low-rank approximations.
problem Efficiently compute low-rank approximations of large matrices.
method Uses a learned sketch matrix instead of random matrix for optimization.
result Learned sketch matrix reduces approximation loss significantly compared to random matrix.
SWD approximates GP for faster regression.
problem Costly GP matrix inversion limits large data analysis.
method SWD replaces matrix inversion with diagonalization for grid and short-range correlations.
result SWD-GP outperforms existing methods in predictive accuracy.
We propose an efficient ADMM method with guarantees for high-dimensional problems. We provide explicit bounds for the sparse optimization problem and the noisy matrix decomposition problem. For sparse optimization, we establish that the modified ADMM method has an optimal convergence rate of O(slogd/T), w…
Improved convergence for matrix eigen-decomposition problems.
problem Slow convergence and sub-optimal solutions in matrix eigen-decomposition.
method Deployed variance reduction technique of SGD to Riemannian manifolds.
result Fixed learning rate leads to exponential global convergence rate.
Faster matrix completion through randomized SVD algorithms.
problem Efficiently completing large sparse matrices for applications like image inpainting and recommender systems.
method Proposed two fast randomized algorithms (rSVD-PI and rSVD-BKI) and a new subspace recycling technique to accelerate singular value thresholding (SVT) method.
result The proposed algorithms achieve up to 15X faster computation time for image inpainting and movie rating estimation problems.
We propose an algorithmic framework for convex minimization problems of a composite function with two terms: a self-concordant function and a possibly nonsmooth regularization term. Our method is a new proximal Newton algorithm that features a local quadratic convergence rate. As a specific instance of our framework, w…
Paper proves optimal decomposition for matrix fields, reducing convex integration steps.
problem Optimizing decomposition of symmetric matrix fields for convex integration.
method Algebraic geometry and topology applications to prove optimality.
result Optimal decomposition with fewer rank-one terms, improving Hölder regularity.
Proposes a Bayesian approach for integrating multiple linked matrices.
problem Integrating multiple linked matrices for diverse applications.
method Empirical Bayes Linked Matrix Decomposition (EB-LMD).
result Efficient estimation algorithm with no tuning parameters.
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
This paper is concerned with the problem of low rank plus sparse matrix decomposition for big data. Conventional algorithms for matrix decomposition use the entire data to extract the low-rank and sparse components, and are based on optimization problems with complexity that scales with the dimension of the data, which…
New method for factor analysis using nuclear and ℓ0 norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, ℓ0 norm, and KL divergence. Used alternating minimization algorithm. result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
New method decomposes corrupted data matrices into sparse and low-rank components.
problem Decomposing corrupted data matrices into sparse and low-rank components.
method Discrete optimization approach with alternating minimization, semidefinite relaxation, and branch-and-bound algorithm.
result High-quality solutions and meaningful bounds for SLR problems.
Optimization problems with rank constraints arise in many applications, including matrix regression, structured PCA, matrix completion and matrix decomposition problems. An attractive heuristic for solving such problems is to factorize the low-rank matrix, and to run projected gradient descent on the nonconvex factoriz…