Sparse Singular Value Decomposition (SVD) models have been proposed for biclustering high dimensional gene expression data to identify block patterns with similar expressions. However, these models do not take into account prior group effects upon variable selection. To this end, we first propose group-sparse SVD model…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “sparse SVD”
Ranky solves SVD for large sparse matrices in distributed systems.
problem Rank problem in large sparse matrices for SVD.
method Distributed approach to solve rank problem.
result Recovers SVD with negligible error for large sparse matrices.
Unified framework for structured principal subspace estimation with bounds and rates.
problem Structured principal subspace estimation problems.
method Unified framework, minimax lower and upper bounds, information-geometric complexity.
result Minimax rates of convergence for specific settings, including optimal rates for non-negative PCA/SVD.
K-SVD algorithm has been successfully applied to image denoising tasks dozens of years but the big bottleneck in speed and accuracy still needs attention to break. For the sparse coding stage in K-SVD, which involves constraint, prevailing methods usually seek approximate solutions greedily but are less effe…
In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure featu…
Improved SVD-based Initialization for Nonnegative Matrix Factorization using Low-Rank Correctionmath.NA
Due to the iterative nature of most nonnegative matrix factorization (\textsc{NMF}) algorithms, initialization is a key aspect as it significantly influences both the convergence and the final solution obtained. Many initialization schemes have been proposed for NMF, among which one of the most popular class of methods…
Learning the "blocking" structure is a central challenge for high dimensional data (e.g., gene expression data). Recently, a sparse singular value decomposition (SVD) has been used as a biclustering tool to achieve this goal. However, this model ignores the structural information between variables (e.g., gene interacti…
Paper develops a new test for high-dimensional matrix-valued data.
problem Hypothesis testing for mean of matrix-valued data in high-dimensional settings.
method Proposes a new test statistic for high-dimensional matrix rank testing.
result Develops a novel approach for sparse singular value decomposition (SVD) estimation.
In this paper we consider the dictionary learning problem for sparse representation. We first show that this problem is NP-hard by polynomial time reduction of the densest cut problem. Then, using successive convex approximation strategies, we propose efficient dictionary learning schemes to solve several practical for…
SOFARI improves inference on multi-task learning latent factors.
problem Challenges in precise inference on multi-task learning latent factor matrices.
method High-dimensional manifold-based Neyman near-orthogonality inference on Stiefel manifold structure.
result Easy-to-use bias-corrected estimators for latent factor vectors and singular values with asymptotic normal distributions.
Principal component analysis (PCA) is widely used for dimension reduction and embedding of real data in social network analysis, information retrieval, and natural language processing, etc. In this work we propose a fast randomized PCA algorithm for processing large sparse data. The algorithm has similar accuracy to th…
Dictionary learning and component analysis are part of one of the most well-studied and active research fields, at the intersection of signal and image processing, computer vision, and statistical machine learning. In dictionary learning, the current methods of choice are arguably K-SVD and its variants, which learn a …
New algorithms improve RPCA for large matrices with upper rank bounds.
problem Efficiently decompose large matrices into low-rank and sparse parts.
method Combine regularization and matrix multiplication approaches with upper rank bounds.
result Proposed algorithms are faster and more robust than existing methods.
New sparse penalty improves biclustering for gene expression data.
problem Identifying significant clusters in gene expression data.
method Prenet penalty applied to SSVD for biclustering.
result Mixed Prenet penalty effectively clusters non-overlapped data.
Method interpolates option prices and volatilities without arbitrage.
problem Interpolating option prices and volatilities without arbitrage.
method Sparse modeling approach based on integral equations and SVD.
result Flexible and efficient framework for arbitrage-free interpolation.
A neural network, IHT-Net, improves DOA estimation with sparse arrays.
problem Single-snapshot DOA estimation with sparse arrays in dynamic settings.
method IHT-inspired neural network with recurrent neural network and autoencoders.
result IHT-Net achieves faster convergence and higher accuracy in DOA estimation.
Paper proposes a new method for exact recovery in robust tensor principal component analysis.
problem Exact recovery of low-rank and sparse components in tensors.
method Proposes a new method based on tensor-tensor product and t-SVD to solve a convex optimization problem.
result Exact recovery achieved in a deterministic fashion without randomness assumptions.
Efficient CF approach using fast adaptive PCA for recommender systems.
problem Matrix completion problem in recommender systems.
method Fast adaptive randomized singular value decomposition (SVD) and termination mechanism for latent factors.
result The approach achieves near optimal prediction accuracy with high runtime efficiency.
This paper conducts a rigorous analysis for provable estimation of multidimensional arrays, in particular third-order tensors, from a random subset of its corrupted entries. Our study rests heavily on a recently proposed tensor algebraic framework in which we can obtain tensor singular value decomposition (t-SVD) that …
Matrix completion is a widely used technique for image inpainting and personalized recommender system, etc. In this work, we focus on accelerating the matrix completion using faster randomized singular value decomposition (rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are proposed for handling …
Randomized SVD shows phase transitions in noisy data.
problem Noise sensitivity of randomized SVD in large rank matrices.
method Analyzed R-SVD under low-rank signal plus noise model.
result R-SVD exhibits BBP-like phase transition with outliers above detectability threshold.
New algorithm reduces matrix multiplication time for sparse matrices.
problem Efficiently multiply large sparse matrices with limited space.
method Exploits sparsity to reduce QR decompositions and time complexity.
result Time complexity reduced to $\widetilde{O}\left((
nz(X)+
nz(Y))\ell+n\ell^2
ight)$ in expectation.
New insights into choosing between two data integration methods based on SVD.
problem Choosing between two data integration methods (Stack-SVD and SVD-Stack) for shared latent structure across multiple datasets.
method Derive exact expressions for the asymptotic performance and phase transitions of Stack-SVD and SVD-Stack, and develop optimal weighting schemes.
result Optimally weighted Stack-SVD outperforms optimally weighted SVD-Stack in the asymptotic regime.
Efficient SVD algorithm robust to outliers.
problem Outliers in data matrix affect SVD accuracy and speed.
method Spherically Normalized SVD (SpherSVD) algorithm.
result Significantly faster and more robust than existing methods.
The Matrix Factorization models, sometimes called the latent factor models, are a family of methods in the recommender system research area to (1) generate the latent factors for the users and the items and (2) predict users' ratings on items based on their latent factors. However, current Matrix Factorization models p…
A new algorithm speeds up sparse regression for discovering equations from data.
problem Learning governing equations from vast data with unsatisfying descriptions.
method SPRINT: a fast algorithm using bisection and analytic bounds to identify optimal rank-1 modifications.
result A calculation that would take millions of years can be done in a day.
Inference of Sparse Networks with Unobserved Variables. Application to Gene Regulatory Networksstat.ML
Networks are a unifying framework for modeling complex systems and network inference problems are frequently encountered in many fields. Here, I develop and apply a generative approach to network inference (RCweb) for the case when the network is sparse and the latent (not observed) variables affect the observed ones. …
SVD training reduces DNN rank and computation load without SVD per step.
problem High memory and computational load in deep neural networks.
method Explicitly achieves low-rank DNNs during training without SVD per step, using orthogonality regularization and sparsity-inducing regularizers.
result Significantly reduces DNN rank and computation load compared to existing methods.
A new PCR method using SVD with sparse regularization.
problem Lack of response variable information in traditional PCR.
method One-stage SVD approach with two loss functions and sparse regularization.
result Obtains principal component loadings with response variable information.
We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix call…
A new method for self-attention models that improves uncertainty estimation.
problem Overconfident predictions and lack of calibrated uncertainty in Transformers.
method Kernel-Eigen Pair Sparse Variational Gaussian Processes (KEP-SVGP) with Kernel SVD (KSVD) to handle asymmetry of attention kernels.
result Reduction in time complexity and improved performance on various benchmarks.
New method uses SVD entropy to price artworks.
problem Lack of fine measurements in traditional art pricing models.
method SVD entropy of painting images for content measurement.
result SVD entropy positively affects sales price at 1% significance level.
SVD-closed subgroups of the unitary group: generalized principal logarithms and minimizing geodesicsmath.DG
Study of logarithms in SVD-closed subgroups of unitary group.
problem Understanding logarithms in SVD-closed subgroups of unitary groups.
method Analysis of generalized principal logarithms and minimizing geodesics.
result Set of generalized principal logarithms is a disjoint union of diffeomorphic subsets.
Dictionary learning and component analysis models are fundamental for learning compact representations that are relevant to a given task (feature extraction, dimensionality reduction, denoising, etc.). The model complexity is encoded by means of specific structure, such as sparsity, low-rankness, or nonnegativity. Unfo…
Deep K-SVD Denoisingcs.LG
This work considers noise removal from images, focusing on the well known K-SVD denoising algorithm. This sparsity-based method was proposed in 2006, and for a short while it was considered as state-of-the-art. However, over the years it has been surpassed by other methods, including the recent deep-learning-based newc…
Paper develops a method to robustly cluster tensors with outliers.
problem Clustering tensors contaminated by outliers or sample-specific corruptions.
method Transformed Tensor Low-Rank Representation (OR-TLRR) method.
result Provably recovers row space of clean data and detects outliers.
Globally optimized SVD compression of LLMs via Fermi-function-based rank selection and gauge fixingcs.LG
Physics-inspired methods optimize SVD compression of LLMs.
problem Efficiently compressing large language models (LLMs) using SVD.
method FermiGrad for globally optimal rank selection and PivGa for lossless compression.
result Global optimization of SVD ranks and lossless compression of low-rank factors.
Singular value decomposition (SVD) is the mathematical basis of principal component analysis (PCA). Together, SVD and PCA are one of the most widely used mathematical formalism/decomposition in machine learning, data mining, pattern recognition, artificial intelligence, computer vision, signal processing, etc. In recen…
Generalizes randomized SVD for better matrix approximations using Gaussian vectors.
problem Computing accurate rank-k approximations of matrices with limited data.
method Extends randomized SVD to multivariate Gaussian vectors, incorporating prior knowledge and using Gaussian processes.
result Demonstrates improved accuracy in approximating matrices and Hilbert-Schmidt operators.
Block based Singular Value Decomposition approach to matrix factorization for recommender systemscs.LG
With the abundance of data in recent years, interesting challenges are posed in the area of recommender systems. Producing high quality recommendations with scalability and performance is the need of the hour. Singular Value Decomposition(SVD) based recommendation algorithms have been leveraged to produce better result…
Paper shows faster core identification in matching markets.
problem Core Identification Problem in one-sided matching markets.
method Randomized SVD on preference-derived Markov matrix.
result CIP solved in O(Ln) time, matching lower bound.
Optimal transport aligns rotated linear regression models across domains.
problem Aligning rotated linear regression models across domains with differing statistical properties.
method Combines K-means clustering, OT, and SVD to estimate rotation angle and adapt regression model.
result Optimal transport map recovers underlying rotation in .
This paper examines the impact of centering in PCA and SVD.
problem The influence of centering in PCA and SVD.
method Detailed exploration of centering in PCA and SVD.
result Two PCA embeddings derived from SVD with and without centering can be aligned under certain conditions.
In this paper, we study robust tensor completion by using transformed tensor singular value decomposition (SVD), which employs unitary transform matrices instead of discrete Fourier transform matrix that is used in the traditional tensor SVD. The main motivation is that a lower tubal rank tensor can be obtained by usin…
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limitscs.LG
Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.
problem The failure of a unified SVD compression method in practical tasks like perplexity and accuracy.
method Unified optimization problem for SVD based compression methods, focusing on cross-layer coupling.
result Downstream metrics like perplexity and accuracy degrade severely compared to standard per layer SVD LLM.
GCNNs gain rotation invariance with more training augmentation, making SVD-Universal more effective.
problem Improving robustness of GCNNs to adversarial attacks.
method SVD-Universal technique applied to GCNNs trained with larger rotations.
result SVD-Universal becomes more effective as GCNNs gain rotation invariance.
Spectral embedding based on the Singular Value Decomposition (SVD) is a widely used "preprocessing" step in many learning tasks, typically leading to dimensionality reduction by projecting onto a number of dominant singular vectors and rescaling the coordinate axes (by a predefined function of the singular value). Howe…
The power of randomized algorithms in numerical methods have led to fast solutions which use the Singular Value Decomposition (SVD) as a core routine. However, given the large data size of modern and the modest runtime of SVD, most practical algorithms would require some form of approximation, such as sketching, when r…