Singular Value Decomposition (SVD) constitutes a bridge between the linear algebra concepts and multi-layer neural networks---it is their linear analogy. Besides of this insight, it can be used as a good initial guess for the network parameters, leading to substantially better optimization results.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “value-decomposition”
The paper uncovers the mathematical structure enabling value decomposition in multi-agent systems.
problem Theoretical justification for why value decomposition works effectively in multi-agent systems remains underexplored.
method The paper introduces the concept of Markov entanglement to measure the underlying structure and demonstrates how it can be used to bound the decomposition error.
result The widely-used class of index policies is weakly entangled and enjoys a sublinear scale of decomposition error for -agent systems.
Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learningcs.LG
The paper explores when and why value decomposition algorithms work in cooperative multi-agent reinforcement learning.
problem The applicability and convergence properties of value decomposition algorithms in cooperative multi-agent reinforcement learning are unclear.
method The paper introduces decomposable games and proves that applying the multi-agent fitted Q-Iteration algorithm leads to an optimal Q-function in these games.
result The paper offers theoretical insights into when and why value decomposition algorithms converge in cooperative multi-agent reinforcement learning.
In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure featu…
We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared…
Derives a primal-dual MLSVD formulation for multilinear data.
problem Efficiently decompose multilinear data for signal analysis and deep learning.
method Kernelizable primal-dual formulation of MLSVD.
result Derives a new MLSVD formulation with computational advantages.
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
We extend the randomized singular value decomposition (SVD) algorithm \citep{Halko2011finding} to estimate the SVD of a shifted data matrix without explicitly constructing the matrix in the memory. With no loss in the accuracy of the original algorithm, the extended algorithm provides for a more efficient way of matrix…
A new PCR method using SVD with sparse regularization.
problem Lack of response variable information in traditional PCR.
method One-stage SVD approach with two loss functions and sparse regularization.
result Obtains principal component loadings with response variable information.
Informed by recent work on tensor singular value decomposition and circulant algebra matrices, this paper presents a new theoretical bridge that unifies the hypercomplex and tensor-based approaches to singular value decomposition and robust principal component analysis. We begin our work by extending the principal comp…
The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.
problem Improving convergence rates of sketch-and-project methods for solving linear systems and non-linear optimization problems.
method Developing a theoretical framework and new spectral bounds for the expected sketched projection matrix.
result The convergence rate improves linearly with sketch size and even faster with certain spectral decays.
Singular value decomposition (SVD) is the mathematical basis of principal component analysis (PCA). Together, SVD and PCA are one of the most widely used mathematical formalism/decomposition in machine learning, data mining, pattern recognition, artificial intelligence, computer vision, signal processing, etc. In recen…
New insights into X-ray transform on hyperbolic disk, with functional relations and range characterizations.
problem Understanding the X-ray transform on hyperbolic geometry.
method Derived new singular value decompositions, range characterizations, and intertwining relations with wedge-type differential operators.
result Sharp understanding of boundary behavior and invertibility settings for the X-ray transform.
This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank constrained optimisation problem. This problem is non-convex and state-of-the-art algor…
A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
New method uses SVD entropy to price artworks.
problem Lack of fine measurements in traditional art pricing models.
method SVD entropy of painting images for content measurement.
result SVD entropy positively affects sales price at 1% significance level.
For a one-parameter family of simple metrics of constant curvature ( for ) on the unit disk , we first make explicit the Pestov-Uhlmann range characterization of the geodesic X-ray transform, by constructing a basis of functions making up its range and co-kernel. Such a range characterization also t…
Efficient Orthogonal Tensor Decomposition, with an Application to Latent Variable Model Learningstat.ML
Decomposing tensors into orthogonal factors is a well-known task in statistics, machine learning, and signal processing. We study orthogonal outer product decompositions where the factors in the summands in the decomposition are required to be orthogonal across summands, by relating this orthogonal decomposition to the…
Two multifidelity trust-region methods use low-fidelity models for efficient optimization.
problem Efficiently solving complex optimization problems with limited data.
method Sketched Trust-Region (STR) and SVD Trust-Region (SVDTR) methods using low-fidelity models.
result Potential gain in efficiency demonstrated through numerical examples.
From linear classifiers to neural networks, image classification has been a widely explored topic in mathematics, and many algorithms have proven to be effective classifiers. However, the most accurate classifiers typically have significantly high storage costs, or require complicated procedures that may be computation…
New method for causal effect estimation with hidden confounders.
problem Estimating causal effects in the presence of hidden confounders.
method Singular value decomposition of a conditional expectation operator followed by saddle-point optimization.
result Our method outperforms existing methods on common benchmarks.
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributi…
Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation such as CUR decomposition, kernel matrix approximation, and stream singular value decomposition (SVD), etc. In this paper, we propose a fast …
Paper improves tensor completion by reducing sample entries needed.
problem Reducing the number of required sample entries for tensor completion.
method Utilizes multi-rank and unitary transformation in tensor singular value decomposition.
result Provides a bound on the number of required sample entries for tensor completion.
The study analyzes XRP transaction networks to understand market dynamics.
problem Understanding market dynamics of XRP through transaction data.
method Weekly weighted directed networks are embedded into a vector space using network embedding techniques. A correlation tensor is calculated and analyzed using singular value decomposition.
result The correlation tensor provides insights into the system's behavior and dependence on model parameters.
Generalizes randomized SVD for better matrix approximations using Gaussian vectors.
problem Computing accurate rank-k approximations of matrices with limited data.
method Extends randomized SVD to multivariate Gaussian vectors, incorporating prior knowledge and using Gaussian processes.
result Demonstrates improved accuracy in approximating matrices and Hilbert-Schmidt operators.
Block based Singular Value Decomposition approach to matrix factorization for recommender systemscs.LG
With the abundance of data in recent years, interesting challenges are posed in the area of recommender systems. Producing high quality recommendations with scalability and performance is the need of the hour. Singular Value Decomposition(SVD) based recommendation algorithms have been leveraged to produce better result…
In this note, we report the back propagation formula for complex valued singular value decompositions (SVD). This formula is an important ingredient for a complete automatic differentiation(AD) infrastructure in terms of complex numbers, and it is also the key to understand and utilize AD in tensor networks.
Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsit…
A new neural network improves the accuracy of predicting constants of motion.
problem Discovering constants of motion in dynamical systems.
method A novel neural network architecture using SVD and a two-phase training algorithm.
result The new approach retains advantages of COMET and improves performance.
The paper updates SVD of evolving matrices using projection techniques.
problem Updating the rank-k truncated SVD of evolving matrices.
method Projection viewpoint, building subspaces to approximate singular vectors.
result The proposed algorithm leads to higher accuracy, especially for large singular values.
To solve deep neural network (DNN)'s huge training dataset and its high computation issue, so-called teacher-student (T-S) DNN which transfers the knowledge of T-DNN to S-DNN has been proposed. However, the existing T-S-DNN has limited range of use, and the knowledge of T-DNN is insufficiently transferred to S-DNN. To …
New asymmetric kernel methods improve feature learning.
problem Improving feature learning with asymmetric kernels.
method Coupled covariance eigenproblem and Nyström method.
result Empirical evaluations show benefits of KSVD.
Simple method for estimating missing panel data entries with confidence intervals.
problem Estimating missing values in panel data with staggered adoption.
method Simple matrix algebra and singular value decomposition for estimation, with data-driven confidence intervals.
result Confidence intervals match non-asymptotic lower bounds, proving instance optimality.
Paper relaxes symmetry conditions for universal feature selection in noisy data.
problem Feature selection in noisy data with weak symmetry.
method Developed a universal feature selection framework using singular value decomposition of canonical dependence matrix.
result Selected features achieve asymptotically optimal error exponents up to a residual term.
Investigates offline RL in factorisable action spaces, overcoming overestimation bias.
problem Overestimation bias in value estimates for unseen state-action pairs.
method Value-decomposition approach in DecQN, adapted for factorised discrete action spaces.
result Demonstrates the effectiveness of factorised approach in offline RL.
New benchmarks provide full training data for NAS research.
problem Limited training data on popular benchmarks restricts multi-fidelity techniques.
method SVD and noise modeling to create surrogate benchmarks with full training info.
result Learning curve extrapolation framework improves single-fidelity algorithms.
New methods improve Fisher Matrix approximations for neural networks at low cost.
problem High cost of solving Fisher Information Matrix (FIM) in neural networks.
method Direct minimization via Kronecker product singular value decomposition.
result Improved approximations to FIM provide more accurate and faster optimization.
The paper analyzes PLS-SVD in high-dimensional data integration, revealing its strengths and limitations.
problem Understanding the behavior of PLS-SVD in high-dimensional data integration.
method Analysis using random matrix theory and singular value decomposition.
result PLS-SVD exhibits counter-intuitive or limiting behavior in certain regimes and outperforms PCA when detecting common latent subspace.
Study examines local extrema and crossing statistics in financial markets.
problem Understanding local extrema and crossing statistics in financial markets.
method Excursion set theory, numerical computation, theoretical prediction, clustering of geometrical measures, cross-correlation, Singular Value Decomposition.
result Excursion sets reveal statistical coherency and sensitivity to crises in financial markets.
New method approximates high-dimensional probability densities efficiently.
problem Approximating high-dimensional probability densities accurately and efficiently.
method Hierarchical tensor-network approach using randomized SVD and linear equations.
result The method effectively approximates high-dimensional densities with linear complexity.
The study examines numerical aspects of Karhunen-Loève expansions for stochastic processes.
problem Constructing Karhunen-Loève expansions for second-order stochastic processes.
method Spectral decomposition of covariance operator via Fredholm integral equation, discretization, singular value decomposition of weight-scaled sample matrix.
result Consistent solutions for model-based and data-driven KLE construction, characterized by convergence of SVD-based eigenvalue estimates and KL coefficients distributions.
Develops a method to estimate the shadow riskless rate from empirical data.
problem No risky asset in market, need for a shadow riskless rate.
method PCA, SVD, regularization to estimate SRR from correlated geometric Brownian motion.
result Estimates the shadow riskless rate from empirical datasets.
This paper speeds up spectral clustering for large graphs by dilating their eigenspectrum.
problem Slow convergence in spectral clustering due to small eigengaps in graph Laplacians.
method Polynomial approximations to matrix operations that dilate the spectrum without changing eigenvectors.
result Significant acceleration of convergence in spectral clustering.
New algorithm rSVDdpd improves robustness and scalability for video surveillance background modeling.
problem Camera tampering and noisy videos make background separation challenging.
method Introduces rSVDdpd, a robust singular value decomposition technique for scalable video surveillance.
result Demonstrates superior performance on benchmark and real-life datasets.
High throughput biomedical measurements normally capture multiple overlaid biologically relevant signals and often also signals representing different types of technical artefacts like e.g. batch effects. Signal identification and decomposition are accordingly main objectives in statistical biomedical modeling and data…
Proposes statistical inference for dependency knowledge graphs from EHR data.
problem Statistical uncertainty in linking entities in EHR data.
method Dynamic log-linear topic model with singular value decomposition.
result Established asymptotic normality for sparse graph edge recovery.