Study inequalities for singular values of rectangular matrices.
problem Inequalities for singular values of rectangular matrices.
method Study convex cones associated to isotropic representations of symmetric spaces.
result Describe inequalities by cohomological conditions.
New technique stabilizes singular values in concatenated matrices.
problem How singular values of concatenated matrices relate to individual components.
method Developed perturbation technique extending classical results to concatenated matrices.
result Dominant singular values remain stable under small perturbations in submatrices.
Study optimizes shared singular subspace estimation from noisy matrices.
problem Estimating shared singular subspaces across multiple noisy matrices.
method Low-rank matrix denoising framework with Stack-SVD and novel estimators.
result Stack-SVD achieves minimax rate-optimality for identical shared subspaces, and novel estimators for partial sharing.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
Bootstrapping regularizes singular correlation matrices, reducing the need for complex regularization.
problem Singular correlation matrices in large datasets.
method Averaging bootstrapped correlation matrices to ensure positive-definiteness.
result The averaged correlation matrix is almost surely positive-definite with a sufficient number of bootstraps.
The paper studies phase transitions in random matrices and tensor unfolding for detecting signals.
problem Phase transitions in singular values and vectors of large random matrices.
method Analysis of singular values and vectors of long rectangular random matrices, and tensor unfolding algorithm for asymmetric rank-one spiked tensor models.
result An exact threshold for tensor unfolding to detect signals, independent of unfolding procedure.
New framework for higher-order singular-value derivatives of rectangular matrices.
problem Challenging to derive higher-order Fréchet derivatives of singular values in real rectangular matrices.
method Using Kato's analytic perturbation theory for self-adjoint operators and embedding rectangular matrices into block self-adjoint operators.
result Closed-form expressions for the n-th order spectral variations of singular values. Ranky solves SVD for large sparse matrices in distributed systems.
problem Rank problem in large sparse matrices for SVD.
method Distributed approach to solve rank problem.
result Recovers SVD with negligible error for large sparse matrices.
The paper updates SVD of evolving matrices using projection techniques.
problem Updating the rank-k truncated SVD of evolving matrices.
method Projection viewpoint, building subspaces to approximate singular vectors.
result The proposed algorithm leads to higher accuracy, especially for large singular values.
We study the differential geometric properties of the manifold of non-singular symmetric real matrices endowed with the trace metric; in case of positive definite matrices we describe the full group of isometries
Random feature matrices' singular values concentrate near their full expectation in high dimensions.
problem Characterizing the spectra of random feature matrices for regression problems.
method Analyzing two settings of input variables (random or well-separated) with conditions on dimension, complexity ratio, and sampling variance.
result The singular values of random feature matrices concentrate near their full expectation and near one with high probability.
We construct and study a family of double-periodic almost entire solutions of the maximal surface equation. The solutions are parameterized by a submanifold of 3×3-matrices (the so-called generating matrices). We show that the constructed solutions are either space-like or of mixed type with the light-cone type…
The paper proves a distribution claim for neural network Jacobians.
problem Distribution of singular values in deep neural networks.
method Free probability and random matrix theory techniques.
result Singular value distribution matches for specific cases.
Generative Adversarial Networks (GANs), though powerful, is hard to train. Several recent works (brock2016neural,miyato2018spectral) suggest that controlling the spectra of weight matrices in the discriminator can significantly improve the training of GANs. Motivated by their discovery, we propose a new framework for t…
We explicitly compute the intrinsic volume of the set of real (and real symmetric) matrices of Frobenius norm one and given corank (the case of matrices with zero determinant as a special case). We give asymptotic formulas for our computations and we discuss several examples and applications.
Study real logarithms of semi-simple matrices, focusing on differential structure.
problem Understanding the differential structure of real logarithms of semi-simple matrices.
method Examines the differential structure of real logarithms of semi-simple matrices under specific matrix types.
result Characterizes the differential structure of real logarithms of semi-simple matrices.
In this article we obtain a result about the uniqueness of factorization in terms of conjugates of the matrix $U=(\xymatrix{1 & 1 0 & 1})$, of some matrices representing the conjugacy classes of those elements of SL(2,Z) arising as the monodromy around a singular fiber in an elliptic fibration (i.e. those matrices th…
Study on random matrices in deep neural networks with IID entries.
problem Distribution of singular values in product of random matrices for deep neural networks.
method Random matrix theory with a streamlined approach for non-Gaussian data.
result Generalization of macroscopic universality property to non-Gaussian data.
Paper studies tensor models using random matrix theory.
problem Analyzing asymmetric order-d spiked tensor models with Gaussian noise.
method Uses variational definition of singular vectors and values, constructs equivalent spiked symmetric block-wise random matrix from tensor contractions.
result Characterizes asymptotic singular values and alignments of singular vectors with true spike components.
New algorithms improve RPCA for large matrices with upper rank bounds.
problem Efficiently decompose large matrices into low-rank and sparse parts.
method Combine regularization and matrix multiplication approaches with upper rank bounds.
result Proposed algorithms are faster and more robust than existing methods.
Study on overlaps of singular vectors in Gaussian matrix submatrices.
problem Analyzing overlaps of singular vectors in submatrices of Gaussian matrices.
method Utilizes dynamics of singular vectors and specific resolvents for Brownian trajectories.
result Explicit forms for limiting rescaled mean squared overlaps in the bulk of spectra.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
We extend the randomized singular value decomposition (SVD) algorithm \citep{Halko2011finding} to estimate the SVD of a shifted data matrix without explicitly constructing the matrix in the memory. With no loss in the accuracy of the original algorithm, the extended algorithm provides for a more efficient way of matrix…
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
The paper tackles joint learning of linear systems, improving accuracy with pooled data.
problem Estimating transition matrices of multiple related linear systems more accurately.
method Developed novel techniques to bound estimation errors and establish high probability bounds for singular values.
result Significant gains in accuracy achieved by pooling data across systems.
A new method reduces task interference in model merging.
problem Model merging overlooks structural information and is susceptible to task interference.
method Task Singular Vectors (TSV) and TSV-Compress for compression and interference reduction.
result TSV-Merge significantly outperforms existing methods.
We address the problem of minimizing a convex function over the space of large matrices with low rank. While this optimization problem is hard in general, we propose an efficient greedy algorithm and derive its formal approximation guarantees. Each iteration of the algorithm involves (approximately) finding the left an…
We present a general method to detect and extract from a finite time sample statistically meaningful correlations between input and output variables of large dimensionality. Our central result is derived from the theory of free random matrices, and gives an explicit expression for the interval where singular values are…
Given a real matrix A with n columns, the problem is to approximate the Gram product AA^T by c << n weighted outer products of columns of A. Necessary and sufficient conditions for the exact computation of AA^T (in exact arithmetic) from c >= rank(A) columns depend on the right singular vector matrix of A. For a Monte-…
New method cleans cross-covariance matrices for better financial forecasting.
problem Asymptotically optimal cross-covariance cleaners fail in real-world, time-varying markets.
method Physics-informed neural network that learns from empirical singular values.
result Trained model outperforms analytical cleaners in out-of-sample cross-covariance prediction.
New algorithm for Coxeter connections with maximally ramified singularities.
problem Constructing connections on the projective line with a maximally ramified irregular singularity.
method Numerical algorithm for matrix completions to solve the Upper Nilpotent Completion Problem.
result Explicit constructions of Coxeter connections with specified singularities.
The paper deals with distribution of singular values of product of random matrices arising in the analysis of deep neural networks. The matrices resemble the product analogs of the sample covariance matrices, however, an important difference is that the population covariance matrices, which are assumed to be non-random…
This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically…
The paper explores totally geodesic submanifolds in SPD matrices and their properties.
problem Characterizing and understanding totally geodesic submanifolds in SPD matrices.
method Detailed geometric analysis and projection properties of SPD matrices.
result A non-linear projection on totally geodesic submanifolds has the minimizing property.
A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
Study infinite-dimensional Toda manifold at irregular singularity, revealing non-uniqueness of formal solutions.
problem Non-uniqueness of formal solutions to the Dubrovin equation at irregular singularity.
method Revisited canonical coordinates, formal solutions analysis, Borel resummation, Stokes matrices computation.
result Infinite-dimensional Stokes matrices computed from resummed formal solutions.
In this paper, we prove a similar result to the fundamental theorem of regular surfaces in classical differential geometry, which extends the classical theorem to the entire class of singular surfaces in Euclidean 3-space known as frontals. Also, we characterize in a simple way these singular surfaces and its fundament…
The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.
problem Testing on non-diagonalizable matrices for network statistics.
method Generalizes Wald and t-tests to non-symmetric matrices, controlling convergence rates.
result Improved inference on network statistics from directed networks.
The paper is on the vanishing topology of singular Milnor fibres of holomorphic families of arbitrary square, symmetric and skew-symmetric matrices with sufficiently many parameters. We define vanishing cycles on such fibres, prove an extended form of the Damon-Pike μ=τ conjecture about the families of a special type…
Bayesian neural networks can be simplified by parameterizing weights as rank-r matrices, reducing parameter count and improving performance.
problem High parameter count in standard Bayesian neural networks.
method Parameterize weights as W=ABop with A∈Rmimesr, B∈Rnimesr, inducing a singular posterior. result PAC-Bayes generalization bounds and loss bounds show improved performance with fewer parameters.
The Davis-Kahan-Wedin sinΘ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin sinΘ theorem when the perturbation is a Gaussian rando…
The paper uses deep learning to detect financial market regimes from correlation matrices.
problem Detecting financial market regimes from correlation dynamics.
method Representation learning on block hierarchical SPD correlation matrices using SPDNet, SPD-NetBN, and U-SPDNet models.
result Deep learning models overfit in financial market data, misleading performance metrics.
Examines learning efficiency in neural networks and related models.
problem Analyzing efficiency in deep learning models with singular learning coefficients.
method Examined learning coefficients in neural networks and three-layer neural networks with ReLU units.
result Extended results to include Softmax function, providing a broader understanding of learning efficiency.
Random matrix analysis reveals that neural network weights are mostly random, with some indicating learned information.
problem Understanding how neural networks store information needed for tasks.
method Random matrix theory (RMT) applied to weight matrices of trained deep neural networks.
result Most singular values and eigenvectors of trained neural networks follow universal RMT predictions, suggesting they are random and do not contain system-specific information.
Smooth resolutions found for quotient of R^2 by infinite discrete groups.
problem Symplectic resolutions of quotient spaces by infinite discrete subgroups.
method Constructing smooth symplectic resolutions for R^2 under infinite discrete subgroups of GL_2(R).
result Minimal resolutions of Du Val singular varieties are symplectic resolutions of R^2/G.
Gradient flow on softmax attention minimizes nuclear norm of weight matrices.
problem Classification with separate key and query weight matrices.
method Gradient flow on exponential loss, separability assumption, reparameterization, approximate KKT conditions.
result Gradient flow implicitly minimizes nuclear norm of weight matrices, contrasting with Frobenius norm minimization.
Continuity of roots of hyperbolic polynomials with smooth coefficients.
problem Continuity of the solution map for hyperbolic polynomials.
method Proving continuity of the solution map from hyperbolic polynomials of degree d with C^d coefficients to their increasingly ordered roots.
result Continuity of the solution map for hyperbolic polynomials with C^d coefficients.