arXiv research

Defines cross product for m vectors in n-dimensional spaces.

Study on Gaussian ensemble of matrix products with mixed moments computed.

A new matrix concentration inequality for random products of matrices.

Curious structure of special orthogonal, unitary, and symplectic groups as products of Grassmannians discovered.

The paper deals with distribution of singular values of product of random matrices arising in the analysis of deep neural networks. The matrices resemble the product analogs of the sample covariance matrices, however, an important difference is that the population covariance matrices, which are assumed to be non-random…

This paper shows universality in spectrum behavior for random inner-product kernel matrices in polynomial regime.

We consider the problem of matrix approximation and denoising induced by the Kronecker product decomposition. Specifically, we propose to approximate a given matrix by the sum of a few Kronecker products of matrices, which we refer to as the Kronecker product approximation (KoPA). Because the Kronecker product is an ex…

We consider three different approaches to define natural Riemannian metrics on polytopes of stochastic matrices. First, we define a natural class of stochastic maps between these polytopes and give a metric characterization of Chentsov type in terms of invariance with respect to these maps. Second, we consider the Fish…

The curvature tensor and the scalar curvature are computed in the space of positive definite real matrices endowed by the Kubo-Mori inner product as a Riemannian metric.

Study on random matrices in deep neural networks with IID entries.

The paper defines projective structures for Lie bialgebras and Poisson-Lie groups.

New method estimates large matrices' spectra from small sub-matrices.

Given a real matrix A with n columns, the problem is to approximate the Gram product AA^T by c << n weighted outer products of columns of A. Necessary and sufficient conditions for the exact computation of AA^T (in exact arithmetic) from c >= rank(A) columns depend on the right singular vector matrix of A. For a Monte-…

Proposes a new graph representation method using tensor products.

We study products of random matrices in the regime where the number of terms and the size of the matrices simultaneously tend to infinity. Our main theorem is that the logarithm of the $\ell_2$ norm of such a product applied to any fixed vector is asymptotically Gaussian. The fluctuations we find can be thought of as a…

Gradient flow on softmax attention minimizes nuclear norm of weight matrices.

It is shown that the multiplicative monoids of Temperley-Lieb algebras generated out of the basis are isomorphic to monoids of endomorphisms in categories where an endofunctor is adjoint to itself. Such a self-adjunction is found in a category whose arrows are matrices, and the functor adjoint to itself is based on the…

This paper presents a new method for estimating high dimensional covariance matrices. The method, permuted rank-penalized least-squares (PRLS), is based on a Kronecker product series expansion of the true covariance matrix. Assuming an i.i.d. Gaussian random sample, we establish high dimensional rates of convergence to…

Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.

Study of strictly accretive matrices using Finsler geometry.

We study algebraic properties of matrices whose rows are mutual neighbours, and are also neigbours of 0 ("neighbour" in the sense of a certain nilpotency condition). The intended application is in synthetic differential geometry. For a square matrix of this kind, the product of the diagonal entries equals the determina…

We propose a statistical model for natural language that begins by considering language as a monoid, then representing it in complex matrices with a compatible translation invariant probability measure. We interpret the probability measure as arising via the Born rule from a translation invariant matrix product state.

At the core of any inference procedure in deep neural networks are dot product operations, which are the component that require the highest computational resources. A common approach to reduce the cost of inference is to reduce its memory complexity by lowering the entropy of the weight matrices of the neural network, …

It is shown that the multiplicative monoids of Brauer's centralizer algebras generated out of the basis are isomorphic to monoids of endomorphisms in categories where an endofunctor is adjoint to itself, and where, moreover, a kind of symmetry involving the self-adjoint functor is satisfied. As in a previous paper, of …

We define the notions of trace, determinant and, more generally, Berezinian of matrices over a (Z_2)^n graded commutative associative algebra. The applications include a new approach to the classical theory of matrices with coefficients in a Clifford algebra, in particular of quaternionic matrices. In a special case, w…

Theoretical analysis of the error landscape of deep neural networks has garnered significant interest in recent years. In this work, we theoretically study the importance of noise in the trajectories of gradient descent towards optimal solutions in multi-layer neural networks. We show that adding noise (in different wa…

This paper uncovers the low-rank structure of neural network Hessians.

This paper introduces a submanifold of the moduli space of unitary representations of the fundamental group of a punctured sphere with fixed local monodromy. The submanifold is defined via products of involutions through Lagrangian subspaces. We show that the moduli space of Lagrangian representations is a Lagrangian s…

A new algorithm POGO optimizes thousands of orthogonal matrices efficiently.

A new algorithm speeds up matrix multiplication without actual multiplication.

Two new metrics assess LLM faithfulness and entropy, improving model reliability.

Study on the geometric Dyson Brownian motion of non-square matrix products.

Coupled nonlinear integrable systems are generated from usual zero curvature equation. The relevant Maurer-Cartan forms are constructed by combining suitably chosen matrices (nilpotent, Hadamard, idempotent and k-idempotent) and Lie algebraic elements via Kronecker product. In each case a closure type property among th…

Character expansion expresses extended HOMFLY polynomials through traces of products of finite dimensional R- and Racah mixing matrices. We conjecture that the mixing matrices are expressed entirely in terms of the eigenvalues of the corresponding R-matrices. Even a weaker (and, perhaps, more reliable) version of this …

This note classifies splittable lattices in a specific Lie group.

Discovering the underlying low dimensional structure of high dimensional data has attracted a significant amount of researches recently and has shown to have a wide range of applications. As an effective dimension reduction tool, singular value decomposition is often used to analyze high dimensional matrices, which are…

New curvature tensor and matrices for connection graphs derived from Bakry-Émery curvature.

In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatio-temporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. To allow a sm…

In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predicting missing relationships---labeled edges. State-of-the-art embedding models propose different trade-offs between modeling expressiveness, …

The crossing matrix of a braid on $N$ strands is the $N\times N$ integer matrix with zero diagonal whose $i,j$ entry is the algebraic number (positive minus negative) of crossings by strand $i$ over strand $j$ . When restricted to the subgroup of pure braids, this defines a homomorphism onto the additive subgroup of $N…

Construction of (colored) knot polynomials for double-fat graphs is further generalized to the case when "fingers" and "propagators" are substituting R-matrices in arbitrary closed braids with m-strands. Original version of arXiv:1504.00371 corresponds to the case m=2, and our generalizations sheds additional light on …

Vectors of data are at the heart of machine learning and data mining. Recently, vector quantization methods have shown great promise in reducing both the time and space costs of operating on vectors. We introduce a vector quantization algorithm that can compress vectors over 12x faster than existing techniques while al…

If a knot is represented by an m-strand braid, then HOMFLY polynomial in representation R is a sum over characters in all representations Q\in R^{\otimes m}. Coefficients in this sum are traces of products of quantum R-matrices along the braid, but these matrices act in the space of intertwiners, and their size is equa…

MCCA extracts shared structure from multiple tensor datasets.

The Schatten quasi-norm can be used to bridge the gap between the nuclear norm and rank function, and is the tighter approximation to matrix rank. However, most existing Schatten quasi-norm minimization (SQNM) algorithms, as well as for nuclear norm minimization, are too slow or even impractical for large-scale problem…

Mirror descent algorithm recovers low-rank matrices in matrix sensing.

The paper proves a distribution claim for neural network Jacobians.

Autoencoder performance is predicted by eigenvalues of weight matrices.