The paper analyzes stability of random matrix products with Markovian noise.
problem Analyzing stability of random matrix products with Markovian noise.
method Using a super-Lyapunov drift condition and controlled growth of matrix-valued functions, the paper provides an exponential stability result for the p-th moment of random matrix product.
result Finite-time p-th moment bounds for linear stochastic approximation and TD learning algorithms.
A new matrix concentration inequality for random products of matrices.
problem Understanding the behavior of random matrix products under bounded independent positive semidefinite matrices.
method Developed a non-asymptotic concentration inequality for the product of matrices.
result The inequality provides a bound on the deviation of the matrix product from its expected value.
Generalizes randomized SVD for better matrix approximations using Gaussian vectors.
problem Computing accurate rank-k approximations of matrices with limited data.
method Extends randomized SVD to multivariate Gaussian vectors, incorporating prior knowledge and using Gaussian processes.
result Demonstrates improved accuracy in approximating matrices and Hilbert-Schmidt operators.
Study examines large deviations in random walks on hyperbolic spaces.
problem Large deviations in random walks on Gromov-hyperbolic spaces.
method Established large deviations results for distance and translation length of random walks.
result Deduced a special case of a conjecture regarding spectral radii of random matrix products.
New insights into identifying mixtures of product distributions using Hadamard extensions.
problem Identifying mixtures of product distributions on binary variables.
method Analysis of Hadamard extensions of matrix products.
result Conditions for full column rank of Hadamard extensions.
We prove a central limit theorem for the components of the largest eigenvectors of the adjacency matrix of a finite-dimensional random dot product graph whose true latent positions are unknown. In particular, we follow the methodology outlined in \citet{sussman2012universally} to construct consistent estimates for the …
Given a real matrix A with n columns, the problem is to approximate the Gram product AA^T by c << n weighted outer products of columns of A. Necessary and sufficient conditions for the exact computation of AA^T (in exact arithmetic) from c >= rank(A) columns depend on the right singular vector matrix of A. For a Monte-…
Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.
problem Understanding the spectrum of random kernel matrices in polynomial scaling regimes.
method Investigates random matrices with nonlinear kernel functions applied to inner products of uniformly distributed vectors.
result The spectrum of the random kernel matrix is asymptotically equivalent to a simpler matrix model through free additive convolution.
New matrix ensembles better match deep neural network spectral densities.
problem Theoretical spectral density models for deep networks do not match empirical observations.
method Introduced new matrix ensemble classes to better fit observed spectral densities.
result Theoretical models for deep networks are significantly flawed.
Random matrix ensembles yield uniform distributions on manifolds.
problem Understanding distributions of vectors in random matrix ensembles.
method Analyzing eigenvalues, singular values, and Autonne-Takagi vectors of various random matrix ensembles.
result Uniform distributions on specific manifolds for different types of random matrix ensembles.
The paper generalizes product inequalities for random vectors and their applications.
problem Understanding concentration of measure for products of random vectors.
method Develops expressions for the concentration of functionals of random vectors based on product norms.
result Provides generalized Hanson-Wright inequalities and applications to random matrices.
Study on random matrices in deep neural networks with IID entries.
problem Distribution of singular values in product of random matrices for deep neural networks.
method Random matrix theory with a streamlined approach for non-Gaussian data.
result Generalization of macroscopic universality property to non-Gaussian data.
The paper examines how well node similarities are preserved by random projections in graph embeddings.
problem The preservation of node similarities under random projections in graph embeddings.
method Investigation of dot product and cosine similarity preservation by random projections over graph matrix rows.
result Random projections produce unreliable embeddings for dot product, especially for high-degree nodes.
Paper finds formulas for mutual information and MMSE in matrix tensor product problems.
problem High-dimensional inference problems involving matrix tensor products.
method Single-letter formulas for mutual information and MMSE, using new techniques.
result Analytical formulas describe leading order terms in mutual information and MMSE.
Lower bounds show linear complexity for linear regression.
problem Computational complexity of linear regression.
method Reduction to estimating the least eigenvalue of a random Wishart matrix.
result Θ(d) calls to the oracle are necessary and sufficient for polynomial accuracy.
The study shows inner-product kernels behave similarly to binary kernels in high dimensions.
problem Understanding the behavior of inner-product kernels in high-dimensional data.
method Investigation of eigenspectrum under binary mixture model using random matrix theory.
result The eigenspectrum of inner-product kernels is asymptotically equivalent to binary kernels.
New tail inequalities for sums of random matrices without matrix-dimension terms.
problem Tail behavior of matrix functions in high-dimensional settings.
method Developed new tail inequalities for matrix sums, independent of matrix dimension.
result Tail inequalities for various matrix functions without matrix-dimension terms.
IDPGs extend RDPGs with a Poisson process for random latent positions.
problem Modeling randomness in latent positions for graph structure.
method Introduce IDPGs using Poisson point processes on latent Euclidean space.
result Continuous analogues of adjacency matrices link latent structure to observed graphs.
Matrices satisfying the Restricted Isometry Property (RIP) play an important role in the areas of compressed sensing and statistical learning. RIP matrices with optimal parameters are mainly obtained via probabilistic arguments, as explicit constructions seem hard. It is therefore interesting to ask whether a fixed mat…
New method estimates large matrices' spectra from small sub-matrices.
problem Estimating large matrices' spectra when full matrix-vector products are not available.
method Free decompression based on free probability theory.
result Estimates eigenspectrum of impalpable matrices from small sub-matrices.
We study covariance matrix estimation for the case of partially observed random vectors, where different samples contain different subsets of vector coordinates. Each observation is the product of the variable of interest with a 0−1 Bernoulli random variable. We analyze an unbiased covariance estimator under this mod…
Study on random matrices in deep neural networks using Gaussian data.
problem Distribution of singular values in product of random matrices in deep learning.
method Free probability theory combined with standard techniques of random matrix theory.
result Justification for applying free probability theory to non-independent random data matrices.
Many important problems are characterized by the eigenvalues of a large matrix. For example, the difficulty of many optimization problems, such as those arising from the fitting of large models in statistics and machine learning, can be investigated via the spectrum of the Hessian of the empirical loss function. Networ…
New algorithm reduces cold-start costs in multi-armed bandits for many products.
problem High burn-in costs in multi-armed bandits for new products.
method Two-phase bandit algorithm using subsampling and low-rank matrix estimation.
result Reduces burn-in costs and expedites experiment in large product sets.
In this work we show that, using the eigen-decomposition of the adjacency matrix, we can consistently estimate latent positions for random dot product graphs provided the latent positions are i.i.d. from some distribution. If class labels are observed for a number of vertices tending to infinity, then we show that the …
Improved perturbation reduces matrix condition number to O(n) with minimal storage.
problem Reducing the condition number of deterministic matrices for efficient algorithmic use.
method Introduced pattern matrices and sparse perturbations with dependent entries.
result Condition number reduced to O(n) with O(n) random numbers in O(log n) precision.
Convex optimization method infers latent structure in random dot product graphs.
problem Inferring latent probability matrix of random dot product graphs.
method Conic programming with nuclear norm regularization.
result Asymptotic consistency of probability estimates and recovery of latent structure.
The paper develops concentration inequalities for structured random data, extending beyond independent terms.
problem Developing concentration inequalities for structured weighted sums of random data, including tensors and matrix-valued data.
method The paper develops Hoeffding and Bernstein bounds for structured weighted sums under exchangeability, extending beyond the classical framework of independent terms.
result The paper develops a sharper concentration bound for combinatorial sums of matrix arrays.
Random walks on hyperbolic spaces follow predictable large deviation principles.
problem Understanding the behavior of random walks on hyperbolic spaces.
method Large deviation principles for displacement and translation distances.
result Translation and displacement distances satisfy large deviation principles with the same rate function.
We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form eAb, which the Lanczos …
The paper proves a distribution claim for neural network Jacobians.
problem Distribution of singular values in deep neural networks.
method Free probability and random matrix theory techniques.
result Singular value distribution matches for specific cases.
Improved statistical computation through efficient matrix sampling.
problem Reducing computational cost in large-scale statistical methods.
method Accumulative sub-sampling method to improve statistical efficiency.
result Effective matrix size control improves computational efficiency.
We prove a central limit theorem for the components of the eigenvectors corresponding to the d largest eigenvalues of the normalized Laplacian matrix of a finite dimensional random dot product graph. As a corollary, we show that for stochastic blockmodel graphs, the rows of the spectral embedding of the normalized La…
Stochastic trace estimation with tensor train random vectors
problem Stochastic trace estimation for large-scale matrices
method Gaussian random tensor train vectors
result Median-of-means variant achieves dimension-independent guarantees
This paper shows universality in spectrum behavior for random inner-product kernel matrices in polynomial regime.
problem Understanding spectrum behavior of random inner-product kernel matrices in polynomial regime.
method Analyzing matrices formed by a nonlinear function applied entrywise to a sample-covariance matrix, considering i.i.d. entries with all finite moments.
result The spectrum of random inner-product kernel matrices is universally described by the free convolution of the semicircular and Marčenko-Pastur distributions, with relative weights given by expanding the nonlinear function in the Hermite basis.
High-dimensional curved diffusions show abrupt convergence at a critical time.
problem Understanding abrupt convergence in high-dimensional curved diffusions.
method Functional inequalities and spectral rigidity.
result Abrupt convergence (cutoff) occurs in high dimensions, linked to spectral rigidity.
Corrected whitening restores orthogonality in high-dimensional spherical Gaussian mixtures.
problem In high-dimensional data, standard whitening fails to preserve orthogonality of mixture means.
method Derived exact limits for whitened means dot products using random matrix theory, constructed a corrected whitening matrix.
result Corrected whitening allows for improved estimation of spherical Gaussian mixtures in the large-dimensional regime.
A method for identifying joint and individual subspaces from multi-view data.
problem Unclear conditions for reliably identifying joint and individual subspaces from noisy, high-dimensional measurements.
method Rigorously quantifies conditions based on signal rank, principal angles, and noise levels. Characterizes spectrum perturbations of product of projection matrices.
result Estimates joint and individual subspaces more accurately than existing approaches in simulations and real-world applications.
Generative modeling, which learns joint probability distribution from data and generates samples according to it, is an important task in machine learning and artificial intelligence. Inspired by probabilistic interpretation of quantum physics, we propose a generative model using matrix product states, which is a tenso…
Product Kanerva Machines dynamically combine smaller models for better memory organization.
problem Limited organization in the Kanerva Machine.
method Introducing Product Kanerva Machines that dynamically combine multiple smaller Kanerva Machines.
result Product Kanerva Machines can discover spatial tunings that approximately factorize simple images by object.
We develop an efficient alternating framework for learning a generalized version of Factorization Machine (gFM) on steaming data with provable guarantees. When the instances are sampled from d dimensional random Gaussian vectors and the target second order coefficient matrix in gFM is of rank k, our algorithm conve…
New algorithm estimates matrix-valued regression parameters efficiently.
problem High-dimensional matrix regression with limited sample size.
method KRO-PRO-FAC algorithm using Kronecker product factorization.
result Algorithm provides accurate parameter estimates without covariance estimation.
Study on the geometric Dyson Brownian motion of non-square matrix products.
problem Understanding the spectrum of a product of non-square random matrices.
method Proportional depth-width limit followed by mean-field limit, solving Burgers equation.
result Free log-normal law is obtained in the identity-start case.
Boolean matrix factorization (BMF) is a popular and powerful technique for inferring knowledge from data. The mining result is the Boolean product of two matrices, approximating the input dataset. The Boolean product is a disjunction of rank-1 binary matrices, each describing a feature-relation, called pattern, for a g…
Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extra…
In this work, we propose a new randomized algorithm for computing a low-rank approximation to a given matrix. Taking an approach different from existing literature, our method first involves a specific biased sampling, with an element being chosen based on the leverage scores of its row and column, and then involves we…
The paper analyzes heavy-tailed multivariate distributions in non-stationary systems using random matrix theory.
problem Risk assessment for rare events in complex, non-stationary systems.
method Generalized scalar product between correlation matrices, model for non-stationary fluctuations.
result Formulae for multivariate distributions with reduced parameters, facilitating applications.
The tensor-tensor product (t-product) [M. E. Kilmer and C. D. Martin, 2011] is a natural generalization of matrix multiplication. Based on t-product, many operations on matrix can be extended to tensor cases, including tensor SVD, tensor spectral norm, tensor nuclear norm [C. Lu, et al., 2018] and many others. The line…