arXiv research

Simple bounds for covariance and Gram matrices across various settings.

Study extends bounds on sample covariance matrices with general dependence.

This work compresses heavy-tailed weight matrices for tighter generalization bounds.

New bound for neural networks with full-rank weights, independent of network width.

We consider the problem of approximate joint triangularization of a set of noisy jointly diagonalizable real matrices. Approximate joint triangularizers are commonly used in the estimation of the joint eigenstructure of a set of matrices, with applications in signal processing, linear algebra, and tensor decomposition.…

We find an upper bound for geodesic distances associated to monotone Riemannian metrics on positive definite matrices and density matrices.

New nodal domain theorems for symmetric matrices via signed graphs.

The study sets limits on heat equation solutions' Hessians on curved spaces.

The density matrices are positively semi-definite Hermitian matrices of unit trace that describe the state of a quantum system. The goal of the paper is to develop minimax lower bounds on error rates of estimation of low rank density matrices in trace regression models used in quantum state tomography (in particular, i…

The paper introduces a new method for tail bounds of random vectors and matrices.

In this paper, we study the problem of compressed sensing using binary measurement matrices and $\ell_1$-norm minimization (basis pursuit) as the recovery algorithm. We derive new upper and lower bounds on the number of measurements to achieve robust sparse recovery with binary matrices. We establish sufficient conditi…

The inverse covariance matrix provides considerable insight for understanding statistical models in the multivariate setting. In particular, when the distribution over variables is assumed to be multivariate normal, the sparsity pattern in the inverse covariance matrix, commonly referred to as the precision matrix, cor…

New algorithms improve RPCA for large matrices with upper rank bounds.

Lower bounds on private estimation of Gaussian covariance matrices.

The paper improves transformer generalization bounds using rank-dependent covering number bounds.

This article provides a new toolbox to derive sparse recovery guarantees from small deviations on extreme singular values or extreme eigenvalues obtained in Random Matrix Theory. This work is based on Restricted Isometry Constants (RICs) which are a pivotal notion in Compressed Sensing and High-Dimensional Statistics a…

Given a real matrix A with n columns, the problem is to approximate the Gram product AA^T by c << n weighted outer products of columns of A. Necessary and sufficient conditions for the exact computation of AA^T (in exact arithmetic) from c >= rank(A) columns depend on the right singular vector matrix of A. For a Monte-…

We provide bounds for kernel matrices and new approximations for high-dimensional data.

The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.

Unified bounds for iterative algorithms with Gaussian data matrices.

Kernel methods are successful approaches for different machine learning problems. This success is mainly rooted in using feature maps and kernel matrices. Some methods rely on the eigenvalues/eigenvectors of the kernel matrix, while for other methods the spectral information can be used to estimate the excess risk. An …

The Riemannian Bures metric on the space of (normalized) complex positive matrices is used for parameter estimation of mixed quantum states based on repeated measurements just as the Fisher information in classical statistics. It appears also in the concept of purifications of mixed states in quantum physics. Here we d…

Improved COD algorithm reduces streaming AMM errors and uses less space.

A new matrix concentration inequality for random products of matrices.

In this paper, we present a new framework to obtain tail inequalities for sums of random matrices. Compared with existing works, our tail inequalities have the following characteristics: 1) high feasibility--they can be used to study the tail behavior of various matrix functions, e.g., arbitrary matrix norms, the absol…

Recently low displacement rank (LDR) matrices, or so-called structured matrices, have been proposed to compress large-scale neural networks. Empirical results have shown that neural networks with weight matrices of LDR matrices, referred as LDR neural networks, can achieve significant reduction in space and computation…

New framework estimates eigenvalues of kernel matrices without full matrix construction.

The paper tackles joint learning of linear systems, improving accuracy with pooled data.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

We derive exponential tail inequalities for sums of random matrices with no dependence on the explicit matrix dimensions. These are similar to the matrix versions of the Chernoff bound and Bernstein inequality except with the explicit matrix dimensions replaced by a trace quantity that can be small even when the dimens…

Estimates matrix trace optimization with statistical learning theory.

Generalization bounds derived for neural ODEs and deep residual networks.

Study resolvent convergence for random matrices with general covariance profiles.

Convolutional neural networks (CNNs) have achieved breakthrough performances in a wide range of applications including image classification, semantic segmentation, and object detection. Previous research on characterizing the generalization ability of neural networks mostly focuses on fully connected neural networks (F…

This paper tightens bounds on the smallest eigenvalue of NTK for deep ReLU networks.

We present a framework to derive upper bounds on the number of regions that feed-forward neural networks with ReLU activation functions are affine linear on. It is based on an inductive analysis that keeps track of the number of such regions per dimensionality of their images within the layers. More precisely, the info…

Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.

For any prime power $q$ and any dimension $s \ge 1$, we present a construction of $(t,s)$-sequences in base $q$ with finite-row generating matrices such that, for fixed $q$, the quality parameter $t$ is asymptotically optimal as a function of $s$ as $s \to \infty$. This is the first construction of $(t,s)$-sequences th…

Paper improves TD learning algorithm bounds with linear approx.

New framework solves low-rank optimization problems to certifiable optimality.

The paper provides bounds for LSA with fixed stepsizes under random estimates.

Paper finds a lower bound for estimating low-rank matrices in logistic regression.

We prove global and local upper bounds for the Hessian of log positive solutions of the heat equation on a Riemannian manifold. The metric is either fixed or evolves under the Ricci flow. These upper bounds supplement the well-known global lower bound.

Optimizes clustering in Gaussian mixtures with varying covariance matrices.

Improved bounds for $\ell_p$ sensitivity sampling reducing the sample complexity for structured matrices.

The study sets limits on how well nonlinear models can generalize from training data.

We consider the problem of sampling from posterior distributions for Bayesian models where some parameters are restricted to be orthogonal matrices. Such matrices are sometimes used in neural networks models for reasons of regularization and stabilization of training procedures, and also can parameterize matrices of bo…

Let ${\mathcal S}_m$ be the set of all $m\times m$ density matrices (Hermitian positively semi-definite matrices of unit trace). Consider a problem of estimation of an unknown density matrix $ρ\in {\mathcal S}_m$ based on outcomes of $n$ measurements of observables $X_1,\dots, X_n\in {\mathbb H}_m$ (${\mathbb H}_m$ bei…