A new matrix concentration inequality for random products of matrices.
problem Understanding the behavior of random matrix products under bounded independent positive semidefinite matrices.
method Developed a non-asymptotic concentration inequality for the product of matrices.
result The inequality provides a bound on the deviation of the matrix product from its expected value.
Random feature matrices' singular values concentrate near their full expectation in high dimensions.
problem Characterizing the spectra of random feature matrices for regression problems.
method Analyzing two settings of input variables (random or well-separated) with conditions on dimension, complexity ratio, and sampling variance.
result The singular values of random feature matrices concentrate near their full expectation and near one with high probability.
The paper introduces a new method for tail bounds of random vectors and matrices.
problem Estimating norms of random vectors and matrices under moment assumptions.
method Variational tail bounds for norms of random vectors and matrices.
result Dimension-free concentration inequalities for various norms of random vectors and matrices.
The paper generalizes product inequalities for random vectors and their applications.
problem Understanding concentration of measure for products of random vectors.
method Develops expressions for the concentration of functionals of random vectors based on product norms.
result Provides generalized Hanson-Wright inequalities and applications to random matrices.
Improved concentration inequalities for sub-Weibull variables enhance statistical and machine learning applications.
problem Improving concentration inequalities for sub-Weibull random variables.
method Developed new concentration inequalities for sums of independent sub-Weibull random variables, including a new sub-Weibull parameter.
result New concentration inequalities with sharper constants and a mixture of sub-Gaussian and sub-Weibull tails.
Kernel methods are successful approaches for different machine learning problems. This success is mainly rooted in using feature maps and kernel matrices. Some methods rely on the eigenvalues/eigenvectors of the kernel matrix, while for other methods the spectral information can be used to estimate the excess risk. An …
New inequalities for matrix supermartingales converge under various conditions.
problem Convergence and maximal inequalities of supermartingales in positive semidefinite matrices.
method Developed new concentration inequalities for matrix supermartingales.
result New inequalities for matrix supermartingales under different tail conditions.
We introduce an universum of the Polish (=complete separable metric) space - the convex cone of distance matrices and study its geometry. It happened that the generic Polish spaces in this sense of this universum is so called Urysohn spaces defined by P.S.Urysohn in 20-th, and generic metric triple (= metric space with…
The paper develops concentration inequalities for structured random data, extending beyond independent terms.
problem Developing concentration inequalities for structured weighted sums of random data, including tensors and matrix-valued data.
method The paper develops Hoeffding and Bernstein bounds for structured weighted sums under exchangeability, extending beyond the classical framework of independent terms.
result The paper develops a sharper concentration bound for combinatorial sums of matrix arrays.
The paper studies neural networks with wide layers and finds a deformed semicircle law.
problem Investigating spectral distributions of neural networks in the ultra-wide regime.
method Analyzes empirical kernel matrices, proves deformed semicircle law, provides nonlinear Hanson-Wright inequality.
result Emergence of a deformed semicircle law in the ultra-wide neural network regime.
In recent years, random matrices have come to play a major role in computational mathematics, but most of the classical areas of random matrix theory remain the province of experts. Over the last decade, with the advent of matrix concentration inequalities, research has advanced to the point where we can conquer many (…
We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving computations of kernels based on random embeddings. Above that, the presented framework covers multivariate randomized functions. As a bypr…
Randomly initialized neural networks can linearly separate arbitrary sets.
problem Mapping two arbitrary sets to linearly separable sets.
method Randomly initialized one-layer neural networks with sufficient width.
result With high probability, these networks can transform two sets into linearly separable sets.
Study on signed graphs with random signs, focusing on community detection.
problem Community detection in signed stochastic block models.
method Strong concentration inequalities for adjacency and Laplacian matrices, applied to signed Laplacian matrix.
result The sign of the first eigenvector of the Laplacian matrix defines a weakly consistent estimator for balanced community detection.
This paper shows that deep learning (DL) representations of data produced by generative adversarial nets (GANs) are random vectors which fall within the class of so-called \textit{concentrated} random vectors. Further exploiting the fact that Gram matrices, of the type G=XTX with $X=[x_1,\ldots,x_n]\in \mathbb{R}…
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
The paper provides bounds for LSA with fixed stepsizes under random estimates.
problem Analyzing the performance of LSA algorithms with fixed stepsize.
method Non-asymptotic analysis based on new results about matrix moments and high probability bounds.
result Derives high probability bounds on LSA performance under weaker conditions than previous works.
New method improves feasibility of fitting Gaussian vectors to an ellipsoid.
problem Feasibility of fitting n Gaussian vectors to an ellipsoid boundary. method Improved concentration of Gram matrices using Bartl & Mendelson (2022) results.
result Feasibility of (P) with high probability when n≤d2/C. This paper gives new concentration inequalities for the spectral norm of a wide class of matrix martingales in continuous time. These results extend previously established Freedman and Bernstein inequalities for series of random matrices to the class of continuous time processes. Our analysis relies on a new supermarti…
We analyze the condition number of random feature matrices and prove their well-conditioned nature.
problem Understanding the condition number of random feature matrices and its impact on generalization error.
method Established concentration bounds and derived risk bounds for regression problems using random feature matrices.
result The risk associated with random feature matrices exhibits the double descent phenomenon, improving even with noise.
We study the concentration of random kernel matrices around their mean. We derive nonasymptotic exponential concentration inequalities for Lipschitz kernels assuming that the data points are independent draws from a class of multivariate distributions on Rd, including the strongly log-concave distributions u…
Matrices satisfying the Restricted Isometry Property (RIP) play an important role in the areas of compressed sensing and statistical learning. RIP matrices with optimal parameters are mainly obtained via probabilistic arguments, as explicit constructions seem hard. It is therefore interesting to ask whether a fixed mat…
Large deviations for fat tailed distributions, i.e. those that decay slower than exponential, are not only relatively likely, but they also occur in a rather peculiar way where a finite fraction of the whole sample deviation is concentrated on a single variable. The regime of large deviations is separated from the regi…
New tools in nonlinear random matrices improve understanding of the Sum of Squares hierarchy.
problem Improving the Sum of Squares (SoS) hierarchy's performance on average-case problems.
method Developed new tools in nonlinear random matrices and applied them to analyze the SoS hierarchy.
result Subexponential-time SoS lower bounds for various problems, offering evidence for the low-degree likelihood ratio hypothesis.
Quantum neural networks converge to Gaussian processes as they grow.
problem Understanding the convergence of quantum neural networks to Gaussian processes.
method Analyzing Haar random unitary and orthogonal deep QNNs, considering input states, measurement observables, and non-independence of unitary matrix entries.
result Quantum neural networks outputs converge to Gaussian processes in the limit of large Hilbert space dimension.
Sharp concentration inequalities for sub-Orlicz random variables with phase transition at α=2.
problem Developing concentration inequalities for sub-Orlicz random variables with phase transition.
method New theoretical analysis framework involving variance and min/max functions of Orlicz tails.
result Sharp concentration inequalities with phase transition at α=2 for sub-Orlicz random variables.
In this paper we study the concentration properties for the eigenvalues of kernel matrices, which are central objects in a wide range of kernel methods and, more recently, in network analysis. We present a set of concentration inequalities tailored for each individual eigenvalue of the kernel matrix with respect to its…
This work gives a simultaneous analysis of both the ordinary least squares estimator and the ridge regression estimator in the random design setting under mild assumptions on the covariate/response distributions. In particular, the analysis provides sharp results on the ``out-of-sample'' prediction error, as opposed to…
The paper analyzes data augmentation for precision matrix estimation in high dimensions.
problem Precision matrix estimation in high-dimensional settings.
method Linear shrinkage estimators and data augmentation methods.
result Concentration bounds for the quadratic error of estimators.
Sharp concentration results for sums of heavy-tailed random variables.
problem Analyzing sums of independent heavy-tailed random variables.
method Using concentration inequalities and large deviation principles for distributions satisfying specific tail bounds.
result Sharp concentration inequalities and large deviation results for sums of heavy-tailed random variables.
New concentration inequalities for tensors with heavy-tailed coefficients.
problem Developing bounds for Euclidean functions of tensors with sub-Weibull distributions.
method Extending concentration inequalities to sub-Weibull random tensors, using new inequalities for heavy-tailed random variables and martingale analysis.
result Established a phase transition between sub-gaussian and heavy-tailed regimes for Euclidean functions of tensors.
We investigate concentration inequalities for Dirichlet and Multinomial random variables.
The paper proves local laws for non-separable sample covariance matrices.
problem Analyzing non-separable sample covariance matrices with dependent or nonlinearly transformed data.
method Tensor network framework for analyzing fluctuation averaging in the presence of higher-order cumulant structure.
result Optimal averaged local law and full anisotropic local law for non-separable sample covariance matrices.
In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.
Study on random matrices in deep neural networks with IID entries.
problem Distribution of singular values in product of random matrices for deep neural networks.
method Random matrix theory with a streamlined approach for non-Gaussian data.
result Generalization of macroscopic universality property to non-Gaussian data.
The paper deals with distribution of singular values of product of random matrices arising in the analysis of deep neural networks. The matrices resemble the product analogs of the sample covariance matrices, however, an important difference is that the population covariance matrices, which are assumed to be non-random…
Study on estimating distances between covariance operators and Gaussian processes.
problem Estimating distances between covariance operators and Gaussian processes.
method Riemannian distances, concentration results for Hilbert space-valued random variables, RKHS covariance and cross-covariance operators.
result Both distances converge in the Hilbert-Schmidt norm and can be consistently and efficiently estimated.
Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.
problem Learning with vector-valued random features in infinite-dimensional settings.
method Direct analysis of risk functional, avoiding random matrix theory.
result Strong consistency and minimax optimal convergence rates established.
The paper corrects for node degree in spectral clustering using random walk Laplacian.
problem Node degree heterogeneity in spectral clustering.
method Graph spectral embedding using the random walk Laplacian.
result The embedding provides uniformly consistent estimates of degree-corrected latent positions.
Proves new concentration inequalities for sub-gaussian and sub-exponential variables.
problem Understanding functions of independent random variables better.
method Sub-gaussian and sub-exponential conditions, Rademacher complexities, Lipschitz function classes.
result Extension of Rademacher complexities to unbounded sub-exponential distributions.
Study on likelihood functions, associative equations, and Frobenius manifolds.
problem Maximum likelihood estimation and associativity equations in statistical models.
method Analyzes the cone of concentration matrices, log-likelihood function, and Frobenius manifolds.
result Maximum likelihood degree is indexed by components of Frobenius residuals.
The paper reviews and improves concentration inequalities for statistical inference.
problem Analyzing statistical inference in various settings with high-dimensional data.
method Review and improvement of concentration inequalities for different types of random variables and statistical measures.
result Fresh new results and improved bounds with sharper constants.
Approximating Gaussian Whittle-Matern Fields over Well-Centered Triangulations of Riemannian Manifolds
problem Convergent approximation of Gaussian Whittle-Matern fields on Riemannian manifolds
method Finite Element approximation of SPDEs
result Universal approximation of precision and covariance matrices
The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.
problem Sparse high-dimensional linear regression with random design and unknown error variance.
method Analysis of posterior concentration rates, employing techniques to address model misspecification.
result Adaptiveness and concentration rates of the posterior for sparse high-dimensional linear regression.
New method shows stochastic momentum can converge quickly on optimization problems.
problem Improving convergence of stochastic optimization methods.
method Stochastic heavy ball momentum with minibatching.
result Stochastic heavy ball momentum retains fast linear rate on quadratic problems.
New approach to concentration inequalities for unbounded state space dynamical systems.
problem Concentration inequalities for unbounded state space dynamical systems.
method Functional analytic framework, transport-entropy inequality.
result Exponential concentration inequalities for sampling from stationary distribution.
Survey on strong convergence in random matrices and its applications.
problem Understanding convergence of random matrices to operators.
method Analysis of operator norms of noncommutative polynomials.
result New insights and applications in random graphs, geometry, and operator algebras.
Repelling random walks improve graph-based sampling efficiency.
problem Efficient graph-based sampling and statistical estimation.
method Induces correlations between trajectories of an ensemble of walkers on a graph, maintaining unbiasedness.
result Improves concentration of statistical estimators on graphs.