Generalizes randomized SVD for better matrix approximations using Gaussian vectors.
problem Computing accurate rank-k approximations of matrices with limited data.
method Extends randomized SVD to multivariate Gaussian vectors, incorporating prior knowledge and using Gaussian processes.
result Demonstrates improved accuracy in approximating matrices and Hilbert-Schmidt operators.
Study improves error bounds for sparse regression with heavy-tailed covariates.
problem Estimating sparse coefficients in linear regression with heavy-tailed covariates.
method Employed an ℓ1-penalized Huber regression method. result Error bound identical to Gaussian case for L-subexponential covariates. Bayesian approach approximates probability functions of Gaussian mixtures.
problem Approximating probability functions of non-spherical Gaussian mixtures.
method Bayesian decomposition, spherical radial decomposition, random sampling.
result Established differentiability and integral representation of gradient for probability functions.
Stochastic trace estimation with tensor train random vectors
problem Stochastic trace estimation for large-scale matrices
method Gaussian random tensor train vectors
result Median-of-means variant achieves dimension-independent guarantees
New method estimates Gaussian vector functions more efficiently.
problem Estimating functions of Gaussian vectors with high dimensions.
method Combines randomized dimension reduction and PCA.
result Algorithm outperforms Monte Carlo method by a factor of d.
Gaussian random vectors exhibit the loss of dimension phenomena, which relate to their joint survival tail behaviour. Besides, the fact that the components of such vectors are light-tailed complicates the approximations of various multivariate risk measures significantly. In this contribution we derive precise approxim…
We describe various sets of conditional independence relationships, sufficient for qualitatively comparing non-vanishing squared partial correlations of a Gaussian random vector. These sufficient conditions are satisfied by several graphical Markov models. Rules for comparing degree of association among the vertices of…
We introduce a new functional measure of tail dependence for weakly dependent (asymptotically independent) random vectors, termed weak tail dependence function. The new measure is defined at the level of copulas and we compute it for several copula families such as the Gaussian copula, copulas of a class of Gaussian mi…
We develop time-uniform confidence spheres for estimating means of random vectors.
problem Sequential mean estimation in high-dimensional spaces.
method Derive time-uniform confidence sphere sequences (CSSs) for various types of random vectors.
result Optimal CSSs for log-concave, sub-Gaussian, and sub-ψ random vectors. Study analyzes perturbations in singular subspaces under random noise.
problem Understanding singular vector and subspace changes in signal-plus-noise models.
method Generalized Davis-Kahan-Wedin theorem for any unitarily invariant norm, considering ℓ∞ and ℓ2,∞ bounds. result Fine-grained insights into singular vector and subspace perturbations, including ℓ∞ and ℓ2,∞ bounds. We study the problem of estimating the mean of a random vector X given a sample of N independent, identically distributed points. We introduce a new estimator that achieves a purely sub-Gaussian performance under the only condition that the second moment of X exists. The estimator is based on a novel concept of a…
The paper introduces a new method for tail bounds of random vectors and matrices.
problem Estimating norms of random vectors and matrices under moment assumptions.
method Variational tail bounds for norms of random vectors and matrices.
result Dimension-free concentration inequalities for various norms of random vectors and matrices.
A new method for approximating softmax and Gaussian kernels with reduced error.
problem Approximating softmax and Gaussian kernels with low error.
method Simplex Random Features (SimRFs) and SimRFs+.
result SimRFs provide the smallest MSE among weight-independent geometrically-coupled PRF mechanisms.
Paper studies tensor models using random matrix theory.
problem Analyzing asymmetric order-d spiked tensor models with Gaussian noise.
method Uses variational definition of singular vectors and values, constructs equivalent spiked symmetric block-wise random matrix from tensor contractions.
result Characterizes asymptotic singular values and alignments of singular vectors with true spike components.
Randomized algorithm solves vector-valued regression problems with low-rank operators.
problem Vector-valued regression problems involving infinite-dimensional spaces.
method Randomized Reduced Rank Regression (R4) using Gaussian sketching for optimization.
result R4 estimators are efficient and accurate, with empirical risk close to optimal.
Study on complexity of random polynomials with deterministic spikes, identifying phase transitions.
problem Complexity of random Gaussian polynomials with deterministic spikes on a sphere.
method Variational formulas, Kac-Rice formula, determinant asymptotics of finite-rank perturbation of Gaussian Wigner matrices.
result Identification of a topological phase transition in the complexity function.
This paper shows that deep learning (DL) representations of data produced by generative adversarial nets (GANs) are random vectors which fall within the class of so-called \textit{concentrated} random vectors. Further exploiting the fact that Gram matrices, of the type G=XTX with $X=[x_1,\ldots,x_n]\in \mathbb{R}…
Gaussianization flows transform any random vector into a Gaussian, enabling efficient computation and sample generation.
problem Transforming any random vector into a Gaussian for efficient computation and sample generation.
method Iterative Gaussianization and normalizing flow model.
result Gaussianization flows are universal approximators and achieve better performance on tabular datasets.
Improved spectral method recovers sparse vectors in random subspaces.
problem Recovering a sparse vector in a random subspace with sub-Gaussian entries.
method Improved spectral method with leave-one-out analysis.
result Spectral method recovers sparse vectors with high probability under certain conditions.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
New bounds for optimal transport using Gaussian processes and rate-distortion functions.
problem Finding bounds for entropic optimal transport with mutual information constraints.
method Lifting technique to construct a Gaussian process and applying the majorizing measure theorem.
result Maximum expected inner product is equivalent to a truncated integral involving the rate-distortion function.
Study spectral properties of sparse random graphs to recover latent vectors.
problem Recovering latent vectors in sparse random geometric graphs.
method Analyzes spectral concentration and uses orthogonal polynomial expansions, decoupling, and matrix concentration.
result Sharpens spectral norm bounds and proves exact recovery for Gaussian mixture models.
Multi-output Gaussian processes (MOGP) are probability distributions over vector-valued functions, and have been previously used for multi-output regression and for multi-class classification. A less explored facet of the multi-output Gaussian process is that it can be used as a generative model for vector-valued rando…
New method improves feasibility of fitting Gaussian vectors to an ellipsoid.
problem Feasibility of fitting n Gaussian vectors to an ellipsoid boundary. method Improved concentration of Gram matrices using Bartl & Mendelson (2022) results.
result Feasibility of (P) with high probability when n≤d2/C. The paper extends consistency results for sequential design strategies to vector-valued Gaussian processes.
problem Estimating excursion sets of vector-valued Gaussian processes.
method Clarifying the connection between continuous Gaussian processes and Gaussian measures in Banach spaces, extending concepts and properties from scalar-valued settings to vector-valued settings.
result Consistency results for sequential design strategies can be applied to vector-valued Gaussian processes.
Sparse neural encoding can store more memories as targets become sparser.
problem Storing sparse input-target associations in neural networks.
method Mathematical proofs using properties of random polytopes and sub-gaussian random vector variables.
result The capacity of neural maps increases with sparsity in target layers.
We prove that a Gaussian ensemble of smooth random sections of a real vector bundle over compact manifold canonically defines a metric on the bundle together with a connection compatible with it. Additionally, we prove a refined Gauss-Bonnet-Chern theorem stating that if the bundle and the manifold are oriented, then t…
In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data p and their number $…
Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse sec…
Study on overlaps of singular vectors in Gaussian matrix submatrices.
problem Analyzing overlaps of singular vectors in submatrices of Gaussian matrices.
method Utilizes dynamics of singular vectors and specific resolvents for Brownian trajectories.
result Explicit forms for limiting rescaled mean squared overlaps in the bulk of spectra.
We study a distributed estimation problem in which two remotely located parties, Alice and Bob, observe an unlimited number of i.i.d. samples corresponding to two different parts of a random vector. Alice can send k bits on average to Bob, who in turn wants to estimate the cross-correlation matrix between the two par…
Random matrix ensembles yield uniform distributions on manifolds.
problem Understanding distributions of vectors in random matrix ensembles.
method Analyzing eigenvalues, singular values, and Autonne-Takagi vectors of various random matrix ensembles.
result Uniform distributions on specific manifolds for different types of random matrix ensembles.
Improved perturbation reduces matrix condition number to O(n) with minimal storage.
problem Reducing the condition number of deterministic matrices for efficient algorithmic use.
method Introduced pattern matrices and sparse perturbations with dependent entries.
result Condition number reduced to O(n) with O(n) random numbers in O(log n) precision.
Random sinusoidal features are a popular approach for speeding up kernel-based inference in large datasets. Prior to the inference stage, the approach suggests performing dimensionality reduction by first multiplying each data vector by a random Gaussian matrix, and then computing an element-wise sinusoid. Theoretical …
Scalable algorithm for sampling Gaussian processes using sparse grids and preconditioners.
problem Generating high-dimensional Gaussian random vectors for GP sampling is computationally challenging.
method Proposes a scalable algorithm using inducing points approximation with sparse grids and additive Schwarz preconditioners.
result Demonstrates the efficacy and accuracy of the proposed method through experiments and comparisons.
Study on using random subspaces for ERM with various loss functions.
problem Improving learning accuracy with computational savings from random subspaces.
method Random subspaces of a hypothesis space, considering data-dependent subspaces.
result Unified analysis showing computational efficiency can be improved without performance loss.
This work introduces efficient sampling methods for Gaussian processes by focusing on pathwise conditioning.
problem Intractable mathematical expressions in Gaussian process posteriors limit practical applications.
method Investigates a pathwise interpretation of conditioning to derive efficient sampling methods.
result Derives a general family of approximations that allow for efficient sampling of Gaussian process posteriors.
Gaussian processes adapted for non-Euclidean spaces enhance decision-making.
problem Applying Gaussian processes in non-Euclidean spaces.
method Developed pathwise conditioning and Gaussian process models over non-Euclidean spaces.
result Efficient Gaussian process models for non-Euclidean spaces.
This paper presents a sequential randomized lowrank matrix factorization approach for incrementally predicting values of an unknown function at test points using the Gaussian Processes framework. It is well-known that in the Gaussian processes framework, the computational bottlenecks are the inversion of the (regulariz…
The paper corrects for node degree in spectral clustering using random walk Laplacian.
problem Node degree heterogeneity in spectral clustering.
method Graph spectral embedding using the random walk Laplacian.
result The embedding provides uniformly consistent estimates of degree-corrected latent positions.
This paper considers inference over distributed linear Gaussian models using factor graphs and Gaussian belief propagation (BP). The distributed inference algorithm involves only local computation of the information matrix and of the mean vector, and message passing between neighbors. Under broad conditions, it is show…
In this paper we show that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices. This class includes several matrices for which matrix-vector multiply can be computed in log-linear time, providing efficient dimensionality reduction of gene…
The study assesses low-rank approximations in Gaussian Process regression.
problem Improving the efficiency of Gaussian Process regression while maintaining accuracy.
method Analyzes two low-rank approximations: random Fourier features and Mercer expansion truncation, and bounds the divergence and error between exact and approximate models.
result Theoretical bounds on the divergence and error between exact and approximate Gaussian Process models are provided.
Several important families of computational and statistical results in machine learning and randomized algorithms rely on uniform bounds on quadratic forms of random vectors or matrices. Such results include the Johnson-Lindenstrauss (J-L) Lemma, the Restricted Isometry Property (RIP), randomized sketching algorithms, …
Spectral embedding is a procedure which can be used to obtain vector representations of the nodes of a graph. This paper proposes a generalisation of the latent position network model known as the random dot product graph, to allow interpretation of those vector representations as latent position estimates. The general…
We propose a probabilistic enhancement of standard kernel Support Vector Machines for binary classification, in order to address the case when, along with given data sets, a description of uncertainty (e.g., error bounds) may be available on each datum. In the present paper, we specifically consider Gaussian distributi…
This paper analyzes the Lipschitz constants of deep neural networks with random weights.
problem Estimating the Lipschitz constants of deep neural networks with random parameters.
method High probability upper and lower bounds derived for ReLU neural networks with He initialization.
result The behavior of the Lipschitz constant varies significantly between p∈[1,2) and p∈[2,∞]. The paper develops sampling methods for ocean phenomena based on temperature and salinity measurements.
problem Improving oceanographic sampling with limited resources.
method Design criterion based on uncertainty in excursions of vector-valued Gaussian random fields.
result Demonstrates effective exploration of ambiguous regions for data-driven sampling.