Study finds the minimum number of finite Gaussian mixtures for best approximation.
problem Finding the minimum number of finite Gaussian mixtures for best approximation.
method Local moment matching for upper bound and spectral analysis for lower bound.
result Corrects a previous lower bound in the case of Gaussian mixing distributions.
Paper extends trigonometric summation formula with weights.
problem Trigonometric summation formula by Grigor'yan, Lin and Yau.
method Weighted trigonometric summation formula derivation.
result Extension of trigonometric summation formula.
Study on Gaussian ensemble of matrix products with mixed moments computed.
problem Understanding the statistical properties of matrix products of Gaussian matrices.
method Analysis of a multi-Wishart ensemble and enumeration of non-crossing pairings.
result Mixed moments of the product matrix are computed and found to be weighted by Fuss-Catalan numbers at large N. The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
Paper provides unbiased spectral moment estimates from finite data.
problem Challenges in estimating spectral moments from limited data.
method Dynamic programming approach to estimate spectral moments of kernel integral operator.
result Demonstrates consistency with theoretical spectra and practical utility in neural networks.
Study resolvent convergence for random matrices with general covariance profiles.
problem Analyzing resolvent convergence for random matrices with non-identically distributed columns.
method Using moments of quadratic forms and deterministic equivalents, the study provides bounds on the trace of matrix products.
result The trace of matrix products is close to the trace of a deterministic equivalent, controlled by matrix norms.
The paper gives a constructive method, based on greedy algorithms, that provides for the classes of functions with small mixed smoothness the best possible in the sense of order approximation error for the m-term approximation with respect to the trigonometric system.
This paper explores how Transformers predict next tokens in autoregressive tasks.
problem Understanding the success of Transformers in autoregressive learning.
method Trained a Transformer on a next-token prediction task, focusing on commuting orthogonal matrices.
result Trained Transformers can be seen as implementing gradient descent for a specific objective function.
The paper introduces a new method for tail bounds of random vectors and matrices.
problem Estimating norms of random vectors and matrices under moment assumptions.
method Variational tail bounds for norms of random vectors and matrices.
result Dimension-free concentration inequalities for various norms of random vectors and matrices.
Computed distortion coefficients for the α-Grushin plane.
problem Analyzing the distortion coefficients of the α-Grushin plane.
method Using generalised trigonometric functions and synthetic curvature conditions.
result Estimates for distortion coefficients and a curvature condition conjecture.
New estimator learns symmetric dynamics from few observations.
problem Learning parameters of stochastic linear dynamics from limited data.
method Method of moments estimator using T=O(logN) observations. result Achieves small maximum element-wise error on symmetric matrices.
Study inequalities for singular values of rectangular matrices.
problem Inequalities for singular values of rectangular matrices.
method Study convex cones associated to isotropic representations of symmetric spaces.
result Describe inequalities by cohomological conditions.
Deep Gaussian Processes are reinterpreted as deep trigonometric networks for tractable inference.
problem Challenging inference in DGPs due to intractable marginalization in latent function space.
method Viewing DGPs as deep trigonometric networks with Bochner's theorem, and using the wide limit with a bottleneck to translate DGPs into deep trigonometric networks.
result The weight space view yields the same effective covariance functions as obtained in function space, and varying prior distributions over network parameters is equivalent to employing different kernels.
Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse sec…
Deformed σ-models linked to Ricci flow and Toda theories.
problem Understanding the relationship between deformed σ-models and geometric flows.
method Exploring trigonometric deformations of CP^n-1 models and their duals, linking to Ricci flow and Toda field theories.
result Trigonometric deformations of CP^n-1 models solve the Ricci flow equation and relate to Toda field theories.
Study heavy-tailed weights' impact on neural network's spectral distribution.
problem Analyzing spectral distribution of conjugate kernel matrices with heavy-tailed weights.
method Computed limiting eigenvalue distribution through moments, considering heavy-tailed distributions and nonlinear activation functions.
result Heavy-tailed weights induce strong correlations, leading to fundamentally different spectral behavior.
Sigma models linked to Gross-Neveu models via quiver varieties.
problem Understanding the relationship between sigma models and Gross-Neveu models.
method Exploring the mathematical correspondence between sigma models and Gross-Neveu models, including their geometric and trigonometric/elliptic deformations.
result Sigma models are mathematically equivalent to Gross-Neveu models under certain conditions.
New approach for classification using trigonometric polynomial kernels from signal processing.
problem Classifying objects in arbitrary compact metric spaces.
method Localized trigonometric polynomial kernels for separating different probability measures.
result Minimal number of queried labels for perfect classification.
We study products of random matrices in the regime where the number of terms and the size of the matrices simultaneously tend to infinity. Our main theorem is that the logarithm of the ℓ2 norm of such a product applied to any fixed vector is asymptotically Gaussian. The fluctuations we find can be thought of as a…
This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically…
Extracts representative scenarios from large data panels.
problem Creating representative scenarios from large data panels.
method Two novel algorithms: one identifies new scenarios, the other selects known important data points.
result Efficient algorithms for consistent scenario-based modeling and multi-dimensional numerical integration.
We introduce SARR for symmetric object pose estimation, improving CNN performance.
problem Ambiguities in symmetric object orientations hinder deep learning pose estimation.
method Numeric rotation representation using symmetry-derived trigonometric identities.
result SARR enables standard CNNs to achieve state-of-the-art performance.
Spectral learning extends matrix methods to tensors for better latent variable modeling.
problem Limitations of matrix-based spectral methods in capturing non-Gaussian data.
method Extend spectral decomposition to tensor-based methods for higher-order moments.
result Tensor decomposition can identify latent effects missed by matrix methods.
This thesis consists of two independent parts: random matrices, which form the first one-third of this thesis, and machine learning, which constitutes the remaining part. The main results of this thesis are as follows: a necessary and sufficient condition for the inverse moments of (m,n,β)-Laguerre matrices and compo…
Study on random matrices in deep neural networks using Gaussian data.
problem Distribution of singular values in product of random matrices in deep learning.
method Free probability theory combined with standard techniques of random matrix theory.
result Justification for applying free probability theory to non-independent random data matrices.
Study geodesics on Grushin spaces, proving upper bounds on conjugate times.
problem Classify geodesics on higher-dimensional Grushin spaces.
method Solve Hamilton's equations using calculus of generalized trigonometric functions, analyze symmetries, and use density arguments.
result Prove a conjectured cut time provides an upper bound on conjugate times.
In this paper we investigate the construction of state models for link invariants using representations of the braid group obtained from various gauge choices for a solution of the trigonometric Yang-Baxter equation. Our results show that it is possible to obtain invariants of regular isotopy (as defined by Kauffman) w…
Study on random matrices in deep neural networks with IID entries.
problem Distribution of singular values in product of random matrices for deep neural networks.
method Random matrix theory with a streamlined approach for non-Gaussian data.
result Generalization of macroscopic universality property to non-Gaussian data.
This work develops efficient methods for computing moments of Gaussian mixtures.
problem Efficient computation of moments for Gaussian mixtures with large dimensions.
method Theory and numerical methods for implicit computations with moment tensors of Gaussian mixtures.
result Reduced computational and storage costs for moment tensors of Gaussian mixtures.
Study shows overparameterization helps in generalizing from smooth interpolants.
problem Understanding generalization in overparameterized linear models.
method Analysis of random Fourier series model with weighted trigonometric interpolation.
result Weighted trigonometric interpolation leads to lower generalization error in overparameterized scenarios.
We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis …
How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove …
Since the 1970's, physicists and mathematicians who study random matrices in the GUE or GOE models are aware of intriguing connections between integrals of such random matrices and enumeration of graphs on surfaces. We establish a new aspect of this theory: for random matrices sampled from the group $\mathcal{U}\left(n…
A novel approach models rating transitions using Lie groups and Deep Learning.
problem Modeling rating transitions with geometric properties and stochastic processes.
method Introducing Itô-SDEs on Lie groups, using TimeGAN for calibration, and examining rating matrix properties.
result The geometric approach using Lie groups and Deep Learning generates a good fit for rating transitions.
The paper provides bounds for LSA with fixed stepsizes under random estimates.
problem Analyzing the performance of LSA algorithms with fixed stepsize.
method Non-asymptotic analysis based on new results about matrix moments and high probability bounds.
result Derives high probability bounds on LSA performance under weaker conditions than previous works.
This paper analyzes various forms of concentrated liquidity in decentralized finance.
problem Understanding different models of concentrated liquidity in DeFi.
method Algebraic and geometric analysis of existing models.
result An authoritative overview of concentrated liquidity models in DeFi.
Study Fourier estimator for spot volatility with unbounded coefficients and jumps.
problem Estimating spot volatility with unbounded coefficients and jumps in price process.
method Fourier estimator for spot volatility, convergence analysis for unbounded coefficients and jumps.
result Convergence of trigonometric polynomial to volatility's path, almost sure convergence of reconstructed volatility.
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approxi…
We provide bounds for kernel matrices and new approximations for high-dimensional data.
problem Approximating high-dimensional empirical kernel matrices.
method Decoupling results for U-statistics and non-commutative Khintchine inequality.
result New tighter approximations for inner-product kernel matrices.
The classical Tait-Kneser theorem states that the osculating circles of a smooth plane curve, free from curvature extrema, are pairwise disjoint. We prove a number of analogs of this theorem, e.g., for ovals of osculating cubics, osculating polynomials and trigonometric polynomials; in each case, we will obtain a non-d…
The present paper considers volume formulae, as well as trigonometric identities, that hold for a tetrahedron in 3-dimensional spherical space of constant sectional curvature +1. The tetrahedron possesses a certain symmetry: namely rotation through angle π in the middle points of a certain pair of its skew edges.
Euler explored spherical geometry using trigonometric formulae and solid geometry methods.
problem Developing trigonometric formulae for spherical geometry.
method Used calculus of variations and classical solid geometry methods.
result Established trigonometric formulae and computed spherical triangle areas.
This paper shows universality in spectrum behavior for random inner-product kernel matrices in polynomial regime.
problem Understanding spectrum behavior of random inner-product kernel matrices in polynomial regime.
method Analyzing matrices formed by a nonlinear function applied entrywise to a sample-covariance matrix, considering i.i.d. entries with all finite moments.
result The spectrum of random inner-product kernel matrices is universally described by the free convolution of the semicircular and Marčenko-Pastur distributions, with relative weights given by expanding the nonlinear function in the Hermite basis.
Study spectral density of neural networks using resolvent method.
problem Investigate spectral density of neural networks with random feature matrices.
method Use resolvent method from random matrix theory, cumulant expansion.
result Impossible to preserve singular value distribution with additive bias.
Reservoir computing's success depends on mapping different input time series to separable states.
problem Quantifying the ability of random linear reservoirs to map different input time series.
method Mathematical framework using spectral properties of the connectivity matrix.
result Separation capacity is fully characterized by the spectral properties of the connectivity matrix.
In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages of squared past gradients. Maintaining these per-parameter second-moment estimators requires memory equal to the number of parameters. For …
Many pattern recognition methods rely on statistical information from centered data, with the eigenanalysis of an empirical central moment, such as the covariance matrix in principal component analysis (PCA), as well as partial least squares regression, canonical-correlation analysis and Fisher discriminant analysis. R…
Unified bounds for iterative algorithms with Gaussian data matrices.
problem Establishing non-asymptotic bounds for iterative algorithms with Gaussian data.
method Explicit coupling between iterates and Gaussian process with deterministic covariance.
result Tight, dimension-free bounds for generalized first-order methods.