Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.
problem Eigenvalue distribution of Wishart matrix with temporal correlation.
method Analysis of moments and convergence to deformed Marchenko-Pastur distribution for Gaussian process with temporal correlation.
result Eigenvalue distribution converges to deformed Marchenko-Pastur distribution with longer tail and higher peak.
We analyze the eigenvalue distribution of a neural network's kernel under specific scaling.
problem Analyzing the eigenvalue distribution of the Neural Tangent Kernel (NTK) of a neural network.
method Asymptotic analysis of the NTK matrix under given scaling conditions.
result The eigenvalue distribution is described as a free multiplicative convolution of the Marchenko-Pastur distribution and a deterministic distribution.
The study finds a trade-off between model size, test loss, and training loss for linear predictors.
problem Finding the optimal balance between model size, test loss, and training loss for linear predictors.
method Established an algorithm and distribution-independent trade-off using non-asymptotic analysis.
result Models with low test loss are either classical (close to noise level training loss) or modern (large number of parameters).
Iterative method 'Concent' corrects spectrum bias in covariance matrices.
problem Consistent bias in the spectrum of covariance matrices.
method 'Concent' iterative algorithm.
result Corrects spectrum bias for small and moderate dimensions.
Study examines NFT market dynamics using correlation and noise analysis.
problem Understanding correlations and noise in NFT market.
method Used detrended correlation coefficient and correlation matrix analysis.
result Correlation strength in NFT market is lower than in cryptocurrency markets.
A parallel optimization method for convex functions using Hessian sketching and debiasing.
problem Massively parallel optimization of convex functions with limited communication.
method Newton method with Hessian sketching and debiasing by workers, server averages descent directions.
result Approximation of Newton step with low-complexity adaptive sketching scheme.
Improved eigenvalue distribution method for financial data.
problem Noise and complexity in financial markets.
method Matrix H theory, hierarchical structure, informational cascade.
result Captures a larger fraction of data variance in financial markets.
Study detects signal in financial stock correlations using phase-ordering kinetics.
problem Detecting meaningful signals in financial stock return correlations.
method Stochastic field theory model to establish a detection threshold.
result Detection of a signal in the largest eigenvalues of the stock return correlation matrix.
Study on eigenvalue distribution of correlated time series deforming the semi-circle law.
problem Eigenvalue distribution of correlated time series differs from the semi-circle law.
method Analysis of Wigner random matrix with temporal correlation.
result Eigenvalue distribution converges to a deformed semi-circle law with longer tail and higher peak.
Analyzes Hessian spectrum for neural networks near optimal learning.
problem Understanding learning dynamics near optimal points in neural networks.
method Characterizes Hessian eigenspectrum for teacher-student problems, using analytical and numerical methods.
result The rank of the Hessian matrix determines effective number of parameters for non-linear networks.
This work analyzes self-attention matrices using random matrix theory.
problem Understanding the theoretical behavior of self-attention layers in neural networks.
method Asymptotic spectral analysis of the attention matrix, Gaussian equivalence, and linearization.
result The singular value distribution of the attention matrix is asymptotically characterized by a linear model.
Complex network analysis reveals dominant stocks in financial stock returns correlations.
problem Inferring financial stock returns correlations from complex network analysis.
method Simulated geometric Brownian motion for stocks, complex network analysis, eigenvector centrality, clustering.
result Returns correlation matrix is dominated by stocks with high eigenvector centrality and clustering.
A non-Hermitean extension of paradigmatic Wishart random matrices is introduced to set up a theoretical framework for statistical analysis of (real, complex and real quaternion) stochastic time series representing two "remote" complex systems. The first paper in a series provides a detailed spectral theory of non-Hermi…
Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.
problem Understanding the spectrum of random kernel matrices in polynomial scaling regimes.
method Investigates random matrices with nonlinear kernel functions applied to inner products of uniformly distributed vectors.
result The spectrum of the random kernel matrix is asymptotically equivalent to a simpler matrix model through free additive convolution.
Study reveals how attention helps in signal recovery from sequence models using random matrix theory.
problem Signal recovery from sequence models with attention mechanisms.
method Analysis of sample covariance matrices constructed from pooled sequence representations with attention weights.
result Optimal attention weights maximize signal-to-noise ratio and improve signal recovery.
Study on the geometric Dyson Brownian motion of non-square matrix products.
problem Understanding the spectrum of a product of non-square random matrices.
method Proportional depth-width limit followed by mean-field limit, solving Burgers equation.
result Free log-normal law is obtained in the identity-start case.
Study on overlaps of singular vectors in Gaussian matrix submatrices.
problem Analyzing overlaps of singular vectors in submatrices of Gaussian matrices.
method Utilizes dynamics of singular vectors and specific resolvents for Brownian trajectories.
result Explicit forms for limiting rescaled mean squared overlaps in the bulk of spectra.
Measures mode separation in high-dimensional densities via a reversible diffusion process.
problem Quantifying how sharply a distribution fragments into barrier-separated clusters in high dimensions.
method A unique reversible diffusion process with f as stationary distribution, extracting SSA and DA from its autocovariance matrix.
result Empirical autocovariance spectrum and readouts (SSA, DA) quantify mode separation using only samples and pretrained score-based models.
Spectral portfolio theory links neural networks to wealth dynamics via SGD weight matrices.
problem Understanding wealth dynamics from neural network training.
method Direct identification of weight matrices as portfolio allocation matrices, linking SGD forces to portfolio dynamics.
result Spectral properties of SGD weight matrices transition between additive and multiplicative regimes, influencing wealth dynamics.
Spectral denoising recovers meaningful network structure from noisy financial correlations.
problem Noise in empirical correlation matrices from financial returns obscures genuine interactions.
method Spectral decomposition to separate structured and random components.
result Structured networks derived from 10-16 eigenmodes exhibit stronger core-periphery organization and scale-free degree distributions.
Cryptocurrency market activity is decomposed into recurring and noise components, revealing patterns tied to macroeconomic reports.
problem Investigating temporal patterns of cryptocurrency market activity.
method Decomposition of market activity measures into recurring and noise components via correlation matrix formalism.
result Recurring market activity bursts coincide with significant U.S. macroeconomic reports, indicating their influence.
The paper decomposes unsupervised learning's generalization error into model, data, and variance components.
problem Understanding the components of unsupervised learning's generalization error.
method Information-geometric decomposition of the Kullback-Leibler generalization error.
result The optimal rank in ε-PCA is the noise floor, balancing model-error gain and data-bias cost. We study the statistical properties of the iterates generated by gradient descent, applied to the fundamental problem of least squares regression. We take a continuous-time view, i.e., consider infinitesimal step sizes in gradient descent, in which case the iterates form a trajectory called gradient flow. Our primary f…
This review covers recent results concerning the estimation of large covariance matrices using tools from Random Matrix Theory (RMT). We introduce several RMT methods and analytical techniques, such as the Replica formalism and Free Probability, with an emphasis on the Marchenko-Pastur equation that provides informatio…
We analyze kernel matrices in polynomial high-dimensional settings and explain double descent in KRR.
problem Understanding the spectrum of kernel matrices in polynomial high-dimensional settings and its implications for KRR risk.
method Generalized decomposition of kernel matrices into low-rank spike matrix, identity, and Gegenbauer matrix.
result The test error in KRR can exhibit double descent behavior, depending on effective regularization and signal-to-noise ratio.
Study investigates asymptotic risk of overparameterized models, including deep neural networks.
problem Understanding the risk of overparameterized models, especially deep neural networks.
method Analyzes the upper bound of an asymptotic risk of an estimator with penalization, combining Fisher information matrix properties and extended Marchenko-Pastur law.
result Generalized results valid for models without linear-in-feature constraints, indicating small asymptotic risk for specific structures like divisibility.
New models explain heavy-tailed behavior in neural networks.
problem Heavy-tailed spectral densities in neural networks.
method High-temperature Marchenko-Pastur (HTMP) ensemble models.
result Heavy-tailed behavior arises from three factors: data structure, training temperature, and eigenvector entropy.
Several recent trends in machine learning theory and practice, from the design of state-of-the-art Gaussian Process to the convergence analysis of deep neural nets (DNNs) under stochastic gradient descent (SGD), have found it fruitful to study wide random neural networks. Central to these approaches are certain scaling…
A criterion for training-free time-lagged spectral embeddings of multivariate time series
problem Applicability of fixed-length descriptors for multivariate time series
method Using a stationary Gaussian VAR(1) model and cosine similarity to classify descriptors
result D(τ) separates two classes when signals are approximately stationary and cross-channel temporal coupling is present
The paper explores solutions to the distributional Bellman equation in reinforcement learning.
problem Distributional reinforcement learning considers complete return distributions, not just expected returns.
method Study existence and uniqueness of solutions to general distributional Bellman equations, linking them to multivariate affine equations.
result Any solution to a distributional Bellman equation can be derived from a multivariate affine distributional equation.
Proposes vMF distribution for skewed elliptical distributions.
problem Skewed distributions not adequately modeled by symmetric distributions.
method Introduces von-Mises-Fisher (vMF) distribution to represent skewed elliptical distributions.
result vMF distribution provides an explicit and simple probability representation of skewed elliptical distributions.
Study calculates tail risk for various mixture distributions.
problem Estimating tail risk for complex distribution mixtures.
method Analyzes tail conditional expectation for location-scale mixtures of elliptical distributions.
result Developed methods for calculating tail risk in various distributions.
We realise the first and second Grushin distributions as symmetry reductions of the 3-dimensional Heisenberg distribution and 4-dimensional Engel distribution respectively. Similarly, we realise the Martinet distribution as an alternative symmetry reduction of the Engel distribution. These reductions allow us to derive…
Recent work has shown that deep generative models assign higher likelihood to out-of-distribution inputs than to training data. We show that a factor underlying this phenomenon is a mismatch between the nature of the prior distribution and that of the data distribution, a problem found in widely used deep generative mo…
Method uses optimal transport to complete distributional matrices.
problem Matrix completion for distributional data.
method Nearest neighbors in Wasserstein space.
result Method recovers distributions in Wasserstein metric.
Income and wealth distribution affect stability of a society to a large extent and high inequality affects it negatively. Moreover, in the case of developed countries, recently has been proven that inequality is closely related to all negative phenomena affecting society. So far, Econophysics papers tried to analyse in…
Study clusters distributions with known or unknown clusters using distribution testing.
problem Cluster distributions that are ε-far in total variation. method Distribution testing approach to establish upper and lower bounds on sample complexity.
result Achieves tight sample complexity bounds for all regimes (up to a logarithmic factor).
Gradually Truncated Log-normal distribution - Size distribution of firms Abstract Many natural and economical phenomena are described through power law or log- normal distributions. In these cases, probability decreases very slowly with step size compared to normal distribution. Thus it is essential to cut-off these di…
A new distribution family extends the α-stable distribution with a degree of freedom parameter.
problem Lack of moments in the α-stable distribution. method Wright function framework to combine and extend distribution families.
result Generalized α-stable distribution with valid moments. Paper develops a new method to improve model calibration under distribution shifts.
problem Challenges in uncertainty quantification with different training and test distributions.
method Develops multi-domain temperature scaling to handle distribution shifts.
result Outperforms existing methods on in-distribution and out-of-distribution test sets.
New class of heavy-tailed distributions shows weighted averages dominate individual variables.
problem Understanding and comparing risks in heavy-tailed distributions.
method Introducing a new class of heavy-tailed distributions and proving stochastic dominance relations.
result Weighted averages of random variables in this class are stochastically larger than individual variables.
Researchers derived formulas for joint moments of elliptical distributions.
problem Calculating joint moments of elliptical distributions.
method Used Stein's lemma and two different methods to derive expressions.
result New formulae for expectations of product of normally distributed random variables and simplified expressions for other distributions.
Paper proposes a new method for designing materials using deep learning.
problem Designing high-performance material distributions from given distributions.
method Iterative process of selecting, generating, and merging material distributions using a deep generative model.
result The method improves material performance through iterative refinement.
One-shot algorithm for feature-distributed kernel PCA reduces communication costs.
problem Efficiently perform kernel PCA in distributed computing environments.
method Inspired by dual relationship between sample-distributed and feature-distributed scenarios, proposes a one-shot algorithm for feature-distributed kernel PCA.
result The algorithm provides high-quality results with low communication costs, especially when eigenvalues decay fast.
Paper finds how many neurons are needed to approximate histogram distributions.
problem How many neurons are needed to approximate a target probability distribution?
method Examined for uniform input distribution and histogram target distributions, using efficient neural net construction.
result Obtained a new upper bound on the number of required neurons, strictly better than previous bounds.
Paper analyzes origami slope gaps and their distribution, finding a unique pattern.
problem Analyzing slope gaps in origami surfaces.
method Derived slope gap distribution of a specific origami by considering return times under the horocycle flow.
result Found a unique distribution of origami slope gaps, not a sum of scaled Hall distributions.
A new distributed clustering framework using distributional kernel.
problem Clustering in distributed networks with arbitrary shapes, sizes, and densities.
method Distributed Clustering based on Distributional Kernel (KDC) using similarity of distributions.
result KDC guarantees equivalent clustering outcomes to centralized methods, reduces runtime, and discovers arbitrary clusters.
Paper introduces a new distributional successor measure for reinforcement learning.
problem Learning the distributional consequences of behavior in reinforcement learning.
method Formulates distributional successor measure as a distribution over distributions, proposes algorithm to learn it from data.
result Demonstrates zero-shot risk-sensitive policy evaluation.