A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Method estimates sparse inverse covariance and partial correlation matrices efficiently.
Non-negative matrix factorization is a basic tool for decomposing data into the feature and weight matrices under non-negativity constraints, and in practice is often solved in the alternating minimization framework. However, it is unclear whether such algorithms can recover the ground-truth feature matrix when the wei…
CCP clusters correlated features and projects them to 1D for efficient dimensionality reduction.
We introduce a method to predict which correlation matrix coefficients are likely to change their signs in the future in the high-dimensional regime, i.e. when the number of features is larger than the number of samples per feature. The stability of correlation signs, two-by-two relationships, is found to depend on thr…
The paper extends Pearson correlation to multi-variables, useful for noise measurement and feature selection.
DPERC efficiently estimates covariance matrices for mixed data with missing values.
This paper compares imputation and direct parameter estimation methods for missing data in correlation matrix visualization.
The paper reduces the complexity of financial market correlation matrices to a 2x2 matrix.
Portfolio allocation and risk management make use of correlation matrices and heavily rely on the choice of a proper correlation matrix to be used. In this regard, one important question is related to the choice of the proper sample period to be used to estimate a stable correlation matrix. This paper addresses this qu…
We consider analysis of relational data (a matrix), in which the rows correspond to subjects (e.g., people) and the columns correspond to attributes. The elements of the matrix may be a mix of real and categorical. Each subject and attribute is characterized by a latent binary feature vector, and an inferred matrix map…
The paper models financial correlation matrices using permutation invariant Gaussian models and predicts market anomalies.
In this study, we attempted to determine how eigenvalues change, according to random matrix theory (RMT), in stock market data as the number of stocks comprising the correlation matrix changes. Specifically, we tested for changes in the eigenvalue properties as a function of the number and type of stocks in the correla…
Algorithm learns stock correlation matrix embedding using graph machine learning.
This paper presents a robust matrix elastic net based canonical correlation analysis (RMEN-CCA) for multiple view unsupervised learning problems, which emphasizes the combination of CCA and the robust matrix elastic net (RMEN) used as coupled feature selection. The RMEN-CCA leverages the strength of the RMEN to distill…
Bootstrapping regularizes singular correlation matrices, reducing the need for complex regularization.
A central problem in machine learning and pattern recognition is the process of recognizing the most important features. In this paper, we provide a new feature selection method (DRPT) that consists of first removing the irrelevant features and then detecting correlations between the remaining features. Let $D=[A\mid \…
Correlations between asset returns are important in many financial applications. In recent years, multivariate volatility models have been used to describe the time-varying feature of the correlations. However, the curse of dimensionality quickly becomes an issue as the number of correlations is for asse…
Feature selection is a dimensionality reduction technique that selects a subset of representative features from high dimensional data by eliminating irrelevant and redundant features. Recently, feature selection combined with sparse learning has attracted significant attention due to its outstanding performance compare…
Unified framework for calculating Shapley values with correlated features.
NARD extends ARD for linear models, promoting sparsity and correlation structure.
Survey of methods to recover CI graphs from feature relationships.
The dynamics of the equal-time cross-correlation matrix of multivariate financial time series is explored by examination of the eigenvalue spectrum over sliding time windows. Empirical results for the S&P 500 and the Dow Jones Euro Stoxx 50 indices reveal that the dynamics of the small eigenvalues of the cross-correlat…
We uncover scaling laws and statistical structure in complex datasets.
We discuss some methods to quantitatively investigate the properties of correlation matrices. Correlation matrices play an important role in portfolio optimization and in several other quantitative descriptions of asset price dynamics in financial markets. Specifically, we discuss how to define and obtain hierarchical …
Feature missing is a serious problem in many applications, which may lead to low quality of training data and further significantly degrade the learning performance. While feature acquisition usually involves special devices or complex process, it is expensive to acquire all feature values for the whole dataset. On the…
We propose improved methods to identify stock groups using the correlation matrix of stock price changes. By filtering out the marketwide effect and the random noise, we construct the correlation matrix of stock groups in which nontrivial high correlations between stocks are found. Using the filtered correlation matrix…
We present a scheme for online, unsupervised state discovery and detection from streaming, multi-featured, asynchronous data in high-frequency financial markets. Online feature correlations are computed using an unbiased, lossless Fourier estimator. A high-speed maximum likelihood clustering algorithm is then used to f…
Interpretable machine-learning models can be unstable under multicollinearity, leading to oscillatory weights that do not reflect meaningful contributions.
Correlation matrices play a key role in many multivariate methods (e.g., graphical model estimation and factor analysis). The current state-of-the-art in estimating large correlation matrices focuses on the use of Pearson's sample correlation matrix. Although Pearson's sample correlation matrix enjoys various good prop…
This paper proposes a method to select relevant features for multi-label learning.
In this paper, we apply tools from the random matrix theory (RMT) to estimates of correlations across volatility of various assets in the S&P 500. The volatility inputs are estimated by modeling price fluctuations as GARCH(1,1) process. The corresponding correlation matrix is constructed. It is found that the distribut…
We simplify matrix computations for block matrices, especially useful for covariance and correlation matrices.
Model forecasts market structure from financial networks using machine learning.
This work explores how feature decorrelation improves self-supervised learning.
MPVAE learns latent embeddings and label correlations for multi-label classification.
The estimation of covariance matrices of gene expressions has many applications in cancer systems biology. Many gene expression studies, however, are hampered by low sample size and it has therefore become popular to increase sample size by collecting gene expression data across studies. Motivated by the traditional me…
Graph attention network improves MLTC by capturing label dependencies.
The paper provides exact multivariate amplitude distributions for non-stationary Gaussian or algebraic fluctuations.
A method to complete incomplete correlation matrices using maximum entropy.
We consider a challenging multi-label classification problem where both feature matrix $\X$ and label matrix $\Y$ have missing entries. An existing method concatenated $\X$ and $\Y$ as $[\X; \Y]$ and applied a matrix completion (MC) method to fill the missing entries, under the assumption that $[\X; \Y]$ is of low-rank…
Financial markets are prominent examples for highly non-stationary systems. Sample averaged observables such as variances and correlation coefficients strongly depend on the time window in which they are evaluated. This implies severe limitations for approaches in the spirit of standard equilibrium statistical mechanic…
We develop a framework for analyzing extreme values in correlated financial data.
Using Random Matrix Theory one can derive exact relations between the eigenvalue spectrum of the covariance matrix and the eigenvalue spectrum of its estimator (experimentally measured correlation matrix). These relations will be used to analyze a particular case of the correlations in financial series and to show that…
We perform a comparative analysis of the Chinese stock market around the occurrence of the 2008 crisis based on the random matrix analysis of high-frequency stock returns of 1228 stocks listed on the Shanghai and Shenzhen stock exchanges. Both raw correlation matrix and partial correlation matrix with respect to the ma…
The paper analyzes heavy-tailed multivariate distributions in non-stationary systems using random matrix theory.
Bayesian feature allocation models are a popular tool for modelling data with a combinatorial latent structure. Exact inference in these models is generally intractable and so practitioners typically apply Markov Chain Monte Carlo (MCMC) methods for posterior inference. The most widely used MCMC strategies rely on an e…
We analyse the structure of the distribution of eigenvalues of the stock market correlation matrix with increasing length of the time series representing the price changes. We use 100 highly-capitalized stocks from the American market and relate result to the corresponding ensemble of Wishart random matrices. It turns …
Study on eigenvalue distribution of correlated time series deforming the semi-circle law.