arXiv research

New estimator reveals intraday betas mainly driven by correlations.

Improved portfolio optimization using Kendall-like correlation coefficients.

This paper benchmarks Bayesian models' ability to estimate predictive correlations, especially for active learning.

Paper studies estimating asset correlations across sectors.

New Hermite series estimator for Spearman rank correlation in non-stationary data.

Paper introduces $ρ$-Perfect to estimate model-human correlation in subjective datasets.

Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms …

Method estimates sparse inverse covariance and partial correlation matrices efficiently.

This paper compares imputation and direct parameter estimation methods for missing data in correlation matrix visualization.

The risk of a credit portfolio depends crucially on correlations between the probability of default (PD) in different economic sectors. Often, PD correlations have to be estimated from relatively short time series of default rates, and the resulting estimation error hinders the detection of a signal. We present statist…

Proposes PSCCA for estimating correlations and canonical correlations in sparse count data.

The instability of historical risk factor correlations renders their use in estimating portfolio risk extremely questionable. In periods of market stress correlations of risk factors have a tendency to quickly go well beyond estimated values. For instance, in times of severe market stress, one would expect with certain…

Correlation matrices play a key role in many multivariate methods (e.g., graphical model estimation and factor analysis). The current state-of-the-art in estimating large correlation matrices focuses on the use of Pearson's sample correlation matrix. Although Pearson's sample correlation matrix enjoys various good prop…

Estimates mean of distributed vectors with sparsification and spatial/temporal correlations.

Paper proposes a simple estimator for DPP correlation kernels.

We examine the performance of six estimators of the power-law cross-correlations -- the detrended cross-correlation analysis, the detrending moving-average cross-correlation analysis, the height cross-correlation analysis, the averaged periodogram estimator, the cross-periodogram estimator and the local cross-Whittle e…

In this paper, we apply tools from the random matrix theory (RMT) to estimates of correlations across volatility of various assets in the S&P 500. The volatility inputs are estimated by modeling price fluctuations as GARCH(1,1) process. The corresponding correlation matrix is constructed. It is found that the distribut…

Estimates covariance matrices with correlations between samples.

We consider a binary sequence generated by thresholding a hidden continuous sequence. The hidden variables are assumed to have a compound symmetry covariance structure with a single parameter characterizing the common correlation. We study the parameter estimation problem under such one-parameter models. We demonstrate…

Enhances sensitivity analysis for correlated inputs.

A new method estimates conditional canonical correlations using random forests.

Financial correlation matrices measure the unsystematic correlations between stocks. Such information is important for risk management. The correlation matrices are known to be ``noise dressed''. We develop a new and alternative method to estimate this noise. To this end, we simulate certain time series and random matr…

The paper proposes new cross-correlators using Price's Theorem and piecewise-linear decomposition.

A new estimator for asynchronous tick data shows better correlation estimates.

Naive Bayes estimator is widely used in text classification problems. However, it doesn't perform well with small-size training dataset. We propose a new method based on Naive Bayes estimator to solve this problem. A correlation factor is introduced to incorporate the correlation among different classes. Experimental r…

We estimate generic statistical properties of a structural credit risk model by considering an ensemble of correlation matrices. This ensemble is set up by Random Matrix Theory. We demonstrate analytically that the presence of correlations severely limits the effect of diversification in a credit portfolio if the corre…

Graph Canonical Correlation Analysis improves CCA for multiomics datasets.

We study a distributed estimation problem in which two remotely located parties, Alice and Bob, observe an unlimited number of i.i.d. samples corresponding to two different parts of a random vector. Alice can send $k$ bits on average to Bob, who in turn wants to estimate the cross-correlation matrix between the two par…

Private method measures nonlinear correlations between data hosted across two entities.

The study analyzes the differences between physical and risk-neutral correlation estimates for equity baskets.

Our goal in this paper is to develop an effective estimator of fractal dimension. We survey existing ideas in dimension estimation, with a focus on the currently popular method of Grassberger and Procaccia for the estimation of correlation dimension. There are two major difficulties in estimation based on this method. …

We study the problem of finding the most mutually correlated arms among many arms. We show that adaptive arms sampling strategies can have significant advantages over the non-adaptive uniform sampling strategy. Our proposed algorithms rely on a novel correlation estimator. The use of this accurate estimator allows us t…

We discuss a weighted estimation of correlation and covariance matrices from historical financial data. To this end, we introduce a weighting scheme that accounts for similarity of previous market conditions to the present one. The resulting estimators are less biased and show lower variance than either unweighted or e…

Structural learning of directed acyclic graphs (DAGs) or Bayesian networks has been studied extensively under the assumption that data are independent. We propose a new Gaussian DAG model for dependent data which assumes the observations are correlated according to an undirected network. Under this model, we develop a …

Bayesian method for dynamic correlation matrices improves accuracy and responsiveness.

A method for estimating the cross-correlation $C_{xy}(τ)$ of long-range correlated series $x(t)$ and $y(t)$, at varying lags $τ$ and scales $n$, is proposed. For fractional Brownian motions with Hurst exponents $H_1$ and $H_2$, the asymptotic expression of $C_{xy}(τ)$ depends only on the lag $τ$ (wide-sense stationarit…

A fast method estimates correlations in hybrid systems using observable market data.

We study finite sample properties of estimators of power-law cross-correlations -- detrended cross-correlation analysis (DCCA), height cross-correlation analysis (HXA) and detrending moving-average cross-correlation analysis (DMCA) -- with a special focus on short-term memory bias as well as power-law coherency. Presen…

Sparse GCA finds linear relationships in multiple datasets, using gradient descent.

We introduce a new test for detection of power-law cross-correlations among a pair of time series - the rescaled covariance test. The test is based on a power-law divergence of the covariance of the partial sums of the long-range cross-correlated processes. Utilizing a heteroskedasticity and auto-correlation robust est…

The study uses the Merton model to estimate PD and finds a phase transition affecting convergence speed.

We quantify how co-jumps impact correlations in currency markets. To disentangle the continuous part of quadratic covariation from co-jumps, and study the influence of co-jumps on correlations, we propose a new wavelet-based estimator. The proposed estimation framework is able to localize the co-jumps very precisely th…

The probability of default (PD) estimation is an important process for financial institutions. The difficulty of the estimation depends on the correlations between borrowers. In this paper, we introduce a hierarchical Bayesian estimation method using the beta binomial distribution and consider a multi-year case with a …

Many nonlinear extensions of the Kalman filter, e.g., the extended and the unscented Kalman filter, reduce the state densities to Gaussian densities. This approximation gives sufficient results in many cases. However, this filters only estimate states that are correlated with the observation. Therefore, sequential esti…

New DP methods for estimating means and frequencies with varying privacy demands.

This work optimizes induced correlation in joint graph embeddings.

Correlation matrices are omnipresent in multivariate data analysis. When the number d of variables is large, the sample estimates of correlation matrices are typically noisy and conceal underlying dependence patterns. We consider the case when the variables can be grouped into K clusters with exchangeable dependence; t…

Using the $\ell_1$-norm to regularize the estimation of the parameter vector of a linear model leads to an unstable estimator when covariates are highly correlated. In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. This norm, ca…