A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This study examines clustering of correlated random variables using k-means and spectral methods.
LMMVAE improves VAE for correlated data by separating latent variables into fixed and random parts.
A new method estimates conditional canonical correlations using random forests.
The paper extends Pearson correlation to multi-variables, useful for noise measurement and feature selection.
Two new methods for analyzing repeated measures data using embeddings into Reproducing Kernel Hilbert Spaces.
FREEtree improves tree-based methods for correlated longitudinal data.
LMLFM tackles predictive modeling from longitudinal data with mixed correlations.
Geometric QHD tests improve hub detection in correlated data.
Better signal detection in undersampled data using joint and cross covariances.
Random forest hyperparameters affect variable selection in omics studies.
We present a general method to detect and extract from a finite time sample statistically meaningful correlations between input and output variables of large dimensionality. Our central result is derived from the theory of free random matrices, and gives an explicit expression for the interval where singular values are…
We introduce the Randomized Dependence Coefficient (RDC), a measure of non-linear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. RDC is defined in terms of correlation of random non-linear copula projections; it is invariant with respec…
Non-symmetric rectangular correlation matrices occur in many problems in economics. We test the method of extracting statistically meaningful correlations between input and output variables of large dimensionality and build a toy model for artificially included correlations in large random time series.The results are t…
While records and order statistics of independent and identically distributed (i.i.d.) random variables X_1, ..., X_N are fully understood, much less is known for strongly correlated random variables, which is often the situation encountered in statistical physics. Recently, it was shown, in a series of works, that one…
Lognormal random variables appear naturally in many engineering disciplines, including wireless communications, reliability theory, and finance. So, too, does the sum of (correlated) lognormal random variables. Unfortunately, no closed form probability distribution exists for such a sum, and it requires approximation. …
This paper treats the problem of screening for variables with high correlations in high dimensional data in which there can be many fewer samples than variables. We focus on threshold-based correlation screening methods for three related applications: screening for variables with large correlations within a single trea…
In this paper, a class of statistics named ART (the alternant recursive topology statistics) is proposed to measure the properties of correlation between two variables. A wide range of bi-variable correlations both linear and nonlinear can be evaluated by ART efficiently and equitably even if nothing is known about the…
The paper develops a test for independence of selected Gaussian variables after thresholding correlations.
Proposes a new method to estimate variable importance in black box models, mitigating correlation effects.
The problem of using observed correlations to infer causal relations is relevant to a wide variety of scientific disciplines. Yet given correlations between just two classical variables, it is impossible to determine whether they arose from a causal influence of one on the other or a common cause influencing both, unle…
Reduces selection bias in estimating individual treatment effects.
We investigate financial market correlations using random matrix theory and principal component analysis. We use random matrix theory to demonstrate that correlation matrices of asset price changes contain structure that is incompatible with uncorrelated random price changes. We then identify the principal components o…
NURD improves model performance by distilling representations independent of nuisance variables.
We study a new ensemble of random correlation matrices related to multivariate Student (or more generally elliptic) random variables. We establish the exact density of states of empirical correlation matrices that generalizes the Marcenko-Pastur result. The comparison between the theoretical density of states in the St…
We present sharp tail asymptotics for the density and the distribution function of linear combinations of correlated log-normal random variables, that is, exponentials of components of a correlated Gaussian vector. The asymptotic behavior turns out to depend on the correlation between the components, and the explicit s…
The paper sets limits on the accuracy of macroeconomic forecasts based on statistical moments and trade volumes.
We introduce Network Maximal Correlation (NMC) as a multivariate measure of nonlinear association among random variables. NMC is defined via an optimization that infers transformations of variables by maximizing aggregate inner products between transformed variables. For finite discrete and jointly Gaussian random vari…
We show how to analyze and interpret the correlation structures, the conditional expectation values and correlation coefficients of exchangeable Bernoulli random variables. We study implied default distributions for the iTraxx-CJ tranches and some popular probabilistic models, including the Gaussian copula model, Beta …
It is ubiquitous in natural and social sciences that two variables, recorded temporally or spatially in a complex system, are cross-correlated and possess multifractal features. We propose a new method called multifractal detrended cross-correlation analysis (MF-DXA) to investigate the multifractal behaviors in the pow…
Gaussian copulas are widely used in the industry to correlate two random variables when there is no prior knowledge about the co-dependence between them. The perturbed Gaussian copula approach allows introducing the skew information of both random variables into the co-dependence structure. The analytical expression of…
Statistical inference can be computationally prohibitive in ultrahigh-dimensional linear models. Correlation-based variable screening, in which one leverages marginal correlations for removal of irrelevant variables from the model prior to statistical inference, can be used to overcome this challenge. Prior works on co…
Functional neuroimaging can measure the brain?s response to an external stimulus. It is used to perform brain mapping: identifying from these observations the brain regions involved. This problem can be cast into a linear supervised learning task where the neuroimaging data are used as predictors for the stimulus. Brai…
Simple conditions for comonotonic additive risk measures from acceptance sets.
This paper compares imputation and direct parameter estimation methods for missing data in correlation matrix visualization.
This paper presents Correlated Nystrom Views (XNV), a fast semi-supervised algorithm for regression and classification. The algorithm draws on two main ideas. First, it generates two views consisting of computationally inexpensive random features. Second, XNV applies multiview regression using Canonical Correlation Ana…
We consider the correlated multiarmed bandit (MAB) problem in which the rewards associated with each arm are modeled by a multivariate Gaussian random variable, and we investigate the influence of the assumptions in the Bayesian prior on the performance of the upper credible limit (UCL) algorithm and a new correlated U…
Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies by finding maximally correlated embeddings of pairs of random variables. CA has found applications in fields ranging from epidemiology to social sciences; however, current methods do not scale to large, hig…
In this manuscript, we analytically and numerically study statistical properties of an heteroskedastic process based on the celebrated ARCH generator of random variables whose variance is defined by a memory of -exponencial, form (). Specifically, we inspect the self-correlation function o…
We review recent advances on the record statistics of strongly correlated time series, whose entries denote the positions of a random walk or a Lévy flight on a line. After a brief survey of the theory of records for independent and identically distributed random variables, we focus on random walks. During the last few…
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased" choice corresponds to the distribution of maximum entropy. The calculation of t…
Neural networks learn faster with correlated latent variables.
Study measures uncertainty in MST identification across different correlation networks.
Inference and learning of graphical models are both well-studied problems in statistics and machine learning that have found many applications in science and engineering. However, exact inference is intractable in general graphical models, which suggests the problem of seeking the best approximation to a collection of …
Researchers identify critical protein residues using advanced graph theory.
BERET improves binary expansion test for multivariate independence.
This paper presents a new ensemble learning method for classification problems called projection pursuit random forest (PPF). PPF uses the PPtree algorithm introduced in Lee et al. (2013). In PPF, trees are constructed by splitting on linear combinations of randomly chosen variables. Projection pursuit is used to choos…
The study improves the perceptron's storage capacity by optimizing variable selection.
Inference and learning of graphical models are both well-studied problems in statistics and machine learning that have found many applications in science and engineering. However, exact inference is intractable in general graphical models, which suggests the problem of seeking the best approximation to a collection of …