Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

84168251335 · Jun 202019922001200920172026
48 results for correlation testing

The paper sets thresholds for testing correlation in hypergraphs, distinguishing between independent and correlated states.

problem Testing correlation between two hypergraphs under different models.
method Derives sharp information-theoretic thresholds for distinguishing between null and alternative hypotheses.
result The testing threshold decreases as the hypergraph's uniformity (m) increases, making correlation testing easier for higher uniformity.

The paper introduces tests for high-dimensional independence using maximum and average distance correlations.

problem Testing independence in high-dimensional data.
method Characterizes consistency properties, compares test statistics, examines null distributions, and presents a fast chi-square-based procedure.
result The proposed tests are non-parametric and applicable to various metrics.

We introduce a new test for detection of power-law cross-correlations among a pair of time series - the rescaled covariance test. The test is based on a power-law divergence of the covariance of the partial sums of the long-range cross-correlated processes. Utilizing a heteroskedasticity and auto-correlation robust est…

2013-07-17abs ↗pdf ↗

Develops a method for stress testing correlations of financial portfolios.

problem Stress testing correlations in financial asset portfolios.
method Parametric representation of correlations, Bayesian variable selection, joint distribution of stress scenarios.
result Inference of worst-case correlation scenarios using stress tests.

Distance correlation has gained much recent attention in the data science community: the sample statistic is straightforward to compute and asymptotically equals zero if and only if independence, making it an ideal choice to discover any type of dependency structure given sufficient sample size. One major bottleneck is…

2019-12-27abs ↗pdf ↗

A new test statistic counts tree co-occurrences to detect edge correlation between networks.

problem Detecting edge correlation between networks using latent vertex correspondence.
method The test statistic is based on counting co-occurrences of signed trees for a family of non-isomorphic trees.
result The test runs in n2+o(1)n^{2+o(1)} time and succeeds with high probability for large nn.

Private method measures nonlinear correlations between data hosted across two entities.

problem Measuring nonlinear correlations between sensitive data hosted across multiple parties while preserving privacy.
method Differentially private estimator of distance correlation.
result First private estimator of nonlinear correlations in a multi-party setup.

New methods test correlation between network structure and node features.

problem Assessing correlation between network structure and node-level covariates.
method Four novel methods based on linear models and canonical correlation analysis.
result Theoretical guarantees and computational efficiency for testing network dependency.

OmniMatch algorithm perfectly matches graphs without edge correlation.

problem Graph matching in the absence of edge correlation.
method OmniMatch algorithm for seeded multiple graph matching.
result OmniMatch aligns O(sα)O(s^α) unseeded vertices across multiple networks efficiently and perfectly.

The paper explores statistical limits for detecting correlation in tree structures.

problem Detecting correlation between two tree structures.
method Investigates conditions for existence of one-sided tests in the limit of large tree depth.
result Identifies a phase transition at correlation parameter s=αs = \sqrt{α}, where tests exist for s>αs > \sqrt{α}.

Paper introduces ρρ-Perfect to estimate model-human correlation in subjective datasets.

problem Inherent noise in subjective ratings limits model-human correlation quantification.
method Defines ρρ-Perfect as highest achievable correlation between perfect predictor and human ratings. Estimates based on heteroscedastic noise scenarios.
result Demonstrates ρρ-Perfect can distinguish model limitations from data quality issues.

Proposes a method to calibrate data for more accurate linear correlation testing.

problem Inaccurate Pearson's correlation coefficient due to sample size and data non-normality.
method Predictive data calibration using machine learning to condition data on expected linear relationship.
result Calibrated Pearson's correlation coefficient yields a calibrated p-value and r estimate for posterior probability interpretation.

Estimates vaccine effectiveness and immune correlates in TND studies with missing data.

problem Confounding and missing data in TND studies of vaccine effectiveness and immune correlates.
method Targeted maximum likelihood estimation using a semiparametric logistic regression model.
result Valid causal inference of vaccine effectiveness and immune correlates in TND studies with missing exposure data.

The paper develops a test for independence of selected Gaussian variables after thresholding correlations.

problem Testing independence of selected Gaussian variables after thresholding correlations.
method The approach involves conditioning on the selection event and using a new characterization of the conditioning event in terms of canonical correlation.
result The proposed test has higher power than a naive approach that ignores selection effects.

It has been shown that instead of learning actual object features, deep networks tend to exploit non-robust (spurious) discriminative features that are shared between training and test sets. Therefore, while they achieve state of the art performance on such test sets, they achieve poor generalization on out of distribu…

2019-10-01abs ↗pdf ↗

HOoD detects near-out-of-distribution groups in correlated biomedical assays.

problem Detecting near-out-of-distribution cases in biased or incomplete data.
method Projects correlated measurements through a trained model and uses permutation-based hypothesis tests.
result HOoD reliably identifies OoD groups, outperforming other detectors.

In the presence of weak overall correlation, it may be useful to investigate if the correlation is significantly and substantially more pronounced over a subpopulation. Two different testing procedures are compared. Both are based on the rankings of the values of two variables from a data set with a large number n of o…

2015-04-21abs ↗pdf ↗

Testing two potentially multivariate variables for statistical dependence on the basis finite samples is a fundamental statistical challenge. Here we explore a family of tests that adapt to the complexity of the relationship between the variables, promising robust power across scenarios. Building on the distance correl…

2018-10-06abs ↗pdf ↗

New algorithm achieves almost exact graph matching in almost quadratic time.

problem Graph matching under correlated Erdős-Rényi models.
method Rank-based graph matching using local tree correlation tests.
result Achieves almost exact recovery in almost quadratic time complexity.

ETF approval boosts Bitcoin's correlation with equities, stabilizes with gold, and maintains negative correlation with fiat currencies.

problem Impact of Bitcoin ETF approval on Bitcoin's relationships with traditional assets.
method Rolling correlation analysis, Chow tests, and DCC-GARCH models.
result Bitcoin's correlation with equities increased significantly post-ETF approval, while its relationship with gold stabilized and remained negatively correlated with fiat currencies.

The paper tackles spurious correlations in machine learning models and introduces counterfactual invariance.

problem Spurious correlations in machine learning models that depend on irrelevant parts of input data.
method The paper uses causal inference to stress test models and introduces counterfactual invariance as a formal requirement.
result Counterfactual invariance is a requirement for models to be robust to irrelevant perturbations in input data.

New method detects intrinsic cross-correlations in non-stationary time series affected by common factors.

problem Bias in cross-correlation analysis due to common external factors.
method Multifractal temporally weighted detrended partial cross-correlation analysis (MF-TWDPCCA).
result MF-TWDPCCA accurately detects intrinsic cross-correlations between non-stationary time series.

Overparameterized models can worsen minority group errors even when overall test error improves.

problem Overparameterization exacerbates spurious correlations, harming minority groups.
method Simulations and experiments on image datasets, theoretical analysis of linear models.
result Subsampling the majority group can achieve low minority error in overparameterized models.

A new method estimates conditional canonical correlations using random forests.

problem Estimating relationships between two sets of variables given covariates.
method Random Forest with Canonical Correlation Analysis (RFCCA)
result RFCCA provides accurate canonical correlation estimations and well-controlled Type-1 error.

Portfolio allocation and risk management make use of correlation matrices and heavily rely on the choice of a proper correlation matrix to be used. In this regard, one important question is related to the choice of the proper sample period to be used to estimate a stable correlation matrix. This paper addresses this qu…

2019-11-14abs ↗pdf ↗

This study examines how sequential correlations affect in-context learning in sequence models.

problem Understanding how in-context learning works with sequentially correlated data.
method Extended linear regression model to sequentially correlated data, tested on transformer architectures.
result Sequential correlations alter the effective context length and attention architecture effectiveness.

Framework uses human annotations to make models robust to spurious correlations.

problem Machine learning models fail when unmeasured variables change test distributions.
method Human annotations to augment training examples, UV-DRO objective for robustness.
result Improvements of 5-10% on digit recognition task and 1.5-5% on NYPD Police Stops analysis.

The paper introduces sanity tests to detect spurious correlations in AI-guided radiology systems.

problem Detecting when AI systems perform well on development data for the wrong reasons.
method Design and implementation of sanity tests to identify spurious correlations.
result Sanity tests can identify spurious correlations in AI-guided radiology systems.

In order to emphasize cross-correlations for fluctuations in major market places, series of up and down spins are built from financial data. Patterns frequencies are measured, and statistical tests performed. Strong cross-correlations are emphasized, proving that market moves are collective behaviors.

2000-01-20abs ↗pdf ↗

We describe a post hoc test for the Sharpe ratio, analogous to Tukey's test for pairwise equality of means. The test can be applied after rejection of the hypothesis that all population Signal-Noise ratios are equal. The test is applicable under a simple correlation structure among asset returns. Simulations indicate t…

2019-11-11abs ↗pdf ↗

The Normal Means problem plays a fundamental role in many areas of modern high-dimensional statistics, both in theory and practice. And the Empirical Bayes (EB) approach to solving this problem has been shown to be highly effective, again both in theory and practice. However, almost all EB treatments of the Normal Mean…

2018-12-18abs ↗pdf ↗

Non-symmetric rectangular correlation matrices occur in many problems in economics. We test the method of extracting statistically meaningful correlations between input and output variables of large dimensionality and build a toy model for artificially included correlations in large random time series.The results are t…

2010-04-26abs ↗pdf ↗

This paper explores the information-theoretic limitations of graph property testing in zero-field Ising models. Instead of learning the entire graph structure, sometimes testing a basic graph property such as connectivity, cycle presence or maximum clique size is a more relevant and attainable objective. Since property…

2017-09-20abs ↗pdf ↗

LCIT tests conditional independence using latent representations.

problem Detecting conditional independencies in statistical and machine learning tasks.
method Generative framework for learning latent representations of target variables X and Y, then testing for remaining dependencies.
result LCIT outperforms state-of-the-art baselines consistently under different metrics and settings.

The article compares predictor importance in classification problems with categorical outcomes.

problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.

Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms …

2017-09-12abs ↗pdf ↗

New insights into ridge regression with correlated data, improving risk prediction.

problem Understanding and predicting risk in ridge regression with correlated samples.
method Random matrix theory and free probability for asymptotic analysis; modified GCV estimator (CorrGCV) for unbiased prediction.
result GCV estimator fails for out-of-sample risk with correlated data; CorrGCV provides an unbiased estimator.

Enhanced metrics for multiclass classification improve on existing methods.

problem Lack of decisive poor classification results in existing multiclass metrics.
method Introduces three new metrics derived from multivariate Pearson correlation coefficients.
result New metrics decisively indicate poor classification results.