The generalized correlation approach, which has been successfully used in statistical radio physics to describe non-Gaussian random processes, is proposed to describe stochastic financial processes. The generalized correlation approach has been used to describe a non-Gaussian random walk with independent, identically d…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “high correlation”
A detailed analysis of correlation between stock returns at high frequency is compared with simple models of random walks. We focus in particular on the dependence of correlations on time scales - the so-called Epps effect. This provides a characterization of stochastic models of stock price returns which is appropriat…
CCP clusters correlated features and projects them to 1D for efficient dimensionality reduction.
problem Efficiency in handling large datasets with high intrinsic dimensions.
method CCP partitions features into correlated clusters and projects them to 1D based on sample correlations.
result CCP achieves efficient dimensionality reduction without matrix diagonalization.
The study finds significant power-law cross correlations in Bitcoin's return-volatility dynamics.
problem Investigating asymmetry in Bitcoin's return-volatility relationships.
method Analysis of daily and high-frequency Bitcoin data to identify cross correlations.
result Power-law cross correlations between returns and future volatilities are observed, indicating long-range dependencies.
Simple model finds high correlation in retail crypto returns.
problem Discerning correlation in retail cryptocurrency markets without factors.
method Used N*(N) statistic to compare models of daily returns.
result High average pairwise correlation (60%) found, supports isotropic model.
This paper treats the problem of screening for variables with high correlations in high dimensional data in which there can be many fewer samples than variables. We focus on threshold-based correlation screening methods for three related applications: screening for variables with large correlations within a single trea…
This paper improves multi-label classification by leveraging high-order label correlations.
problem Improving accuracy in multi-label classification tasks using label correlations.
method Exploiting high-order label correlations through a supervised learning classifier system (UCS) and label powerset (LP) strategy.
result The proposed method outperforms other LP-based methods on multiple benchmark datasets.
New model analyzes dynamic correlations in stock returns.
problem Analyzing time-varying correlations in high-dimensional data.
method Dynamic factor correlation model with novel parametrization.
result Model accurately captures heterogeneous heavy-tailed distributions and dependent shocks.
SPPCSO addresses multicollinearity in high-dimensional data, improving model stability and predictive accuracy.
problem Multicollinearity in high-dimensional data leads to unstable estimation and reduced predictive accuracy.
method SPPCSO integrates principal component regression and L1 regularization to adaptively adjust shrinkage factors.
result SPPCSO achieves stable and reliable estimation in high-noise settings, distinguishing signal variables from noise.
Model explains Netflix Prize success with structured correlation.
problem Understanding structured correlation in high-dimensional data.
method Developed a new statistical learning model.
result Characterized learnability in terms of VCN-dimension.
Robust Estimation of Realized Correlation: New Insight about Intraday Fluctuations in Market Betasecon.EM
New estimator reveals intraday betas mainly driven by correlations.
problem Intraday fluctuations in market betas due to time-varying volatility.
method Proposes a novel subsampled quadrant estimator for high-frequency financial data.
result Intraday variation in betas primarily driven by intraday variation in correlations.
DTCCA learns nonlinear transformations of multi-view data for high-order correlation.
problem Learning complex nonlinear transformations of multiple data views.
method Maximizes high-order canonical correlation by jointly learning transformations of each view using a reformulated tensor decomposition.
result DTCCA efficiently handles high-dimensional and large number of views, overcoming scalability issues.
In this paper we use wavelet concepts to show that correlation coefficient between two financial data's is not constant but varies with scale from high correlation value to strongly anti-correlation value This studies is important because correlation coefficient is used to quantify degree of independence between two va…
Canonical Correlation Analysis (CCA) is a classical tool for finding correlations among the components of two random vectors. In recent years, CCA has been widely applied to the analysis of genomic data, where it is common for researchers to perform multiple assays on a single set of patient samples. Recent work has pr…
Extreme Value Analysis for Finite, Multivariate and Correlated Systems with Finance as an Exampleq-fin.ST
We develop a framework for analyzing extreme values in correlated financial data.
problem Quantifying and mitigating risk in complex financial systems.
method Developed a practical framework for handling finite, multivariate, and correlated time series in finance.
result We successfully analyze high-frequency stock returns using univariate extreme value tools.
We introduce a method to predict which correlation matrix coefficients are likely to change their signs in the future in the high-dimensional regime, i.e. when the number of features is larger than the number of samples per feature. The stability of correlation signs, two-by-two relationships, is found to depend on thr…
Proposes FarmHazard model for hazard regression with correlated covariates.
problem Model selection challenges in high-dimensional data with correlated covariates.
method Factor-Augmented Regularized Model for Hazard Regression (FarmHazard) that learns latent factors and idiosyncratic components.
result Proves model selection and estimation consistency under mild conditions.
Spatially relaxed inference tackles high-dimensional linear models with correlated covariates.
problem Accurate inference is challenging in high-dimensional settings with spatially correlated covariates.
method Proposes ensembled clustered inference algorithms that control the -FWER under standard assumptions.
result Ensembled clustered inference algorithms control the -FWER and achieve decent power.
Proposes a method to select features for deep learning in noisy, high-dimensional data.
problem Feature selection for deep learning in ultra-high dimensional and highly correlated data.
method Data-adaptive multi-resolutional screening and cleaning with deep learning.
result Achieves high power while keeping false discovery rate low.
New methods test correlation between network structure and node features.
problem Assessing correlation between network structure and node-level covariates.
method Four novel methods based on linear models and canonical correlation analysis.
result Theoretical guarantees and computational efficiency for testing network dependency.
The study identifies spurious correlations in high-dimensional regression and quantifies their impact.
problem Spurious correlations in high-dimensional regression models.
method Statistical characterization of spurious correlations, quantifying their amount via ridge regularization.
result The value of regularization strength that minimizes test loss is in an interval where spurious correlations increase.
We describe a new optimization scheme for finding high-quality correlation clusterings in planar graphs that uses weighted perfect matching as a subroutine. Our method provides lower-bounds on the energy of the optimal correlation clustering that are typically fast to compute and tight in practice. We demonstrate our a…
This paper investigates the utilization of maximum and average distance correlations for multivariate independence testing. We characterize their consistency properties in high-dimensional settings with respect to the number of marginally dependent dimensions, compare the advantages of each test statistic, examine thei…
A new method for real-time CCA on streaming data.
problem Finding correlated features in online data streams.
method Sliding Window Informative Canonical Correlation Analysis (SWICCA) using streaming PCA.
result SWICCA provides real-time CCA components in high dimensions with theoretical guarantees.
On the overlaps between eigenvectors of correlated random matricescond-mat.stat-mech
We obtain general, exact formulas for the overlaps between the eigenvectors of large correlated random matrices, with additive or multiplicative noise. These results have potential applications in many different contexts, from quantum thermalisation to high dimensional statistics. We find that the overlaps only depend …
New framework selects high-quality pretraining data without training LLMs.
problem Slow progress in understanding pretraining data due to costly experiments.
method Statistical framework based on perplexity-benchmark correlations.
result Approach outperforms existing methods on multiple benchmarks.
New insights into ridge regression with correlated data, improving risk prediction.
problem Understanding and predicting risk in ridge regression with correlated samples.
method Random matrix theory and free probability for asymptotic analysis; modified GCV estimator (CorrGCV) for unbiased prediction.
result GCV estimator fails for out-of-sample risk with correlated data; CorrGCV provides an unbiased estimator.
SGE-Kriging reduces high-dimensional surrogate modelling costs.
problem High-dimensional function approximation for expensive models.
method Splitting training data into slices, using sliced likelihood function, and learning hyper-parameters from sensitivity indices.
result SGE-Kriging achieves comparable accuracy to standard GE-Kriging but with lower training costs.
In a very high-dimensional vector space, two randomly-chosen vectors are almost orthogonal with high probability. Starting from this observation, we develop a statistical factor model, the random factor model, in which factors are chosen at random based on the random projection method. Randomness of factors has the con…
Predictive Correlation Screening: Application to Two-stage Predictor Design in High Dimensionstat.ML
We introduce a new approach to variable selection, called Predictive Correlation Screening, for predictor design. Predictive Correlation Screening (PCS) implements false positive control on the selected variables, is well suited to small sample sizes, and is scalable to high dimensions. We establish asymptotic bounds f…
Canonical correlation analysis was proposed by Hotelling [6] and it measures linear relationship between two multidimensional variables. In high dimensional setting, the classical canonical correlation analysis breaks down. We propose a sparse canonical correlation analysis by adding l1 constraints on the canonical vec…
We introduce a method to learn a hierarchy of successively more abstract representations of complex data based on optimizing an information-theoretic objective. Intuitively, the optimization searches for a set of latent factors that best explain the correlations in the data as measured by multivariate mutual informatio…
Cluster GARCHecon.EM
Cluster GARCH model improves multivariate GARCH for high-dimensional asset returns.
problem Modeling high-dimensional asset returns with flexible tail dependencies and cluster structures.
method Introduced a novel multivariate GARCH model with flexible convolution-t distributions, tractable likelihood and derivatives for dynamic correlation structure.
result Cluster GARCH model outperforms existing models in daily returns of 100 assets, both in-sample and out-of-sample.
We propose improved methods to identify stock groups using the correlation matrix of stock price changes. By filtering out the marketwide effect and the random noise, we construct the correlation matrix of stock groups in which nontrivial high correlations between stocks are found. Using the filtered correlation matrix…
Study finds multifractal cross-correlations between agricultural markets and external uncertainties.
problem Investigating relationships between agricultural spot markets and external uncertainties.
method Multifractal detrending moving-average cross-correlation analysis (MF-X-DMA).
result Maize exhibits intrinsic joint multifractality with all uncertainty proxies.
The Hessian Screening Rulestat.ML
A new screening rule improves lasso solving speed.
problem Efficiently solving lasso problems with high correlation.
method Uses second-order information from the Hessian to screen predictors.
result Outperforms alternatives on simulated and real data.
Paper evaluates and improves private feature selection methods.
problem Feature selection in high-dimensional datasets with privacy constraints.
method Correlations-based order statistic privatized for feature selection.
result Our method significantly outperforms established baseline for private feature selection.
New hierarchical model improves on standard practice for high-dimensional data.
problem Poor statistical performance in high-dimensional hierarchical models.
method Model effects as exchangeable across covariates and correlated across datasets.
result Empirical Bayes estimator outperforms classic approach in high-dimensional settings.
Correlation structure of extreme stock returnscond-mat.dis-nn
It is commonly believed that the correlations between stock returns increase in high volatility periods. We investigate how much of these correlations can be explained within a simple non-Gaussian one-factor description with time independent correlations. Using surrogate data with the true market return as the dominant…
We propose a correlated stochastic process of which the novel non-Gaussian probability mass function is constructed by exactly solving moment generating function. The calculation of cumulants and auto-correlation shows that the process is convergent and scale invariant in the large but finite number limit. We demonstra…
Sparse GCA finds linear relationships in multiple datasets, using gradient descent.
problem Finding linear relationships across multiple datasets with sparse loading vectors.
method Formulated as generalized eigenvalue problems, used a thresholded gradient descent algorithm.
result Proposed algorithm yields tight estimation error bounds and demonstrates effectiveness on synthetic datasets.
Zooming into market statesq-fin.ST
We analyze the daily stock data of the Nasdaq Composite index in the 22-year period 1992-2013 and identify market states as clusters of correlation matrices with similar correlation structures. We investigate the stability of the correlation structure of each state by estimating the statistical fluctuations of correlat…
Improved CEM for fast real-time planning in high-dimensional control tasks.
problem Sampling inefficiency of CEM in real-time planning.
method Novel additions to CEM including temporally-correlated actions and memory.
result 2.7-22x less samples and 1.2-10x performance increase.
We derive high-order compact finite difference schemes for option pricing in stochastic volatility models on non-uniform grids. The schemes are fourth-order accurate in space and second-order accurate in time for vanishing correlation. In our numerical study we obtain high-order numerical convergence also for non-zero …
Neurons in the visual cortex are correlated in their variability. The presence of correlation impacts cortical processing because noise cannot be averaged out over many neurons. In an effort to understand the functional purpose of correlated variability, we implement and evaluate correlated noise models in deep convolu…
A new screening method for high-dimensional data reduces computational cost.
problem Challenges in variable selection for ultrahigh-dimensional linear regression.
method Ordering absolute sample ridge partial correlations to screen variables.
result The method provides sure screening property without strong assumptions.
Algorithm recovers permutations of high-dimensional Gaussian vectors with constant correlation.
problem Recovering permutations of high-dimensional Gaussian vectors with constant correlation.
method Computing and comparing weighted counts of specially chosen wide trees.
result Polynomial-time algorithm for exact recovery at constant correlation.
It has been shown that instead of learning actual object features, deep networks tend to exploit non-robust (spurious) discriminative features that are shared between training and test sets. Therefore, while they achieve state of the art performance on such test sets, they achieve poor generalization on out of distribu…