Statistical inference can be computationally prohibitive in ultrahigh-dimensional linear models. Correlation-based variable screening, in which one leverages marginal correlations for removal of irrelevant variables from the model prior to statistical inference, can be used to overcome this challenge. Prior works on co…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Correlation-Based Methods”
Review of correlation-based financial networks and entropy measures.
problem Understanding the dynamics of financial markets through correlation networks.
method Analysis of empirical correlation matrices and entropy measures.
result Entropy measures help in continuous monitoring of financial networks.
We investigate the trading behavior of Finnish individual investors trading the stocks selected to compute the OMXH25 index in 2003 by tracking the individual daily investment decisions. We verify that the set of investors is a highly heterogeneous system under many aspects. We introduce a correlation based method that…
CaLoNet integrates spatial and local correlations for multivariate time series classification.
problem Ignoring spatial and local correlations in multivariate time series classification.
method Model spatial correlations using causality modeling, extract local correlations, integrate into graph neural network.
result Competitive performance compared to state-of-the-art methods on UEA datasets.
We investigate the daily correlation present among market indices of stock exchanges located all over the world in the time period Jan 1996 - Jul 2009. We discover that the correlation among market indices presents both a fast and a slow dynamics. The slow dynamics reflects the development and consolidation of globaliz…
This paper studies an intelligent ultimate technique for health-monitoring and prognostic of common rotary machine components, particularly bearings. During a run-to-failure experiment, rich unsupervised features from vibration sensory data are extracted by a trained sparse auto-encoder. Then, the correlation of the ex…
Networks of equities in financial marketscond-mat.stat-mech
We review the recent approach of correlation based networks of financial equities. We investigate portfolio of stocks at different time horizons, financial indices and volatility time series and we show that meaningful economic information can be extracted from noise dressed correlation matrices. We show that the metho…
Community detection improves stock market portfolio optimization.
problem Improving portfolio optimization in financial markets.
method Community detection in correlation-based networks of worldwide stock markets.
result Portfolios constructed using community detection outperform traditional methods.
According to the leading models in modern finance, the presence of intraday lead-lag relationships between financial assets is negligible in efficient markets. With the advance of technology, however, markets have become more sophisticated. To determine whether this has resulted in an improved market efficiency, we inv…
We propose improved methods to identify stock groups using the correlation matrix of stock price changes. By filtering out the marketwide effect and the random noise, we construct the correlation matrix of stock groups in which nontrivial high correlations between stocks are found. Using the filtered correlation matrix…
CFS (Correlation-Based Feature Selection) is an FS algorithm that has been successfully applied to classification problems in many domains. We describe Distributed CFS (DiCFS) as a completely redesigned, scalable, parallel and distributed version of the CFS algorithm, capable of dealing with the large volumes of data t…
Causal methods for GRN inference from single-cell data often fail in real-world benchmarks.
problem Understanding when and why causal methods for GRN inference from single-cell data fail in real-world benchmarks.
method Introduced a controlled diagnostic framework to isolate and measure seven pathologies.
result Causal methods dominate in clean and structurally favorable regimes but fail in specific pathologies.
CHANI learns classification tasks with local transformations inspired by biology.
problem Proving neural networks can learn classification tasks with local transformations.
method CHANI uses spiking neurons modeled by Hawkes processes with expert aggregation for local learning.
result CHANI can learn and encode multiple classes, forming assemblies of neurons.
PLD distills knowledge using choice-theoretic Plackett-Luce model.
problem Model compression and knowledge transfer between large and small networks.
method PLD uses a weighted list-wise ranking loss based on the Plackett-Luce model.
result PLD achieves consistent gains across diverse architectures and distillation methods.
We discuss some methods to quantitatively investigate the properties of correlation matrices. Correlation matrices play an important role in portfolio optimization and in several other quantitative descriptions of asset price dynamics in financial markets. Specifically, we discuss how to define and obtain hierarchical …
The paper uses distance correlation for brain connectivity and a novel multi-task learning model for age prediction.
problem Estimating age-related gender differences in brain functional connectivity.
method Estimates functional connectivity using distance correlation and proposes a non-convex multi-task learning model.
result The proposed non-convex multi-task learning model outperforms other models in age prediction and gender-specific connectivity.
AlphaEval evaluates alpha mining models efficiently and comprehensively.
problem Lack of systematic evaluation for alpha mining models.
method Unified, parallelizable evaluation framework assessing predictive power, stability, robustness, financial logic, and diversity.
result AlphaEval achieves evaluation consistency comparable to comprehensive backtesting, providing more comprehensive insights and higher efficiency.
Complex systems are composed of mutually interacting components and the output values of these components are usually long-range cross-correlated. We propose a method to characterize the joint multifractal nature of such long-range cross correlations based on wavelet analysis, termed multifractal cross wavelet analysis…
Predicting missing links via correlation between nodesphysics.soc-ph
As a fundamental problem in many different fields, link prediction aims to estimate the likelihood of an existing link between two nodes based on the observed information. Since this problem is related to many applications ranging from uncovering missing data to predicting the evolution of networks, link prediction has…
Financial markets are well known examples of multi-fractal complex systems that have garnered much interest in their characterization through complex network theory. The recent studies have used correlation based distance metrics for defining and analyzing financial networks. In this work the singularity strength is em…
This paper presents a novel adaptive-filter approach for predicting assets on the stock markets. Concepts are introduced here, which allow understanding this method and computing of the corresponding forecast. This approach is applied, as an example, through the prediction over the actual valuation of the PETR3 shares …
We demonstrate that the gain/loss asymmetry observed for stock indices vanishes if the temporal dependence structure is destroyed by scrambling the time series. We also show that an artificial index constructed by a simple average of a number of individual stocks display gain/loss asymmetry - this allows us to explicit…
This paper proposes a model-free and data-adaptive feature screening method for ultra-high dimensional datasets. The proposed method is based on the projection correlation which measures the dependence between two random vectors. This projection correlation based method does not require specifying a regression model an…
Motivated by an abstract notion of low-level edge detector filters, we propose a simple method of unsupervised feature construction based on pairwise statistics of features. In the first step, we construct neighborhoods of features by regrouping features that correlate. Then we use these subsets as filters to produce n…
While sparse inverse covariance matrices are very popular for modeling network connectivity, the value of the dense solution is often overlooked. In fact the L2-regularized solution has deep connections to a number of important applications to spectral graph theory, dimensionality reduction, and uncertainty quantificat…
This paper develops a new framework to assess crypto portfolio risk using simulation methods.
problem Traditional financial risk models fail to capture crypto market characteristics like volatility and contagion.
method The framework integrates four components: volatility stress testing, hedging, contagion modeling, and Monte Carlo simulation.
result The framework robustly assesses crypto portfolio risk and is validated with real data.
New cluster validity index detects optimal number of clusters and secondary options.
problem Determining the optimal number of clusters in fuzzy clustering.
method Correlation-based fuzzy cluster validity index (WP index) using fuzzy c-means algorithm.
result WP index outperforms existing indexes in detecting optimal number of clusters and secondary options.
The major challenge of learning from multi-label data has arisen from the overwhelming size of label space which makes this problem NP-hard. This problem can be alleviated by gradually involving easy to hard tags into the learning process. Besides, the utilization of a diversity maintenance approach avoids overfitting …
Paper evaluates and improves private feature selection methods.
problem Feature selection in high-dimensional datasets with privacy constraints.
method Correlations-based order statistic privatized for feature selection.
result Our method significantly outperforms established baseline for private feature selection.
Paper extends CCA for multiview learning, improving performance.
problem Learning representations across multiple data views.
method Extends CCA to a multiview mixture model with heuristics.
result Improves performance on downstream tasks compared to standard CCA.
In the last years efforts in econophysics have been shifted to study how network theory can facilitate understanding of complex financial markets. Main part of these efforts is the study of correlation-based hierarchical networks. This is somewhat surprising as the underlying assumptions of research looking at financia…
We investigate the planar maximally filtered graphs of the portfolio of the 300 most capitalized stocks traded at the New York Stock Exchange during the time period 2001-2003. Topological properties such as the average length of shortest paths, the betweenness and the degree are computed on different planar maximally f…
Study measures uncertainty in MST identification across different correlation networks.
problem Uncertainty in MST identification across various correlation-based market networks.
method Developed a framework using random variable networks (RVN) to measure uncertainty of MST identification.
result FDR is the most appropriate measure for MST identification reliability.
We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive Agglomerative Likelihood Clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) wit…
This work explains how maximizing latent correlations across multiple data views helps in identifying shared and private components.
problem Understanding how to identify shared and private components in multiview data.
method An intuitive generative model of multiview data is adopted, and latent correlation maximization is shown to guarantee the extraction of shared components.
result Latent correlation maximization guarantees the extraction of shared components across views and disentangles private information.
We introduce deep scale-spaces (DSS), a generalization of convolutional neural networks, exploiting the scale symmetry structure of conventional image recognition tasks. Put plainly, the class of an image is invariant to the scale at which it is viewed. We construct scale equivariant cross-correlations based on a princ…
The study reveals the hierarchical structure of the international FOREX market using currency fluctuation distribution similarities.
problem Understanding the hierarchical structure of the international FOREX market.
method Using Jensen-Shannon divergence to quantify the similarity between normalized logarithmic return distributions of currencies.
result Clusters of currencies are consistent with the nature of underlying economies but diverge during crises.
Variable clustering is important for explanatory analysis. However, only few dedicated methods for variable clustering with the Gaussian graphical model have been proposed. Even more severe, small insignificant partial correlations due to noise can dramatically change the clustering result when evaluating for example w…
Proposes Causal Loss to improve machine learning models' causal inference.
problem Machine learning algorithms often fail to capture causal relationships when data is inconsistent.
method Introduces Causal Loss, a model-agnostic loss function that enhances interventional capabilities.
result Causal Loss improves non-causal associative models to have interventional capabilities.
We present a filter correlation based model compression approach for deep convolutional neural networks. Our approach iteratively identifies pairs of filters with the largest pairwise correlations and drops one of the filters from each such pair. However, instead of discarding one of the filters from each such pair naï…
An increasing number of datasets contain multiple views, such as video, sound and automatic captions. A basic challenge in representation learning is how to leverage multiple views to learn better representations. This is further complicated by the existence of a latent alignment between views, such as between speech a…
We investigate the properties of correlation based networks originating from economic complex systems, such as the network of stocks traded at the New York Stock Exchange (NYSE). The weaker links (low correlation) of the system are found to contribute to the overall connectivity of the network significantly more than t…
Graph Canonical Correlation Analysis improves CCA for multiomics datasets.
problem Limited ability of conventional CCA methods to incorporate structured patterns in cross-correlation matrices.
method Graph Canonical Correlation Analysis (gCCA) calculates canonical correlations based on the graph structure of cross-correlation matrices.
result gCCA outperforms competing CCA methods in simulations and multiomics dataset analysis.
We investigate the time series of the degree of minimum spanning trees obtained by using a correlation based clustering procedure which is starting from (i) asset return and (ii) volatility time series. The minimum spanning tree is obtained at different times by computing correlation among time series over a time windo…
A method for MRI brain tumor segmentation using feature vectors and kernel dictionary learning.
problem Segmenting brain tumor regions in MRI images.
method Extracting feature vectors, training kernel dictionaries, and selecting informative feature vectors.
result The method outperforms other methods in segmentation accuracy and reduces training time.
What does past correlation structure tell us about the future? An answer from network filteringq-fin.PM
We discovered that past changes in the market correlation structure are significantly related with future changes in the market volatility. By using correlation-based information filtering networks we device a new tool for forecasting the market volatility changes. In particular, we introduce a new measure, the "correl…
Variable selection is a challenging issue in statistical applications when the number of predictors far exceeds the number of observations . In this ultra-high dimensional setting, the sure independence screening (SIS) procedure was introduced to significantly reduce the dimensionality by preserving the true mod…
The properties of q-dependent cross-correlation matrices of stock market have been analyzed by using the random matrix theory and complex network. The correlation structures of the fluctuations at different magnitudes have unique properties. The cross-correlations among small fluctuations are much stronger than those a…