This paper compares imputation and direct parameter estimation methods for missing data in correlation matrix visualization.
problem Missing data challenges in estimating correlation coefficients for accurate visualization.
method Comparison of imputation and direct parameter estimation methods for handling missing data.
result Direct parameter estimation (DPER) outperforms imputation for accurate correlation matrix visualization.
We consider a binary sequence generated by thresholding a hidden continuous sequence. The hidden variables are assumed to have a compound symmetry covariance structure with a single parameter characterizing the common correlation. We study the parameter estimation problem under such one-parameter models. We demonstrate…
A new method scales CCA parameters by input to learn more correlated representations.
problem Limitation of conventional CCA models in learning highly correlated representations.
method Introduces a dynamic scaling method for training input-dependent canonical correlation models.
result Learned representations are more correlated and retrieval results are preferable.
Symmetry of neural network densities can be determined from correlation functions.
problem Determining symmetries of neural network densities without knowing the density itself.
method Symmetry-via-duality approach using invariance properties of correlation functions.
result Symmetries of neural network densities can be determined via dual computations of correlation functions.
We analyze the daily stock data of the Nasdaq Composite index in the 22-year period 1992-2013 and identify market states as clusters of correlation matrices with similar correlation structures. We investigate the stability of the correlation structure of each state by estimating the statistical fluctuations of correlat…
A new natural gradient accounts for correlated variational parameters in variational inference.
problem Traditional natural gradients fail to correct for correlations in variational inference.
method Construct a new natural gradient called the Variational Predictive Natural Gradient (VPNG).
result VPNG accounts for the relationship between model parameters and variational parameters.
CDEFs reduce model complexity and uncover time correlations.
problem Model complexity and data efficiency in probabilistic modeling.
method Builds on deep exponential families, ties weights for reduced parameters.
result CDEFs uncover time correlations with fewer parameters.
CMoS improves time series forecasting with minimal parameters.
problem Efficiently forecasting time series data with limited resources.
method CMoS directly models chunk-wise spatial correlations, using Correlation Mixing and Periodicity Injection techniques.
result CMoS outperforms state-of-the-art models with minimal parameters.
The paper reduces the complexity of financial market correlation matrices to a 2x2 matrix.
problem Reducing the complexity of financial market correlation matrices for easier analysis.
method Sectorial coarse graining followed by averaging over blocks of stocks.
result Averaging over blocks of stocks results in a reduced matrix with specific properties.
Proposes PSCCA for estimating correlations and canonical correlations in sparse count data.
problem Estimating correlations and canonical correlations in sparse count data from next-generation sequencing.
method Probabilistic approach for sparse count data sets (PSCCA).
result PSCCA outperforms other methods in estimating true correlations and canonical correlations at the natural parameter level.
This work improves texture segmentation by automatically tuning hyperparameters for Total-Variation.
problem The challenge is to automatically select hyperparameters for Total-Variation texture segmentation.
method The approach involves extending Stein's unbiased gradient estimator to handle correlated Gaussian noise, leading to an automatic tuning method.
result The method provides an automatic way to select hyperparameters for Total-Variation texture segmentation.
Proposes a new method to estimate variable importance in black box models, mitigating correlation effects.
problem Correlation between covariates affects the interpretation of variable importance parameters.
method Develops a modified LOCO (Leave Out COvariates) method and uses semiparametric models for estimation.
result Shows how to estimate a modified LOCO method that mitigates correlation effects.
The study uses the Merton model to estimate PD and finds a phase transition affecting convergence speed.
problem Estimating the probability of default (PD) using limited historical data.
method Adopted the Merton model and analyzed phase transitions in default correlation.
result PD estimation converges slowly when temporal correlation decays by power law less than one.
Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of …
Many nonlinear extensions of the Kalman filter, e.g., the extended and the unscented Kalman filter, reduce the state densities to Gaussian densities. This approximation gives sufficient results in many cases. However, this filters only estimate states that are correlated with the observation. Therefore, sequential esti…
Improved sample complexity for Gaussian Mixture Models using Pair Correlation Factor.
problem Understanding the sample complexity of Gaussian Mixture Models.
method Introducing Pair Correlation Factor (PCF) to measure clustering of component means and improving sample complexity bounds.
result The Pair Correlation Factor (PCF) more accurately determines the difficulty of parameter recovery in Gaussian Mixture Models.
Modeling correlated mutations in cancer for personalized treatment.
problem Identifying mutations for personalized cancer therapy in heterogeneous profiles.
method Proposed correlated zero-inflated negative binomial process with mixed beta-Bernoulli and variational inference.
result Identified biologically relevant correlations between somatic mutations.
Proposes meTS for efficient exploration in correlated bandits.
problem Efficiently explore in bandits with correlated actions.
method Mixed-effect model with Thompson Sampling.
result Bound on Bayes regret with two terms.
CATS adapts multivariate time series models by addressing correlation shift.
problem Correlation differences across domains in multivariate time series data.
method CATS introduces correlation shift to measure domain differences, and uses a graph attention module and temporal convolution to align target correlations with source correlations.
result CATS increases over 10% average accuracy compared to vanilla Transformer-based models with minimal additional parameters.
Weak correlations explain linear dynamics in deep learning models.
problem Understanding the linear structure in gradient-based learning algorithms.
method Characterization of weak correlations between derivatives and parameters.
result Weak correlations are the underlying principle for linearization in deep learning models.
The risk of a credit portfolio depends crucially on correlations between the probability of default (PD) in different economic sectors. Often, PD correlations have to be estimated from relatively short time series of default rates, and the resulting estimation error hinders the detection of a signal. We present statist…
Detecting correlated trees helps align sparse graphs.
problem Detecting correlation between trees for sparse random graphs.
method MPAlign message-passing algorithm for graph alignment.
result MPAlign succeeds in polynomial time for partial alignment.
Temporal aggregation reveals latent default correlation from monthly data.
problem Understanding effective default correlation from monthly default data.
method Temporal coarse-graining of latent default-probability paths.
result Temporal coarse-graining improves identifiability and reduces over-allocation of long-horizon fluctuations.
The statistical dependencies which independent component analysis (ICA) cannot remove often provide rich information beyond the linear independent components. It would thus be very useful to estimate the dependency structure from data. While such models have been proposed, they usually concentrated on higher-order corr…
Efficient algorithm for graph matching in correlated stochastic block models.
problem Graph matching in correlated stochastic block models with balanced communities.
method Extends previous work on centered subgraph counts to handle estimation errors and edge correlation.
result First efficient algorithm for graph matching in the logarithmic average degree regime, matching all but a vanishing fraction of vertices with high probability.
In the paper, we introduce a new measure of correlation between possibly non-stationary series. As the measure is based on the detrending moving-average cross-correlation analysis (DMCA), we label it as the DMCA coefficient ρDMCA(λ) with a moving average window length λ. We analytically show that the coefficient…
Temporal coarse-graining of latent default paths explains effective correlation in corporate defaults.
problem Understanding effective default correlation in corporate defaults.
method Temporal coarse-graining of latent default-probability paths, applied to corporate default-count data.
result Temporal coarse-graining provides a scale-consistent baseline that improves identifiability and reduces over-allocation of long-horizon fluctuations.
In this short report, we investigate the ability of the DCCA coefficient to measure correlation level between non-stationary series. Based on a wide Monte Carlo simulation study, we show that the DCCA coefficient can estimate the correlation coefficient accurately regardless the strength of non-stationarity (measured b…
Methods to find correlation among variables are of interest to many disciplines, including statistics, machine learning, (big) data mining and neurosciences. Parameters that measure correlation between two variables are of limited utility when used with multiple variables. In this work, I propose a simple criterion to …
A method for estimating the cross-correlation Cxy(τ) of long-range correlated series x(t) and y(t), at varying lags τ and scales n, is proposed. For fractional Brownian motions with Hurst exponents H1 and H2, the asymptotic expression of Cxy(τ) depends only on the lag τ (wide-sense stationarit…
Develops a bi-variate stochastic framework to model mortality and interest rates with long-range dependence.
problem Captures long-range dependence and instantaneous correlation in mortality and interest rates.
method Mixed fractional Brownian motions, analytical solutions, risk-neutral measure, sequential parameter estimation.
result Explicit pricing of zero-coupon bonds and extreme mortality bonds, practical implications for pricing and risk management.
We study finite sample properties of estimators of power-law cross-correlations -- detrended cross-correlation analysis (DCCA), height cross-correlation analysis (HXA) and detrending moving-average cross-correlation analysis (DMCA) -- with a special focus on short-term memory bias as well as power-law coherency. Presen…
We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression. Individual canonical correlation trees are binary decision trees with hyperplane splits based on local canonical correlation coefficients calculated during training. Unlike axis-aligned alternatives…
We introduce a new test for detection of power-law cross-correlations among a pair of time series - the rescaled covariance test. The test is based on a power-law divergence of the covariance of the partial sums of the long-range cross-correlated processes. Utilizing a heteroskedasticity and auto-correlation robust est…
The study uses statistical methods to analyze nuclear mass models.
problem Understanding the information content of nuclear masses from models.
method Bayesian calibration, Bayesian model averaging, chi-square correlation analysis, principal component analysis.
result A dramatic parameter reduction can be achieved in both 4-parameter and 14-parameter models.
Two methods are proposed to filter correlations in DCC-GARCH residuals for foreign exchange rates.
problem Filtering correlations in DCC-GARCH residuals for accurate foreign exchange rate prediction.
method Two approaches: estimating correlation matrix as a parameter and using eigenvalue decomposition.
result The DCC-GARCH residual can be almost independent using these methods.
SGE-Kriging reduces high-dimensional surrogate modelling costs.
problem High-dimensional function approximation for expensive models.
method Splitting training data into slices, using sliced likelihood function, and learning hyper-parameters from sensitivity indices.
result SGE-Kriging achieves comparable accuracy to standard GE-Kriging but with lower training costs.
Novel trading strategy for generalized lattice markets ensures positive profits.
problem Trading in markets with serially correlated returns and asset correlation.
method Multi-double linear policies in a generalized lattice market model.
result Proposed policies ensure positive expected profits in a lattice market.
Neighbor Mixture Model captures node correlations in graphs.
problem Modeling correlations between node labels in graphs.
method Neighbor Mixture Model (NMM) designed for efficient computation and scalability.
result NMM outperforms state-of-the-art models in various graph tasks.
Dynamics of the major USA market indices DJIA, S&P, Nasdaq, and NYSE is analyzed from the point of view of the random walking problem with two-step correlations of the market moves. The parameters characterizing the stochastic dynamics are determined empirically from the historical quotes for the daily, weekly, and mon…
The study analyzes XRP transaction networks to understand market dynamics.
problem Understanding market dynamics of XRP through transaction data.
method Weekly weighted directed networks are embedded into a vector space using network embedding techniques. A correlation tensor is calculated and analyzed using singular value decomposition.
result The correlation tensor provides insights into the system's behavior and dependence on model parameters.
Study on martingale property and moment explosions in signature volatility models.
problem Analyzing the martingale property and moment explosions in signature volatility models.
method Fine analysis of the explosion time of a signature stochastic differential equation.
result The price process is a true martingale if and only if the order of the linear form is odd and a correlation parameter is negative.
Graph neural networks often assume vertex labels are independent, but we show this is rarely true and propose a method to improve predictions.
problem Graph neural networks often assume vertex labels are conditionally independent given their neighborhood features, which is rarely true.
method We model the joint distribution of residuals on vertices with a parameterized multivariate Gaussian and estimate parameters by maximizing the marginal likelihood of the observed labels.
result Our method achieves substantially higher accuracy than competing baselines and can be interpreted as the strength of correlation among connected vertices.
DPERC efficiently estimates covariance matrices for mixed data with missing values.
problem Estimating covariance matrices for datasets with missing values and mixed features.
method Direct Parameter Estimation for Randomly Missing Data with Categorical Features (DPERC).
result DPERC outperforms other methods in estimating covariance matrices for mixed data with missing values.
We show that financial correlations exhibit a non-trivial dynamic behavior. We introduce a simple phenomenological model of a multi-asset financial market, which takes into account the impact of portfolio investment on price dynamics. This captures the fact that correlations determine the optimal portfolio but are affe…
Correlation mixtures of elliptical copulas arise when the correlation parameter is driven itself by a latent random process. For such copulas, both penultimate and asymptotic tail dependence are much larger than for ordinary elliptical copulas with the same unconditional correlation. Furthermore, for Gaussian and Stude…
Transformer predicts Ethereum prices using cross-currency correlation and sentiment analysis.
problem Predicting Ethereum cryptocurrency prices with limited data.
method Transformer-based neural network with cross-currency correlation and sentiment analysis.
result Transformer model outperforms other models on some parameters.
We analyze correlations among stock returns via a series of widely adopted parameters which we refer to as explanatory variables. We subsequently exploit the results to propose a long only quantitative adaptive technique to construct a profitable portfolio of assets which exhibits minor drawdowns and higher recoveries …