Proposes FarmHazard model for hazard regression with correlated covariates.
problem Model selection challenges in high-dimensional data with correlated covariates.
method Factor-Augmented Regularized Model for Hazard Regression (FarmHazard) that learns latent factors and idiosyncratic components.
result Proves model selection and estimation consistency under mild conditions.
We analyze ridge interpolators in correlated factor regression models using RDT.
problem Performance analysis of ridge interpolators in correlated factor regression models.
method Utilizing Random Duality Theory (RDT), we obtain precise closed form characterizations of optimization problems.
result Ridge interpolators can smooth out the excess prediction risk and exhibit double-descent behavior.
Estimates linear model from noisy covariates and instruments using spectral regularization.
problem Estimating a linear model from many noisy covariates and instruments.
method Two-stage least squares with spectral regularization of canonical correlations.
result Upper and lower bounds on estimation error, proving optimality of the method with noisy data.
With the widespread engineering applications ranging from artificial intelligence and big data decision-making, originally a lot of tedious financial data processing, processing and analysis have become more and more convenient and effective. This paper aims to improve the accuracy of stock price forecasting. It improv…
Study shows past market trends reduce or increase correlations between futures contracts.
problem Estimating and managing risk in non-stationary futures markets.
method Applied Principal Regression Analysis (PRA) to quantify past market movements' effect on correlations.
result Past up or down 10-day trends reduce or increase instantaneous correlations, respectively.
The paper compares traditional regression with modern neural network methods for financial hedging and risk compression.
problem Finding optimal hedge ratios and managing portfolio risk using traditional regression methods has limitations.
method The paper introduces regularization techniques and common factor analyses using neural networks to improve upon regression methods.
result Neural network methods provide better performance in hedge ratio estimation and risk compression compared to traditional regression.
We investigate the trading behavior of a large set of single investors trading the highly liquid Nokia stock over the period 2003-2008 with the aim of determining the relative role of endogenous and exogenous factors that may affect their behavior. As endogenous factors we consider returns and volatility, whereas the e…
In high-dimensional data, structured noise caused by observed and unobserved factors affecting multiple target variables simultaneously, imposes a serious challenge for modeling, by masking the often weak signal. Therefore, (1) explaining away the structured noise in multiple-output regression is of paramount importanc…
Study uses ML and causal analysis to predict student performance factors.
problem Understanding socio-academic and economic factors affecting student performance.
method Employed machine learning techniques and causal analysis on 1,050 student profiles.
result Ridge Regression achieved robust predictions with MAE of 0.12 and MSE of 0.024.
This paper proposes a new HDP based online review rating regression model named Topic-Sentiment-Preference Regression Analysis (TSPRA). TSPRA combines topics (i.e. product aspects), word sentiment and user preference as regression factors, and is able to perform topic clustering, review rating prediction, sentiment ana…
LMLFM tackles predictive modeling from longitudinal data with mixed correlations.
problem Learning predictive models from longitudinal data with complex correlations and non-linear interactions.
method Longitudinal Multi-Level Factorization Machine (LMLFM) that selects predictive fixed and random effects.
result LMLFM outperforms state-of-the-art methods in predictive accuracy, variable selection, and scalability.
Develops a method for stress testing correlations of financial portfolios.
problem Stress testing correlations in financial asset portfolios.
method Parametric representation of correlations, Bayesian variable selection, joint distribution of stress scenarios.
result Inference of worst-case correlation scenarios using stress tests.
Bayesian models link multiview data to outcomes.
problem Inferring relationships between diverse data types and outcomes.
method Developed two factor regression models: JFR and JAFAR.
result Improved prediction of clinical outcomes from multi-omics data.
New method disentangles correlated factors without independence assumption.
problem Learning disentangled representations from correlated data.
method Hausdorff Factorized Support (HFS) criterion for disentanglement.
result HFS consistently improves disentanglement and recovery across various correlation settings.
Trend change prediction in complex systems with a large number of noisy time series is a problem with many applications for real-world phenomena, with stock markets as a notoriously difficult to predict example of such systems. We approach predictions of directional trend changes via complex lagged correlations between…
We study how language on social media is linked to diseases such as atherosclerotic heart disease (AHD), diabetes and various types of cancer. Our proposed model leverages state-of-the-art sentence embeddings, followed by a regression model and clustering, without the need of additional labelled data. It allows to pred…
This paper compares two stock factor models in China's A-share market.
problem Contradicting results in existing research on stock factor models.
method Empirical analysis using China's A-share data from 2005-2020, orthogonalizing redundant factors, and 25-group portfolio returns calculation.
result The five-factor model outperforms the three-factor model in explaining excess return rates.
Paper studies quantized LRMR with random dithering for correlated tasks.
problem Estimating coefficient matrix in quantized multivariate regression.
method Uniform quantization with random dithering, constrained and regularized Lasso estimators.
result Achieves minimax optimal rate with dithering, slightly worsens quantization effect.
We discuss the foundations of factor or regression models in the light of the self-consistency condition that the market portfolio (and more generally the risk factors) is (are) constituted of the assets whose returns it is (they are) supposed to explain. As already reported in several articles, self-consistency implie…
We investigate the tendency for financial instruments to form clusters when there are multiple factors influencing the correlation structure. Specifically, we consider a stock portfolio which contains companies from different industrial sectors, located in several different countries. Both sector membership and geograp…
The paper identifies universal features for high-dimensional data inference.
problem Identifying universal low-dimensional features from high-dimensional data for inference tasks.
method Introduces natural notions of universality and shows a local equivalence among them, using information geometry.
result Reveals the complementary roles of various data analysis techniques.
Technical trading rules and linear regressive models are often used by practitioners to find trends in financial data. However, these models are unsuited to find non-linearly separable patterns. We propose a decision tree forecasting model that has the flexibility to capture arbitrary patterns. To illustrate, we constr…
Paper defines conditions for feasible correlation matrices from factor structures.
problem Feasibility of option implied correlation matrices in non-FX markets.
method Quantitative and economic approaches to solve the nearest correlation matrix problem.
result Introduces methods to ensure feasible correlation matrices from factor structures.
Study analyzes correlation structure in two-factor Hull-White model for XVA calculations.
problem Capturing the correlation structure in two-factor Hull-White model for accurate XVA calculations.
method Combination of approximation formula and Monte-Carlo simulation to investigate correlation structure.
result Hull-White model effectively captures de-correlation of the yield curve under specific parameter conditions.
New trees-based models handle correlated data better.
problem Standard trees-based models ignore correlation structure.
method Explicitly accounts for correlation structure in splitting criterion, stopping rules, and fitted values.
result New approach superior to standard models in simulations and real data.
It is commonly believed that the correlations between stock returns increase in high volatility periods. We investigate how much of these correlations can be explained within a simple non-Gaussian one-factor description with time independent correlations. Using surrogate data with the true market return as the dominant…
New model analyzes dynamic correlations in stock returns.
problem Analyzing time-varying correlations in high-dimensional data.
method Dynamic factor correlation model with novel parametrization.
result Model accurately captures heterogeneous heavy-tailed distributions and dependent shocks.
New method detects intrinsic cross-correlations in non-stationary time series affected by common factors.
problem Bias in cross-correlation analysis due to common external factors.
method Multifractal temporally weighted detrended partial cross-correlation analysis (MF-TWDPCCA).
result MF-TWDPCCA accurately detects intrinsic cross-correlations between non-stationary time series.
We determine the number of statistically significant factors in a forecast model using a random matrices test. The applied forecast model is of the type of Reduced Rank Regression (RRR), in particular, we chose a flavor which can be seen as the Canonical Correlation Analysis (CCA). As empirical data, we use cryptocurre…
The paper analyzes Nordic stock markets' correlation structures and regime shifts.
problem Understanding and exploiting regime shifts in Nordic stock markets.
method Examined two decades of daily data for OMXS30, OMXC20, and OMXH25 universes; proposed an adaptive portfolio allocation framework.
result Documented pronounced regime dependence in rolling correlation matrices; proposed an adaptive portfolio allocation framework.
Paper addresses the disparity between sampled and mean representations in disentangled learning.
problem Disparity between sampled and mean representations in disentangled learning.
method Proposes a method to eliminate the disparity by proving and utilizing the relationship between total correlation of sampled and mean representations for multivariate normal distributions.
result Demonstrates that a factorized mean representation can have lower total correlation than the sampled representation.
Proposes FARM model combining latent factor and sparse regression.
problem Testing adequacy of latent factor and sparse regression models.
method Factor Augmented sparse linear Regression Model (FARM) with FabTest and ANOVA type tests.
result Model robustness and effectiveness validated through experiments.
SPPCSO addresses multicollinearity in high-dimensional data, improving model stability and predictive accuracy.
problem Multicollinearity in high-dimensional data leads to unstable estimation and reduced predictive accuracy.
method SPPCSO integrates principal component regression and L1 regularization to adaptively adjust shrinkage factors.
result SPPCSO achieves stable and reliable estimation in high-noise settings, distinguishing signal variables from noise.
Proposes a graph neural network for futures price prediction.
problem Challenges in high-frequency trading of futures prices.
method Heterogeneous Continual Graph Neural Network (STGNN) integrating multi-factor pricing theories.
result Outperforms other models in prediction accuracy on 49 commodity futures.
Method estimates sparse inverse covariance and partial correlation matrices efficiently.
problem Sparse high-dimensional inverse covariance and partial correlation matrix estimation.
method Two-stage estimation method using partial regression with positive semi-definiteness.
result Efficient estimation of inverse covariance and partial correlation matrices with derived non-asymptotic rates.
In a very high-dimensional vector space, two randomly-chosen vectors are almost orthogonal with high probability. Starting from this observation, we develop a statistical factor model, the random factor model, in which factors are chosen at random based on the random projection method. Randomness of factors has the con…
New tensor-based method for estimating stock correlation matrices.
problem Choosing a proper sample period for estimating correlation matrices.
method Slice-Diagonal Tensor (SDT) factorization technique.
result The new method produces a stable correlation matrix unaffected by the sample period.
Unified framework for generating data by modeling causal and correlational dependencies.
problem Modeling both causal and correlational dependencies among latent factors.
method Causal-Correlation Variational Autoencoder (C2VAE) framework.
result Improves generation quality, disentanglement, and intervention fidelity.
Improved calibration of HJM models using small volatility approximation.
problem Calibration issues in HJM models with deterministic correlations and mean reversals.
method Use of Small Volatility Approximation in calibration of Multi-Factor HJM models.
result Calibration quality is very good and independent of the number of factors.
In 2012, JPMorgan accumulated a USD~6.2 billion loss on a credit derivatives portfolio, the so-called `London Whale', partly as a consequence of de-correlations of non-perfectly correlated positions that were supposed to hedge each other. Motivated by this case, we devise a factor model for correlations that allows for…
The stock market has been known to form homogeneous stock groups with a higher correlation among different stocks according to common economic factors that influence individual stocks. We investigate the role of common economic factors in the market in the formation of stock networks, using the arbitrage pricing model …
The location-based social network, Foursquare, reflects the human activities of a city. The mobility dynamics inferred from Foursquare helps us understanding urban social events like crime In this paper, we propose a directed graph from the aggregated movement between regions using Foursquare data. We derive region ris…
Improved sample complexity for Gaussian Mixture Models using Pair Correlation Factor.
problem Understanding the sample complexity of Gaussian Mixture Models.
method Introducing Pair Correlation Factor (PCF) to measure clustering of component means and improving sample complexity bounds.
result The Pair Correlation Factor (PCF) more accurately determines the difficulty of parameter recovery in Gaussian Mixture Models.
We propose a nonparametric Bayesian factor regression model that accounts for uncertainty in the number of factors, and the relationship between factors. To accomplish this, we propose a sparse variant of the Indian Buffet Process and couple this with a hierarchical model over factors, based on Kingman's coalescent. We…
LaCIM avoids spurious correlation by modeling latent causal factors.
problem Avoiding spurious correlation in supervised learning.
method Introducing latent variables for causal prediction and optimizing over latent space.
result Improved interpretability, robustness, and prediction power on OOD scenarios.
Improved stock selection through predictive fundamentals and uncertainty estimates.
problem Selecting stocks based on future financial data to outperform traditional factor models.
method Train deep nets to forecast future fundamentals, incorporate uncertainty estimates, and adjust portfolios to manage risk.
result Simulated annualized return of 17.7% and Sharpe ratio of 0.84 for uncertainty-aware model, significantly higher than 14.0% and 0.52 for standard factor models.
GRU-PFG model extracts inter-stock correlations from stock factors using graph neural networks.
problem Limited effectiveness of models relying solely on stock factors for capturing stock correlations.
method Project stock factors into a graph and use graph neural networks to extract inter-stock correlations.
result Achieves better prediction results than models relying solely on stock factors and comparable to second category models.
Sparse models for high-dimensional linear regression and machine learning have received substantial attention over the past two decades. Model selection, or determining which features or covariates are the best explanatory variables, is critical to the interpretability of a learned model. Much of the current literature…