Improves test set performance and reduces out-of-sample disappointment for unstable models.
problem Ensuring strong test set performance via cross-validation for unstable models.
method Nested k-fold cross-validation with hyperparameter selection based on a weighted sum of cross-validation metric and model stability measure.
result Improves out-of-sample MSE for sparse ridge regression and CART by 4% and 2% respectively, compared to k-fold cross-validation.
The paper uses machine learning to forecast macroeconomic outcomes with high-dimensional data.
problem Forecasting the full conditional distribution of macroeconomic outcomes.
method Systematically integrating three key principles: high-dimensional data with regularization, rigorous out-of-sample validation, and incorporating nonlinearities.
result Regularization via shrinkage is essential to control model complexity, while nonlinearities yield limited improvements in predictive accuracy.
The paper analyzes LOCV for high-dimensional risk estimation, proving error bounds.
problem Estimating out-of-sample prediction error in high-dimensional settings.
method Theoretical analysis of leave-one-out cross validation (LOCV) in penalized regression.
result Finite sample upper bounds on LOCV error, showing it converges to zero as n,p → ∞.
We introduce an exploratory study on Mutation Validation (MV), a model validation method using mutated training labels for supervised learning. MV mutates training data labels, retrains the model against the mutated data, then uses the metamorphic relation that captures the consequent training performance changes to as…
A new framework for time series forecasting that adapts to varying patterns.
problem Forecasting multivariate time series with predictive heterogeneity.
method Validation-driven clustering framework that applies specialization based on out-of-sample predictive performance.
result Improves robustness to heavy-tailed errors and local anomalies.
RandALO speeds up risk estimation for large datasets.
problem Estimating out-of-sample risk for large, high-dimensional models.
method RandALO: a randomized approximate leave-one-out estimator.
result RandALO is a computationally efficient risk estimator in high dimensions.
Validates policies using past observational data with guarantees about out-of-sample performance.
problem Evaluating decision policies using past data observed under a different policy.
method Sample-splitting method to draw inferences about the entire loss distribution with finite-sample coverage guarantees.
result Valid inferences about out-of-sample loss with finite-sample coverage guarantees, accounting for model misspecifications.
Study improves prediction accuracy and uncertainty for mobile sensor data using randomized neural networks.
problem Improving prediction accuracy and uncertainty for mobile sensor data.
method Cross-validation and uncertainty determination for randomized neural networks.
result Improved out-of-sample performance and confidence intervals for prediction error.
Paper develops a method to predict spatial point processes with guarantees.
problem Predicting the number of events in space with uncertainty.
method Regularized method to learn spatial models with out-of-sample guarantees.
result Method provides valid prediction intervals even when model is misspecified.
New method for estimating out-of-sample R² from gene expression data.
problem Lack of a well-defined and unbiased estimator for out-of-sample R².
method Explicitly defined out-of-sample R², provided an unbiased estimator, and calculated standard error.
result Demonstrated improved model comparison for gene expression phenotypes.
The study evaluates financial risk using copulas and statistical tests.
problem Validating bivariate forecasts in risk evaluation.
method Using copulas to characterize dependencies, applying statistical tests to validate forecasts, removing heteroskedasticity.
result A Student copula accurately describes financial time series dependencies.
The global minimum-variance portfolio is a typical choice for investors because of its simplicity and broad applicability. Although it requires only one input, namely the covariance matrix of asset returns, estimating the optimal solution remains a challenge. In the presence of high-dimensionality in the data, the samp…
Paper develops an AI-driven framework for systematic investing.
problem Manual prompts limit model adaptability and data snooping biases.
method Closed-loop system with self-evolving AI, out-of-sample validation, and economic rationale.
result Long-short portfolios on factor signals outperform with Sharpe ratio 3.11 and return 59.53%.
Study improves understanding of non-differentiable penalties in high-dimensional settings.
problem Theoretical understanding of non-differentiable penalties like generalized LASSO and nuclear norm in high-dimensional settings.
method Proportional high-dimensional regime analysis with finite sample upper bounds on expected squared error.
result LO provides accurate estimation of out-of-sample risk in high-dimensional settings.
We identify and validate a model for PCR in high dimensions, improving prediction guarantees.
problem Model identification and out-of-sample prediction in high-dimensional error-in-variables settings.
method Analysis of principal component regression (PCR) in fixed design settings, introducing a linear algebraic condition.
result Consistent model identification and improved out-of-sample prediction guarantees.
Random feature mapping (RFM) is a popular method for speeding up kernel methods at the cost of losing a little accuracy. We study kernel ridge regression with random feature mapping (RFM-KRR) and establish novel out-of-sample error upper and lower bounds. While out-of-sample bounds for RFM-KRR have been established by …
The paper improves ALO for ℓ1-regularized models.
problem Estimating out-of-sample error for ℓ1-regularized models. method Developed a novel theory for ℓ1-regularized problems, bounding ALO error. result For ℓ1-regularized problems, ALO error goes to zero as p goes to infinity. Study identifies key ESG variables for assessing financial risk.
problem Assessing financial risk from ESG data with many variables.
method Proposed framework for hierarchical ESG data, selecting relevant variables.
result Selected ESG variables are more relevant to financial risk than aggregated scores.
We propose a Genetic Programming architecture for the generation of foreign exchange trading strategies. The system's principal features are the evolution of free-form strategies which do not rely on any prior models and the utilization of price series from multiple instruments as input data. This latter feature consti…
Proposes a new model to maximize out-of-sample Sharpe ratios by forecasting tangency portfolios.
problem Maximizing Sharpe ratios when returns and covariances are not stationary.
method Forecast the tangency portfolio using vector autoregressions and invest in the minimum Euclidean distance portfolio.
result Empirically validated superior out-of-sample Sharpe ratios.
New insights into ridge regression with correlated data, improving risk prediction.
problem Understanding and predicting risk in ridge regression with correlated samples.
method Random matrix theory and free probability for asymptotic analysis; modified GCV estimator (CorrGCV) for unbiased prediction.
result GCV estimator fails for out-of-sample risk with correlated data; CorrGCV provides an unbiased estimator.
Performance estimation aims at estimating the loss that a predictive model will incur on unseen data. These procedures are part of the pipeline in every machine learning project and are used for assessing the overall generalisation ability of predictive models. In this paper we address the application of these methods …
Let X=X∪Z be a data set in RD, where X is the training set and Z is the test one. Many unsupervised learning algorithms based on kernel methods have been developed to provide dimensionality reduction (DR) embedding for a given training set $Φ: \mathbf{X} \to \mat…
New method improves feature selection in tree-based models.
problem Previous feature selection methods in tree-based models lack sufficient regularization and sub-optimal performance.
method Developed a new gain penalization approach for tree-based models that allows for flexible feature-specific importance weights.
result The new method improves out-of-sample performance, especially with correlated features.
RGRR allocates between QQQ and DIA based on relative states, improving Sharpe and CAGR.
problem Optimizing ETF allocation between QQQ and DIA for better risk-adjusted returns.
method Screened relative and macro states, globally screened interactions, fixed position mapping, walk-forward validation.
result RGRR improves Sharpe and CAGR compared to 100% QQQ and 50/50 QQQ-DIA allocations.
Ridge regression CV loss may have multiple local optima.
problem Can we globally optimize cross-validation loss in ridge regression?
method Analyzing quasiconvexity of CV loss in ridge regression.
result CV loss may fail to be quasiconvex and have multiple local optima.
Estimates error for robust M-estimators with convex penalties.
problem Estimating out-of-sample error for robust M-estimators in high-dimensional linear regression.
method Proposes a generic out-of-sample error estimate for robust M-estimators with convex penalties, using observed data and derivatives. result The out-of-sample error estimate has a relative error of order n−1/2 under certain conditions. High-performing equity factor with Sharpe ratio above 13 out-of-sample.
problem Hidden cross-sectional predictability in stock returns.
method Regime-conditional signal activation combining value and short-term reversal signals.
result Annualized returns of 158.6% with 12.0% volatility, strong performance out-of-sample.
We study model evaluation and model selection from the perspective of generalization ability (GA): the ability of a model to predict outcomes in new samples from the same population. We believe that GA is one way formally to address concerns about the external validity of a model. The GA of a model estimated on a sampl…
GT-Score reduces overfitting in trading strategies by integrating multiple criteria.
problem Overfitting in data-driven financial models leads to unreliable out-of-sample performance.
method Integrates performance, statistical significance, consistency, and downside risk into a composite objective function.
result Improves generalization ratio by 98% compared to baseline objective functions in walk-forward validation.
An ensemble method enhances cryptocurrency trading strategies using deep reinforcement learning.
problem Improving generalization performance in stochastic cryptocurrency trading environments.
method Model selection and mixture distribution policy to ensemble deep reinforcement learning models.
result Improved out-of-sample performance compared to benchmarks.
ASRI index detects crypto market risks with high precision and lead time.
problem Detecting systemic risks in cryptocurrency markets.
method Four weighted sub-indices (Stablecoin, DeFi, Contagion, Regulatory) validated against historical crises.
result ASRI detects significant abnormal signals with high statistical significance and lead time.
Develops a Bayesian framework for portfolio choice with a new posterior distribution.
problem Estimation risk in parametric portfolio policies.
method Generalized Bayesian framework with Gibbs posterior, utility maximization, and KNEEDLE algorithm.
result Optimal scaling parameter λ controls the balance between prior and data. We study the consistency of sample mean-variance portfolios of arbitrarily high dimension that are based on Bayesian or shrinkage estimation of the input parameters as well as weighted sampling. In an asymptotic setting where the number of assets remains comparable in magnitude to the sample size, we provide a characte…
Regime switching volatility models provide a tractable method of modelling stochastic volatility. Currently the most popular method of regime switching calibration is the Hamilton filter. We propose using the Baum-Welch algorithm, an established technique from Engineering, to calibrate regime switching models instead. …
Paper analyzes high-dimensional portfolio risks and finds empirical out-of-sample relative loss is more reliable.
problem Analyzing risks in high-dimensional portfolios using empirical variance.
method Derives asymptotic behavior of out-of-sample variance and relative loss in high-dimensional settings.
result Empirical out-of-sample relative loss is more reliable than variance in high-dimensional portfolios.
New method for cross-validation in high-dimensional data with dependent or heavy-tailed covariates.
problem Inconsistent cross-validation in high-dimensional settings with dependent or heavy-tailed covariates.
method ROTI-GCV framework for cross-validation under proportional asymptotics regime.
result Demonstrated accuracy of ROTI-GCV in synthetic and semi-synthetic settings.
Unified framework for estimating high-dimensional conditional factor models.
problem Estimating high-dimensional conditional latent factor models with practical limitations.
method Constrained nuclear norm regularization and cross-validation for parameter selection.
result Imposing homogeneity improves model predictability, with new method outperforming alternatives.
This paper proposes an out-of-sample extension framework for a global manifold learning algorithm (Isomap) that uses temporal information in out-of-sample points in order to make the embedding more robust to noise and artifacts. Given a set of noise-free training data and its embedding, the proposed framework extends t…
We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedu…
We consider the problem of vertex classification for graphs constructed from the latent position model. It was shown previously that the approach of embedding the graphs into some Euclidean space followed by classification in that space can yields a universally consistent vertex classifier. However, a major technical d…
Graph embeddings, a class of dimensionality reduction techniques designed for relational data, have proven useful in exploring and modeling network structure. Most dimensionality reduction methods allow out-of-sample extensions, by which an embedding can be applied to observations not present in the training set. Appli…
Using virtual stock markets with artificial interacting software investors, aka agent-based models (ABMs), we present a method to reverse engineer real-world financial time series. We model financial markets as made of a large number of interacting boundedly rational agents. By optimizing the similarity between the act…
Paper introduces DOO models to outperform SAA out-of-sample.
problem Outperforming SAA in out-of-sample performance.
method Introduces DOO models that consider both worst-case and best-case scenarios.
result DOO models can always outperform SAA out-of-sample.
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
LSTM and gradient boosting models fail to outperform random chance in predicting MNQ futures.
problem Predicting intraday direction in MNQ futures using LSTM and gradient boosting.
method Comparing LSTM and gradient boosting models on 944 trading days of MNQ futures data.
result No model achieves statistically significant accuracy above random chance.
Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial training sample. In this work, we consider the problem of obtaining an out-of-sample extension for the adjacency spectral embedding, a procedure…
FinBERT model identifies key speakers in earnings calls, boosting stock returns.
problem Unequal impact of all speakers in earnings call transcripts on stock returns.
method Utilized FinBERT, a domain-specific transformer model, to parse transcripts and weight speakers' sentiment.
result FinBERT section-weighted sentiment generates significant long-short alpha of 2.03%.