This work discusses verifying the correctness of backtest engines for trading strategies.
problem Ensuring the correctness of backtest engines for trading strategies is challenging.
method Develop models for candle data and intra-period prices, provide tests for verification, and suggest algorithmic considerations.
result A proof of correctness for a backtest engine can be conducted if specific model tests are successful.
New metrics quantify implementation risk in portfolio backtesting, revealing systematic differences in engine implementations.
problem Systematic divergence in backtested portfolio metrics due to differences in engine implementations.
method Formalized implementation risk, proposed four metrics, executed 15 strategies through five engines, analyzed source-code defects.
result Implementation risk introduces measurable ambiguity in performance attribution, but does not alter investment decisions.
Calibrating a trading rule using a historical simulation (also called backtest) contributes to backtest overfitting, which in turn leads to underperformance. In this paper we propose a procedure for determining the optimal trading rule (OTR) without running alternative model configurations through a backtest engine. We…
Neural-SDE model accurately simulates option risks.
problem Estimating accurate risk scenarios for option portfolios.
method Arbitrage-free neural-SDE market model for joint option dynamics.
result Models produce more efficient and accurate VaR evaluations.
Factor Engine simplifies financial factor computation and analysis in Python.
problem Efficient computation and analysis of financial factors.
method Modular, extensible Python library with decorators, integrates with data science ecosystem.
result Mispricing factors computed by Factor Engine and Stata implementation are highly similar.
A new method for backtesting ES forecasts in banking.
problem Designing a model-free backtesting procedure for Expected Shortfall forecasts.
method Use e-values and e-processes to introduce backtest e-statistics for VaR and ES.
result The proposed method can be applied to various risk measures and statistical quantities.
The paper develops new backtests for Expected Shortfall risk measure.
problem Estimating ES forecasts directly is challenging; existing tests require Value at Risk forecasts.
method Developed a joint regression framework for Value at Risk and Expected Shortfall, providing robust covariance estimators.
result The new backtests significantly outperform existing methods in simulations and empirical applications.
New spectral backtests improve forecast distribution evaluation in risk management.
problem Evaluating forecast accuracy in risk management.
method Spectral backtests using kernel measures to weight exceedance events.
result Novel spectral backtests offer improved performance and are easy to implement.
New method tests risk measures for various distortions.
problem Testing risk measures for different distortions.
method Stratification and randomization of risk levels.
result Method performs well in numerical case studies.
New backtests improve bank risk measure forecasts.
problem Traditional backtests inadequately assess risk measure forecasting accuracy.
method Proposes comparative backtests for assessing forecasting accuracy of risk measures.
result Comparative backtests enhance trading book regulatory framework and internal risk management.
Cryptocurrency patterns stable across market caps, validated by microstructure theory.
problem Stable patterns in cryptocurrency microstructure across different market caps.
method Unified CatBoost modeling pipeline with time-series cross validation, validated by backtests.
result Feature rankings and partial effects are stable across assets despite heterogeneous liquidity and volatility.
Backtesting ES and VaR is possible using Diebold-Mariano tests.
problem Eliciting and testing risk measures like ES and VaR.
method Use Diebold-Mariano tests for ES, based on Fissler and Ziegel (2015).
result ES can be jointly elicitable with VaR.
The paper proposes a method to discount backtest PnLs due to in-sample overfitting.
problem In-sample overfitting in backtest-based investment strategies.
method A simple framework to model and quantify in-sample PnL overfitting.
result Computes the appropriate discount factor for PnLs of in-sample investment strategies.
Backtesting framework for CLMMs on Uniswap V3 reduces reward estimation error.
problem Estimating rewards for CLMMs in Uniswap V3 liquidity pools.
method Parametric model for liquidity distribution, historical data analysis.
result Error in reward estimation less than 1% for each pool.
A new backtesting framework for Expected Shortfall simplifies risk measurement.
problem Backtesting Expected Shortfall for regulatory compliance.
method Combining risk and cash-flow into a secured position, using monotonicity of Expected Shortfall.
result A simple test statistic efficiently backtests Expected Shortfall.
New method detects and adjusts for temporal leakage in LLM backtests.
problem Standard backtest leakage detection is ineffective for modern models.
method Developed new methods to measure and adjust for temporal leakage.
result Demonstrated that models legitimately know more about times near their cutoffs, leading to structural leakage.
Backtests of structured strategies lose much of their predictive power in live trading.
problem Uncertainty in how marketed backtests predict live performance of structured strategies.
method Analysis of 1,726 structured strategies from ten global institutions.
result Raw backtests have limited portability into live trading and deteriorate sharply.
Expected Shortfall (ES) has been widely accepted as a risk measure that is conceptually superior to Value-at-Risk (VaR). At the same time, however, it has been criticised for issues relating to backtesting. In particular, ES has been found not to be elicitable which means that backtesting for ES is less straightforward…
The paper tackles backtest overfitting in cryptocurrency trading using deep reinforcement learning.
problem Backtest overfitting in deep reinforcement learning for cryptocurrency trading.
method Formulated hypothesis test for overfitting detection, trained agents, estimated overfitting probability, and rejected overfitted agents.
result Less overfitted deep reinforcement learning agents outperformed more overfitted agents and market benchmarks.
The paper introduces Lambda VaR, a new risk measure, and develops three backtesting methods.
problem Theoretical limitations and practical challenges of VaR in risk management.
method Three nonparametric backtesting methodologies for Lambda VaR.
result Demonstrates the effectiveness of Lambda VaR and its backtesting methods.
New method allows backtesting of systemic risk forecasts.
problem Systemic risk measures are not elitable and identifiable, making backtesting impossible.
method Introduces multi-objective elicitability and Diebold--Mariano type tests.
result Proposes a traffic-light approach for backtesting.
A new method tests Expected Shortfall by analyzing both duration and severity of VaR violations.
problem Lack of separate testing for frequency and severity in ES backtesting.
method Uses bivariate orthogonal polynomials to derive moment conditions for durations and severities.
result Proposes a Wald test for identifying mis-specified components in ES models.
The paper examines sizing strategies for algorithmic trading in volatile markets.
problem High volatility creates challenges for algorithmic traders.
method Investigates different sizing models and backtesting techniques for financial trading.
result Sizing models can lower Value at Risk (VaR) during crisis events.
The paper uses machine learning to simulate financial markets and improve trading strategy backtesting.
problem Improving risk management of quantitative investment strategies.
method Simulates financial markets using Boltzmann Machines and Generative Adversarial Networks to preserve asset return distributions and dependencies.
result Developed a framework to estimate backtest statistics more accurately.
In this paper we try to design the necessary calculation needed for backtesting trading systems when only candle chart data are available. We lay particular emphasis on situations which are not or not uniquely decidable and give possible strategies to handle such situations.
This paper investigates bias in resampled backtests for financial portfolios, finding it often negligible.
problem Bias in resampled backtests for financial portfolio evaluation.
method Investigation of bias in rolling-window mean-variance portfolios using resampling techniques.
result The bias in Sharpe Ratio estimates from IID resampling is often a fraction of estimation noise, making it tolerable.
This paper proposes a new method to prevent backtesting overfitting in trading strategies.
problem Preventing misleading results in backtesting of trading strategies.
method Covariance-Penalty Correction approach to reduce risk metrics based on the number of parameters and data used.
result Covariance-Penalties are effective in avoiding backtesting overfitting, with Total Least Squares outperforming Ordinary Least Squares.
A new stock selection strategy uses combined machine learning with dynamic weighting methods.
problem Improving stock selection accuracy and performance.
method Combined machine learning algorithms with static and dynamic weighting methods.
result IC-based dynamic weighting outperforms static evaluation metrics in backtested returns and predictive performance.
Using non-linear machine learning methods and a proper backtest procedure, we critically examine the claim that Google Trends can predict future price returns. We first review the many potential biases that may influence backtests with this kind of data positively, the choice of keywords being by far the greatest culpr…
The paper proposes a simple multinomial VaR test for backtesting expected shortfall risk measures.
problem Backtesting of expected shortfall risk measures for the trading book under FRTB.
method Multinomial test of VaR exceptions at different levels, approximating ES in terms of multiple quantiles.
result Multinomial tests with N≥4 are more powerful than binomial tests at detecting model misspecifications. Benchmark detects decision-time leakage in financial backtests.
problem Detecting decision-time leakage in financial machine-learning backtests.
method Toggles one evaluation convention at a time around a clean t+1-open reference, holding other factors fixed. result Inflation is highly selective, affecting specific features and execution methods.
New method corrects risk estimation bias, improving backtesting results.
problem Underestimation of risk by existing methods, especially in small samples.
method Proposes a new algorithm for bias correction using generalized Pareto distributions.
result The new algorithm leads to improved efficiency in estimating risk with heavy tails or heteroscedasticity.
Modified Anderson-Darling test improves counterparty credit risk model accuracy.
problem Limited sample size impacts Anderson-Darling test effectiveness in counterparty credit risk models.
method Proposed a modified Anderson-Darling test for better volatility detection in counterparty credit risk models.
result Modified test detects underestimation of model's volatility more efficiently.
NoxTrader predicts stock returns using LSTM for profitable trading.
problem Predicting profitable stock returns for quantitative trading.
method LSTM model for time-series analysis of stock data.
result Improved investment return from -60% to 325%.
The paper optimizes portfolios using clustering and Sharpe ratio-based optimization.
problem Optimizing portfolio performance in financial modeling.
method Combines K-Means clustering for asset segmentation and Sharpe ratio-based optimization.
result Optimized portfolios outperform traditional equal-weighted benchmarks.
AutoQuant addresses cryptocurrency backtesting fragility by modeling execution costs and improving strategy selection.
problem Fragile backtests of cryptocurrency perpetual futures ignoring microstructure frictions and execution costs.
method Execution-centric framework with Bayesian optimization, double screening, and strict T+1 semantics.
result Fee-only and zero-cost backtests overestimate returns, highlighting the importance of modeling execution costs.
Robust forecast framework reduces distribution error by 63%.
problem Accurate distribution forecast for planning decisions.
method Backtest-based bootstrap and adaptive residual selection.
result Reduces Absolute Coverage Error by more than 63%.
Machine learning creates non-factor covariance matrices for risk models.
problem Creating robust risk models for financial portfolios.
method Developed an explicit algorithm and source code for machine learning risk models.
result Machine learning models outperform traditional risk models in empirical backtests.
AlphaEval evaluates alpha mining models efficiently and comprehensively.
problem Lack of systematic evaluation for alpha mining models.
method Unified, parallelizable evaluation framework assessing predictive power, stability, robustness, financial logic, and diversity.
result AlphaEval achieves evaluation consistency comparable to comprehensive backtesting, providing more comprehensive insights and higher efficiency.
This research improves forecasting and testing of risk contributions using Expected Shortfall.
problem Improving risk allocation and testing methods for regulatory standards.
method Developed a comprehensive framework for backtesting and forecasting Expected Shortfall contributions.
result Proposed a novel semiparametric model for forecasting dynamic Expected Shortfall contributions.
This paper evaluates LLMs for technical market analysis, finding GPT-4 Turbo and FinGPT outperform passive benchmarks.
problem Evaluating LLMs for technical market analysis in financial markets.
method Structured evaluation of five LLMs (GPT-4 Turbo, Claude 3 Opus, Gemini 1.5 Pro, Llama 3 70B, FinGPT) on four tasks: candlestick pattern recognition, directional signal generation, backtesting, and financial report comprehension.
result GPT-4 Turbo and FinGPT outperform passive benchmarks in simulated backtesting, with GPT-4 Turbo achieving the highest annualized return and Sharpe ratio.
A new property fixes look-ahead bias in backtesting and trading pipelines.
problem Fixing look-ahead bias in backtesting and trading pipelines.
method Developed a pipeline calculus separating availability from reference time, and a type-and-effect system for the value-independent fragment.
result The check scales linearly and catches all leaks, including those missed by differential and tiling detectors.
Anonymizing company names in financial news improves trading performance, contrary to initial expectations.
problem Look-ahead and distraction biases in sentiment analysis of financial news.
method Investigated trading strategies based on original and anonymized headlines, comparing performance.
result Anonymized headlines outperform original in-sample, suggesting distraction effect is stronger.
Study compares quantum and classical ML in crypto trading, finding hybrid models outperform.
problem Comparing quantum and classical machine learning in crypto trading strategies.
method Backtesting 10 models across multiple crypto assets using classical ML, quantum ML, hybrid models, and transformer models.
result Hybrid quantum models achieve superior performance with 13.99% return and 1.76 Sharpe ratio.
New algorithm corrects risk estimation bias for heavy-tailed data.
problem Underestimation of risk in banking and insurance due to bias in estimation procedures.
method Proposes a new algorithm for bias correction and applies it to generalized Pareto distributions.
result The algorithm leads to more accurate risk estimation, especially in heavy-tailed data.
LLMs struggle to outperform markets over long periods and diverse stocks.
problem Overstated effectiveness of LLM-based investing strategies due to biases.
method FINSABER framework for systematic backtests over two decades and 100+ symbols.
result Previously reported LLM advantages deteriorate significantly under broader evaluation.
Deep RL for portfolio management shows poor robustness.
problem Robustness of Deep RL algorithms in online portfolio management.
method Proposed a training and evaluation process for assessing DRL algorithms.
result Most Deep RL algorithms are not robust, generalizing poorly and degrading quickly.
Study shows survivorship bias inflates returns in India's small-cap index.
problem Survivorship bias in emerging market small-cap indices.
method Reconstructing historical index composition through market capitalization ranking and comparing equal-weight portfolios of current constituents versus all historical members.
result Survivor-only backtesting overstates returns by 4.94 percentage points and Sharpe ratios by 0.097.