Benchmarking deep time series models for equity portfolios
problem Selecting the best deep time series model for equity portfolios
method Using a CRSP benchmark and multi-criteria acceptability analysis
result No architecture dominates the benchmark, with TransEnc-8 having the highest rank-1 acceptability
Study decomposes market portfolio into body and tail legs, revealing systematic differences.
problem Understanding the relationship between body and tail components in market portfolios.
method Decomposes CRSP market portfolio into body and tail legs, analyzes their recombination identity.
result Recombination identity holds for all models but not for all, indicating systematic differences.
Performance of investment managers are evaluated in comparison with benchmarks, such as financial indices. Due to the operational constraint that most professional databases do not track the change of constitution of benchmark portfolios, standard tests of performance suffer from the "look-ahead benchmark bias," when t…
Market portfolio decomposed into body and tail legs
problem Separation of market portfolio into body and tail legs
method Dynamic value-weighted body and tail legs
result Recombination identity holds for all models
Study macroscopic equity market properties affecting active strategies.
problem Lack of adequate models for active equity strategies.
method Empirical study using CRSP Database, focusing on market capitalizations and returns.
result Highlight stylized facts and open questions in equity markets.
Tests factor models by decomposing market into body and tail legs, revealing inconsistent results.
problem Inconsistency between factor models and market behavior.
method Decomposes market into body and tail legs, testing factor models at daily and monthly frequencies.
result q5 model shows inconsistent results, with negative body and positive tail alphas at all split ratios.
This paper diagnoses factor-model pricing errors using a new method.
problem Measuring pricing errors in factor models with general characteristic axes.
method Developed a method to measure factor-model pricing errors as bridge-alpha curves, using a predetermined characteristic order and prefix portfolios.
result Adding a counterpart factor flips the curve's sign on every axis, but only HML and CMA overcorrect enough to be rejected.
In this article, the long-term behavior of the stock market index of the New York Stock Exchange is studied, for the period 1950 to 2013. Specifically, the CRSP Value-Weighted and CRSP Equal-Weighted index are analyzed in terms of market efficiency, using the standard ratio variance test, considering over 1600 one week…
The MSPI predicts market stress with machine learning.
problem Estimating the probability of high market stress.
method L1-regularized logistic regression on stock fragility signals.
result MSPI tracks major stress episodes and improves accuracy.
Develops a new method for optimizing portfolios in stochastic markets.
problem Optimizing functionally generated portfolios in stochastic portfolio theory.
method Optimizes over a family of rank-based portfolios parameterized by an exponentially concave function.
result Proves existence and uniqueness of the optimization problem and provides stability estimates.
The study extends SPT to account for real-world transaction costs, improving portfolio performance.
problem Real-world transaction costs affect portfolio performance, especially during market stress.
method Developed a continuous-time model with stochastic transaction costs and derived lower bounds for cost-adjusted wealth.
result Functionally generated portfolios can still achieve relative arbitrage after accounting for transaction costs.
Researchers have constantly asked whether stock returns can be predicted by some macroeconomic data. However, it is known that macroeconomic data may exhibit nonstationarity and/or heavy tails, which complicates existing testing procedures for predictability. In this paper we propose novel empirical likelihood methods …
Test-asset construction affects factor model performance.
problem How test assets are constructed impacts factor model performance.
method Forming characteristic-unsorted random portfolios and varying stock selection, initial weighting, holding, and rebalancing.
result Test-asset construction shifts factor model rankings materially.
The paper diagnoses factor models using characteristic axes and zero-curve restrictions.
problem Tackles systematic sign reversals and overcorrections in factor model pricing errors.
method Extends cap-axis integral diagnostic to general characteristic axes, measuring pricing errors as bridge-alpha curves.
result Axis-level pricing errors are nearly orthogonal to maximum-Sharpe gains, showing systematic sign reversals and overcorrections.
The paper diagnoses factor-model pricing errors using characteristic axes and bridge-alpha curves.
problem Tackles systematic sign reversals and overcorrections in factor-model pricing errors.
method Extends cap-axis integral diagnostic to characteristic axes, measures pricing errors as bridge-alpha curves, and uses a predetermined characteristic order to generate zero-curve restrictions.
result Axis-level pricing errors are nearly orthogonal to maximum-Sharpe gains, showing significant sign reversals and overcorrections.
Proposes a diagnostic method to evaluate factor models using cap-axis integrals.
problem Improving factor model evaluation in low-dimensional spaces.
method Lifts pricing errors into a bridge-alpha curve along the market-capitalization rank axis.
result The cap-axis norm is distinct from Sharpe gain and size exposure.
Proposes a diagnostic method to evaluate factor models using cap-axis integrals.
problem Improving factor model evaluation for low-dimensional models.
method Lifts pricing errors into a bridge-alpha curve along the market-capitalization rank axis.
result The cap-axis norm is distinct from Sharpe gain and size exposure.
This paper improves risk control for financial markets by calibrating VaR forecasts using conformal methods.
problem Nonstationary and regime-dependent losses in financial markets.
method Regime-weighted conformal risk control (RWC) for VaR forecasting.
result RWC improves regime-conditional stability in some settings with modest conservativeness changes.
Research evaluates three risk models for portfolio construction during market downturns.
problem Challenges in constructing quantitative portfolios using statistical risk models.
method Three statistical risk models tested on 1,000 stocks across four periods.
result Models consistently outperform market returns in various crises.
The study uses equity order flow to forecast stock returns and resolves the liquidity premium puzzle.
problem The liquidity premium and its relation to investment horizons.
method Directly estimated Kyle's price-impact coefficient λ from daily equity order flow data.
result Signed order flow predicts stock returns, with volume volatility predicting lower returns.
Detecting changes in asset co-movements is of much importance to financial practitioners, with numerous risk management benefits arising from the timely detection of breakdowns in historical correlations. In this article, we propose a real-time indicator to detect temporary increases in asset co-movements, the Autoenco…
New method identifies algo trading strategies as liquidity consumers or providers.
problem Determining if algo trading strategies consume or provide liquidity.
method Analyzes trade and price history to classify strategies as liquidity consumers or providers.
result Identifies net liquidity consumption or provision of algo trading strategies.
Bayesian approach confirms no return predictability for 1926-2004 data, weak evidence for 1953-2021.
problem Investigating return predictability using Bayesian methods.
method Developed a new shrinkage type prior for a model parameter in a VAR system, compared to other estimation methods.
result Bayesian approach outperforms reduced-bias estimator in terms of size and power.
New method accurately reconstructs Russell 3000 index, revealing crowded portfolios.
problem Crowding in index portfolios during reconstitution events.
method Developed a Python package for accurate index reconstruction using CRSP US Stock data.
result Annual Russell 3000 portfolios are more crowded than quarterly ones, suggesting lower transaction costs.
New model predicts stock performance in large equity markets.
problem Predicting stock performance in large equity markets over long time horizons.
method Rank-based volatility stabilized models calibrated to empirical data.
result The model exhibits relative arbitrage and statistically fits empirical features.
Develops a continuous compliance index for Islamic equity screening.
problem Binary rulebooks lead to inconsistent compliance assessment of firms.
method Integrates six leading financial and business activity standards into a single continuous index.
result Firms with the same pass/fail label can differ significantly in compliance strength.
Continuous Hidden Markov Models for Equity Returns
problem Generating synthetic equity returns that match real return characteristics
method Continuous Hidden Markov Models
result Recovered volatility clustering and narrowed kurtosis gap
YAHPO Gym introduces a new benchmark for evaluating hyperparameter optimization methods.
problem Evaluating and comparing hyperparameter optimization methods on well-curated benchmark suites.
method Surrogate-based benchmark collection of 14 scenarios, each with multi-fidelity and multi-objective hyperparameter optimization problems.
result Surrogate-based benchmarks produce more faithful results than tabular benchmarks.
Study constructs a Japanese financial LLM benchmark.
problem Need for domain-specific benchmarks for LLMs.
method Constructed a benchmark with multiple Japanese and financial domain tasks.
result GPT-4 outperforms other models in the benchmark.
A new sparse benchmark metabench identifies key abilities from large benchmarks.
problem Redundancy and compression in existing benchmarks.
method Data from 5000+ LLMs to identify most informative items, distilling a sparse benchmark.
result Sparse benchmark metabench captures underlying abilities with high accuracy.
Machine learning research depends on objectively interpretable, comparable, and reproducible algorithm benchmarks. We advocate the use of curated, comprehensive suites of machine learning tasks to standardize the setup, execution, and reporting of benchmarks. We enable this through software tools that help to create an…
This article provides a comprehensive study of different ways to make speed benchmarks of gradient boosted decision trees algorithm. We show main problems of several straight forward ways to make benchmarks, explain, why a speed benchmarking is a challenging task and provide a set of reasonable requirements for a bench…
Study optimizes portfolio to minimize relative drawdown duration, penalizing unfavorable performance states.
problem Minimizing relative drawdown duration in portfolio optimization relative to a benchmark.
method Introduces a benchmark-relative drawdown-duration criterion penalizing unfavorable performance states. Uses a one-dimensional Markovian representation and Hamilton-Jacobi-Bellman equation.
result Derives explicit projection-based characterization of the optimal feedback control and identifies geometric settings for unique strong solutions.
Deployment-complete benchmarking assesses if evidence leads to consistent deployment actions.
problem Lack of clear evidence leading to consistent deployment actions.
method Introduces deployment-complete benchmarking to test if benchmark evidence determines deployment actions.
result Benchmark evidence must be complete for a claim to lead to a consistent deployment action.
We give an explicit formulaic algorithm and source code for building long-only benchmark portfolios and then using these benchmarks in long-only market outperformance strategies. The benchmarks (or the corresponding betas) do not involve any principal components, nor do they require iterations. Instead, we use a multif…
Tiny benchmarks reduce LLM evaluation costs by using fewer examples.
problem Expensive evaluation of LLMs with tens of thousands of examples.
method Developed evaluation tools and tiny versions of popular benchmarks.
result Accurately estimate LLM performance with just 100 curated examples.
The past few years have seen a surge of applying Deep Learning (DL) models for a wide array of tasks such as image classification, object detection, machine translation, etc. While DL models provide an opportunity to solve otherwise intractable tasks, their adoption relies on them being optimized to meet latency and re…
Benchmark for DL inference on embedded HWAs, focusing on autonomous driving.
problem Lack of comprehensive benchmarks for DL hardware.
method Developed a benchmark for inference on embedded HWAs, focusing on autonomous driving. Proposed new granularity, benchmark procedures, and performance indicators.
result Identifies mismatches between HWAs and DL models.
The optimization of algorithm (hyper-)parameters is crucial for achieving peak performance across a wide range of domains, ranging from deep neural networks to solvers for hard combinatorial problems. The resulting algorithm configuration (AC) problem has attracted much attention from the machine learning community. Ho…
Optimal benchmark design varies based on costs in financial manipulation.
problem Manipulation of price benchmarks in finance.
method Analyzes empirical pattern and cost structures to determine optimal benchmark design.
result The optimal benchmark depends on the relative sizes of fixed and variable costs.
Generates synthetic data for benchmarking unsupervised outlier detection.
problem Difficulty in benchmarking unsupervised outlier detection due to rare and varied outliers in real data.
method Proposes a generic process to generate synthetic data with insightful characteristics.
result Demonstrates practicality of the generic process through a benchmark with state-of-the-art detection methods.
Paper introduces benchmark-neutral pricing for long-term contracts.
problem High prices of long-term contracts under risk-neutral pricing.
method Uses growth optimal portfolio as numeraire and new pricing measure.
result Identifies minimal possible prices for contingent claims.
In this report, we present a new reinforcement learning (RL) benchmark based on the Sonic the Hedgehog (TM) video game franchise. This benchmark is intended to measure the performance of transfer learning and few-shot learning algorithms in the RL domain. We also present and evaluate some baseline algorithms on the new…
Fidel-TS creates a new benchmark for time series forecasting models.
problem Lack of high-quality benchmarks for time series forecasting models.
method Formalized high-fidelity benchmark principles, including data sourcing integrity, leak-free design, and structural clarity. Created Fidel-TS, a new large-scale benchmark.
result Demonstrated the limitations of prior benchmarks and potential discrepancies in model evaluation.
New framework assesses and benchmarks ML methods for multivariate time series.
problem Benchmarking and explaining performance of machine learning methods.
method Proposes a new framework with systematized performance-explainability characteristics.
result Illustrates application to multivariate time series classifiers.
Study proposes new methods to calculate probabilistic benchmarks in noisy data.
problem Identifying opportunities for improvement in comparable units with noisy data.
method 2-step methodology involving undersampling and relevance vector machine.
result Higher discrimination power achieved with macro-economic environment variables.
We solve the multi-criteria benchmarking problem by formalizing it as a social choice problem and identifying conditions for meaningful rankings.
problem Aggregating multiple metrics into a single ranking for models in benchmarking problems.
method Formalizing multi-criteria benchmarking as a social choice problem and identifying sufficient conditions for meaningful rankings.
result We prove that meaningful multi-criteria benchmarking becomes possible under certain preference conditions (single-peaked, group-separable, distance-restricted).
We study the pricing and hedging of derivatives in incomplete financial markets by considering the local risk-minimization method in the context of the benchmark approach, which will be called benchmarked local risk-minimization. We show that the proposed benchmarked local risk-minimization allows to handle under extre…