New framework for learning policies that converge in out-of-sample regions.
problem Reliable out-of-sample recovery in imitation learning.
method Contractive dynamical systems and recurrent equilibrium networks.
result Policy rollouts converge regardless of perturbations, enabling efficient OOS recovery.
Paper analyzes high-dimensional portfolio risks and finds empirical out-of-sample relative loss is more reliable.
problem Analyzing risks in high-dimensional portfolios using empirical variance.
method Derives asymptotic behavior of out-of-sample variance and relative loss in high-dimensional settings.
result Empirical out-of-sample relative loss is more reliable than variance in high-dimensional portfolios.
This research extends the Pareto/NBD model using neural networks for better out-of-sample predictions.
problem The limitations of the Pareto/NBD model in predicting out-of-sample data.
method A neural network-based extension of the Pareto/NBD model.
result The proposed method shows extraordinary predictability on repeat purchases at individual and aggregate levels.
Automates selection and visualization of model responses in various directions.
problem Manual selection and limited visualization of PDPs.
method Formalizes method for automating PDP selection and extends to arbitrary directions.
result Demonstrates usefulness across model selection, bias detection, and latent space exploration.
High-performing equity factor with Sharpe ratio above 13 out-of-sample.
problem Hidden cross-sectional predictability in stock returns.
method Regime-conditional signal activation combining value and short-term reversal signals.
result Annualized returns of 158.6% with 12.0% volatility, strong performance out-of-sample.
Managing a portfolio to a risk model can tilt the portfolio toward weaknesses of the model. As a result, the optimized portfolio acquires downside exposure to uncertainty in the model itself, what we call "second order risk." We propose a risk measure that accounts for this bias. Studies of real portfolios, in asset-by…
Paper finds significant impact of stock market swings on equity risk premium predictability.
problem Predicting equity risk premium based on stock market behavior changes.
method Introduced Bullish Index and used FDMAA for returns analysis; considered 28 indicators.
result Positive shocks in Bullish Index correlate with strong equity risk premium predictability for up to six months, while negative shocks correlate for up to nine months.
Researchers adaptively analyze market regimes to reveal investor behavior shifts.
problem Market relationships shift across different regimes, affecting investor behavior.
method Combining Kalman filtering, Markov-switching, and asymmetric response estimation.
result Foreign investors' predictive power increases during crises, while individual investors react more strongly to positive shocks.
Model predicts stock returns from order arrivals and cancellations.
problem Forecasting intraday stock returns using limit order book dynamics.
method Microscopic model based on operator algebra for order arrivals and cancellations, estimating arrival and cancellation rate distributions.
result The model explains 80% of returns in in-sample forecasts and 15% in out-of-sample forecasts.
In this paper we estimate the mean-variance portfolio in the high-dimensional case using the recent results from the theory of random matrices. We construct a linear shrinkage estimator which is distribution-free and is optimal in the sense of maximizing with probability 1 the asymptotic out-of-sample expected utilit…
New bounds for DeepONets reduce out-of-sample error without width dependence.
problem Measuring out-of-sample error in DeepONets without width dependence.
method Proved Rademacher complexity bound and Huber loss choice for DeepONets.
result Generalization error bounds have no explicit dependence on net size.
This paper proposes an out-of-sample extension framework for a global manifold learning algorithm (Isomap) that uses temporal information in out-of-sample points in order to make the embedding more robust to noise and artifacts. Given a set of noise-free training data and its embedding, the proposed framework extends t…
We consider the problem of vertex classification for graphs constructed from the latent position model. It was shown previously that the approach of embedding the graphs into some Euclidean space followed by classification in that space can yields a universally consistent vertex classifier. However, a major technical d…
A study finds that only a few factors explain corporate bond risk, rendering extensive bond factor literature redundant.
problem The redundancy of extensive bond factor literature in explaining corporate bond risk premia.
method Bayesian Model Averaging Stochastic Discount Factor analysis of 18 quadrillion models.
result A Bayesian Model Averaging SDF explains risk premia better than low-dimensional models, with an out-of-sample Sharpe ratio of 1.5 to 1.8.
New method for estimating out-of-sample R² from gene expression data.
problem Lack of a well-defined and unbiased estimator for out-of-sample R².
method Explicitly defined out-of-sample R², provided an unbiased estimator, and calculated standard error.
result Demonstrated improved model comparison for gene expression phenotypes.
A growing number of threats to Android phones creates challenges for malware detection. Manually labeling the samples into benign or different malicious families requires tremendous human efforts, while it is comparably easy and cheap to obtain a large amount of unlabeled APKs from various sources. Moreover, the fast-p…
Graph embeddings, a class of dimensionality reduction techniques designed for relational data, have proven useful in exploring and modeling network structure. Most dimensionality reduction methods allow out-of-sample extensions, by which an embedding can be applied to observations not present in the training set. Appli…
The "double descent" risk curve was proposed to qualitatively describe the out-of-sample prediction accuracy of variably-parameterized machine learning models. This article provides a precise mathematical analysis for the shape of this curve in two simple data models with the least squares/least norm predictor. Specifi…
The paper introduces eigen-portfolios using PCA to improve portfolio construction in finance.
problem Overfitting and poor generalization in selecting a single eigen-portfolio.
method Principal Component Analysis (PCA) to derive eigen-portfolios from asset return correlation matrices.
result An ensemble strategy combining multiple top-performing eigen-portfolios significantly improves out-of-sample performance.
Paper introduces DOO models to outperform SAA out-of-sample.
problem Outperforming SAA in out-of-sample performance.
method Introduces DOO models that consider both worst-case and best-case scenarios.
result DOO models can always outperform SAA out-of-sample.
Logit models are usually applied when studying individual travel behavior, i.e., to predict travel mode choice and to gain behavioral insights on traveler preferences. Recently, some studies have applied machine learning to model travel mode choice and reported higher out-of-sample predictive accuracy than traditional …
Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial training sample. In this work, we consider the problem of obtaining an out-of-sample extension for the adjacency spectral embedding, a procedure…
AugBagg improves random forest accuracy with added noise variables.
problem Improving model accuracy with random forest.
method AugBagg procedure using additional noise variables.
result Out-of-sample predictive accuracy improved with AugBagg.
OTSL improves structure learning accuracy with out-of-sample and resampling strategies.
problem Determining optimal hyperparameters for structure learning algorithms.
method Out-of-sample Tuning for Structure Learning (OTSL) using resampling strategies.
result Improves graphical accuracy of structure learning algorithms.
Stochastic Gradient Descent can overfit after just a few passes, contrary to initial expectations.
problem Understanding the out-of-sample performance of multi-pass SGD in stochastic convex optimization.
method Analysis of multi-pass SGD in the stochastic convex optimization model.
result Multi-pass SGD can lead to significant overfitting after just a few passes, contrary to initial expectations.
Machine learning models accurately predict molecular magnetic anisotropy tensors.
problem Accurately modeling molecular magnetic anisotropy tensors.
method Gaussian-moment neural-network approach for machine learning.
result Achieved accuracy of 0.3--0.4 cm−1 for magnetic anisotropy tensor predictions. A new model captures financial asset returns' tail behaviors and outperforms GARCH family.
problem Capturing the dynamic tail behaviors of financial asset returns.
method Combines LSTM with a novel parametric quantile function.
result Out-of-sample forecasts of conditional quantiles or VaR outperform GARCH family.
Bayesian additive regression trees (BART) (Chipman et. al., 2010) is a powerful predictive model that often outperforms alternative models at out-of-sample prediction. BART is especially well-suited to settings with unstructured predictor variables and substantial sources of unmeasured variation as is typical in the so…
Dimensionality reduction methods are very common in the field of high dimensional data analysis. Typically, algorithms for dimensionality reduction are computationally expensive. Therefore, their applications for the analysis of massive amounts of data are impractical. For example, repeated computations due to accumula…
Paper proposes a diagnostic tool for evaluating model performance out-of-sample.
problem Evaluating model performance on unseen data.
method Uses a finite calibration dataset to assess future losses.
result Provides guarantees under weak assumptions and quantifies distribution shifts.
In this paper we present a regression based model for day-ahead electricity spot prices. We estimate the considered linear regression model by the lasso estimation method. The lasso approach allows for many possible parameters in the model, but also shrinks and sparsifies the parameters automatically to avoid overfitti…
We identify and validate a model for PCR in high dimensions, improving prediction guarantees.
problem Model identification and out-of-sample prediction in high-dimensional error-in-variables settings.
method Analysis of principal component regression (PCR) in fixed design settings, introducing a linear algebraic condition.
result Consistent model identification and improved out-of-sample prediction guarantees.
Study the impact of overfitting on linear predictive models' performance.
problem Overfitting reduces the out-of-sample performance of linear predictive trading strategies.
method Computed in- and out-of-sample means and variances of PnLs to derive replication ratios.
result Replication ratio diminishes for complex strategies with many assets.
Optimizes decisions without knowing the true distribution using historical data.
problem Optimizing decisions without knowing the true distribution.
method Combines sampling and bisection search algorithms to solve an optimization problem.
result Proves sufficient conditions for local out-of-sample optimality.
Deep reinforcement learning techniques have demonstrated superior performance in a wide variety of environments. As improvements in training algorithms continue at a brisk pace, theoretical or empirical studies on understanding what these networks seem to learn, are far behind. In this paper we propose an interpretable…
Bayesian neural networks show good correlation between out-of-sample performance and Bayesian evidence.
problem Improving the out-of-sample performance of Bayesian neural networks.
method Numerical sampling of Bayesian posterior, ensembling over architectures, analysis of evidence vs. model size.
result Good correlation between out-of-sample performance and Bayesian evidence; ensembling improves performance.
Paper develops a method to predict spatial point processes with guarantees.
problem Predicting the number of events in space with uncertainty.
method Regularized method to learn spatial models with out-of-sample guarantees.
result Method provides valid prediction intervals even when model is misspecified.
Improves test set performance and reduces out-of-sample disappointment for unstable models.
problem Ensuring strong test set performance via cross-validation for unstable models.
method Nested k-fold cross-validation with hyperparameter selection based on a weighted sum of cross-validation metric and model stability measure.
result Improves out-of-sample MSE for sparse ridge regression and CART by 4% and 2% respectively, compared to k-fold cross-validation.
The paper analyzes LOCV for high-dimensional risk estimation, proving error bounds.
problem Estimating out-of-sample prediction error in high-dimensional settings.
method Theoretical analysis of leave-one-out cross validation (LOCV) in penalized regression.
result Finite sample upper bounds on LOCV error, showing it converges to zero as n,p → ∞.
Paper presents a new way to analyze machine learning generalization without probabilistic assumptions.
problem Traditional generalization analysis assumes i.i.d. data, which is often unverifiable.
method Uses sensitivity analysis of optimization problems to derive deterministic generalization bounds.
result Obtains generalization bounds that relate in-sample and out-of-sample evaluations through an error term quantifying data similarity.
Under the framework of spectral clustering, the key of subspace clustering is building a similarity graph which describes the neighborhood relations among data points. Some recent works build the graph using sparse, low-rank, and ℓ2-norm-based representation, and have achieved state-of-the-art performance. Howeve…
Improved genetic programming by optimizing mutation operators for continuous program search.
problem Small syntactic mutations in genetic programming can lead to unpredictable behavioral shifts.
method Learned a compact trading-strategy DSL, created a block-factorized embedding, and designed geometry-compiled mutation operators.
result Geometry-compiled mutation operators discover strong strategies using fewer evaluations and achieve higher Sharpe ratios.
Diffusion maps are a nonlinear manifold learning technique based on harmonic analysis of a diffusion process over the data. Out-of-sample extensions with computational complexity O(N), where N is the number of points comprising the manifold, frustrate applications to online learning applications requiring…
Enhances supervised visualization for unseen data using autoencoders and random forest.
problem Lack of generalization to unseen test sets in supervised dimensionality reduction.
method Combines autoencoder and random forest proximities for out-of-sample extension.
result 40% reduction in training time with 10% of training data, achieving consistent quality.
The paper uses machine learning to forecast macroeconomic outcomes with high-dimensional data.
problem Forecasting the full conditional distribution of macroeconomic outcomes.
method Systematically integrating three key principles: high-dimensional data with regularization, rigorous out-of-sample validation, and incorporating nonlinearities.
result Regularization via shrinkage is essential to control model complexity, while nonlinearities yield limited improvements in predictive accuracy.
Proposes a new model to maximize out-of-sample Sharpe ratios by forecasting tangency portfolios.
problem Maximizing Sharpe ratios when returns and covariances are not stationary.
method Forecast the tangency portfolio using vector autoregressions and invest in the minimum Euclidean distance portfolio.
result Empirically validated superior out-of-sample Sharpe ratios.
Framework mitigates risk non-monotonicity in high-dimensional predictions.
problem Risk non-monotonicity in high-dimensional predictions.
method Model-agnostic framework using cross-validation and data-driven methodologies (zero- and one-step).
result Modified prediction procedures achieve monotonic asymptotic risk behavior.
Quarter-hour market bursts predict algorithmic trading and returns in crypto futures.
problem Predicting returns in cryptocurrency futures markets using quarter-hour market bursts.
method Analysis of trade data and Autocorrelation Map to identify and quantify algorithmic trading activity.
result Quarter-hour market bursts are associated with algorithmic trading and can predict returns.