Machine learning forecasts show bias at long horizons, contrary to standard tests.
problem Forecast efficiency tests misinterpret machine learning performance.
method Theoretical and empirical analysis of regularization and measurement noise.
result Machine learning forecasts exhibit overreaction at longer horizons, not bias.
We detect lookahead bias in LLM forecasts using a novel statistical method.
problem Detecting lookahead bias in LLM-generated economic forecasts.
method Developed a statistical procedure using date-only recall queries and estimated Lookahead Propensity (LAP).
result LLM forecasts are contaminated with lookahead bias, as indicated by a positive interaction between LAP and the forecast in accuracy regressions.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
BC-ACI corrects time series forecast bias, improving prediction intervals.
problem Persistent bias in time series forecasts leads to overly conservative prediction intervals.
method Augments ACI with an EWM estimate of forecast bias to correct nonconformity scores and re-center intervals.
result Reduces Winkler interval scores by 13-17% under distribution shifts, improving calibration.
New measure corrects news bias in NLP stock return forecasting.
problem Improving stock return and volatility forecasting accuracy.
method Hype-Adjusted Probability Measure, sentiment score equation.
result Significantly improved forecast accuracy for U.S. semiconductor tickers.
TCFimt forecasts causal effects of multiple interventions from individual data.
problem Estimating causal effects of temporal multi-interventions from individual data.
method TCFimt uses adversarial tasks in seq2seq framework to alleviate bias and contrastive learning to decouple effects.
result TCFimt outperforms state-of-the-art methods in predicting future outcomes and choosing optimal treatments.
Introduces recency bias to improve time-series forecasting.
problem Lack of recency bias in standard Transformer attention for time-series data.
method Reweights attention scores with a smooth heavy-tailed decay to emphasize nearby observations.
result Recency-biased attention consistently improves sequential modeling and achieves competitive performance on time-series forecasting benchmarks.
ABC method improves subseasonal weather forecasting by 60-90%.
problem Improving subseasonal temperature and precipitation forecasting accuracy.
method Combines dynamical forecasts with machine learning-based bias correction.
result Significant improvement in temperature and precipitation forecasting skills.
Study compares ML algorithms for predicting stock market directional bias.
problem Predicting the direction of stock market movements.
method Examined and contrasted logistic regression, decision tree, random forest, and a deep neural network.
result All models consistently reach above 50% in directional bias forecasting.
PBC improves AI and dynamical subseasonal forecasts by reducing biases.
problem Subseasonal forecast accuracy drops due to model biases and compounding errors.
method Probabilistic bias correction (PBC) using machine learning to correct historical forecasts.
result PBC doubles AI Forecasting System's subseasonal skill and improves dynamical model skill.
Bayesian consensus improves accuracy of forecasts from miscalibrated sources.
problem Aggregating predictions from miscalibrated and noisy sources.
method Bayesian approach to adjust for bias and noise, using hierarchical models.
result Bayesian consensus estimator is unbiased and more efficient than alternatives.
New method corrects seasonal Arctic sea ice predictions with probabilistic models.
problem Systematic biases and errors in climate model forecasts of Arctic sea ice.
method Conditional Variational Autoencoder model to map observation distribution given biased model predictions.
result Probabilistic adjusted forecasts are better calibrated and have smaller errors.
A major source of risk in project management is inaccurate forecasts of project costs, demand, and other impacts. The paper presents a promising new approach to mitigating such risk, based on theories of decision making under uncertainty which won the 2002 Nobel prize in economics. First, the paper documents inaccuracy…
A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.
problem Lack of effective short-term load forecasting methods.
method Hybrid strategy integrating four different inference models: tree-based ensemble method Warm-start Gradient Tree Boosting (WGTB).
result Demonstrates effectiveness of hybrid strategy on real datasets.
DatedGPT prevents lookahead bias in financial forecasting models.
problem Lookahead bias in large language models trained on internet-scale data.
method Time-aware pretraining with annual data cutoffs and instruction fine-tuning.
result Models' knowledge is effectively bounded by their data cutoff year, improving forecasting validity.
Study on NNs for forecasting time series with novel control variable combinations.
problem Forecast future time series with novel combinations of control variables.
method Modular NN architecture with inductive bias for independence of control variables.
result Modular NN architecture improves forecasting of dependent variables up to large horizons.
Research reveals simplicity bias in random logistic map, impacting data analysis and forecasting.
problem Simplicity bias in dynamical systems and its impact on data analysis and prediction.
method Examined the logistic map and random logistic map, focusing on simplicity bias and noise effects.
result Simplicity bias is observable in the random logistic map, persisting even with small noise levels.
The primary goal of this study is doing a meta-analysis research on two groups of published studies. First, the ones that focus on the evaluation of the United States Department of Agriculture (USDA) forecasts and second, the ones that evaluate the market reactions to the USDA forecasts. We investigate four questions. …
The paper tackles fairness in forecasting and learning linear dynamical systems.
problem Under-representation bias in training data for multiple subgroups.
method Introducing subgroup-fair and instant-fair learning of LDS from multiple trajectories of varying lengths, using hierarchies of convexifications of non-commutative polynomial optimisation problems.
result Empirical results show both the beneficial impact of fairness considerations on statistical performance and encouraging effects of exploiting sparsity on run time.
Dan Lovallo and Daniel Kahneman must be commended for their clear identification of causes and cures to the planning fallacy in "Delusions of Success: How Optimism Undermines Executives' Decisions" (HBR July 2003). Their look at overoptimism, anchoring, competitor neglect, and the outside view in forecasting is highly …
This paper corrects climate model biases using a factor model approach.
problem Systematic biases in GCM outputs due to unobserved confounders.
method Factor model approach to learn latent confounders from historical data and apply them to enhance bias correction.
result Significant improvements in the accuracy of precipitation outputs.
STAS selects optimal spatio-temporal scales for bias correction in precipitation forecasts.
problem Limited prior data and fixed ST scale in existing BCoPs lead to biases in numerical weather predictions.
method End-to-end deep-learning BCoP model STAS with SFM/TFM to automatically adjust spatial and temporal scales.
result STAS outperforms 8 published BCoP methods on threat scores (TS).
The paper introduces a new σ-LSTM cell for volatility forecasting using stylized facts.
problem Lack of explainability and stylized knowledge in neural network volatility modeling.
method Introduces a new σ-LSTM cell with a stochastic processing layer, designed to incorporate stylized facts about volatility. result Shows good out-of-sample forecasting performance with the σ-LSTM cell. Deep generative models improve global precipitation forecasts.
problem Accurately forecasting extreme rainfall is challenging and costly.
method Trained a Conditional Generative Adversarial Network (CorrectorGAN) to correct and super-resolve global precipitation forecasts.
result CorrectorGAN produces high-resolution, bias-corrected forecasts in seconds.
FreST Loss decorrelates spatio-temporal dependencies in graph signals.
problem Complex spatio-temporal dependencies in graph-structured signals are not well captured by standard forecasting models.
method FreST Loss extends supervision to the joint spatio-temporal spectrum using Joint Fourier Transform (JFT).
result FreST Loss reduces estimation bias and improves forecasting accuracy on real-world datasets.
Paper proposes an efficient method for calibrating spatio-temporal forecasts.
problem Real-world spatio-temporal forecasting challenges like signal anomalies and distributional shifts.
method Learning with Calibration (ST-TTC) for real-time bias correction.
result ST-TTC improves spatio-temporal forecasting accuracy with reduced computational cost.
FreDF improves forecasting by learning in the frequency domain.
problem Label autocorrelation in future forecasts is often overlooked in time series modeling.
method FreDF learns to forecast in the frequency domain to mitigate label autocorrelation.
result FreDF significantly outperforms existing methods in forecasting accuracy.
Enhanced TSFMs improve time series forecasting accuracy and reliability.
problem Variance, bias, and uncertainty in TSFMs' predictions on real data.
method Statistical and ensemble techniques including bagging, stacking, residual modeling, and prediction intervals.
result Hybrid models consistently outperform standalone TSFMs across multiple horizons.
SOR-Mamba improves Mamba for robust time series forecasting by minimizing channel order bias.
problem Robust time series forecasting with Mamba's sequential order bias.
method SOR-Mamba incorporates regularization to minimize channel order discrepancy and introduces CCM for channel correlation preservation.
result SOR-Mamba enhances robustness to channel order and improves forecasting accuracy.
Reference class forecasting is a method to remove optimism bias and strategic misrepresentation in infrastructure projects and programmes. In 2012 the Hong Kong government's Development Bureau commissioned a feasibility study on reference class forecasting in Hong Kong - a first for the Asia-Pacific region. This study …
A hybrid loss framework improves time series forecasting by balancing global and component errors.
problem Current time series methods may prioritize less significant sub-series, leading to forecasting bias.
method Proposes a hybrid loss framework combining global and component losses, dynamically adjusting weights.
result Improves time series forecasting performance by 0.5-2% on average.
Proposes a transformer model with geostatistical inductive bias for spatio-temporal forecasting.
problem Combining probabilistic rigor of geostatistics with flexible deep learning representations.
method Spatially-informed transformer with learnable covariance kernel.
result Successfully recovers spatial decay parameters end-to-end via backpropagation.
M2VN forecasts financial volatility by fusing time series data with news embeddings.
problem Forecasting financial volatility with unstructured news data.
method Combines deep neural networks with open-source market features and news embeddings.
result M2VN outperforms existing models in financial volatility forecasting.
DYffusion improves diffusion models for spatiotemporal forecasting.
problem Challenges in generating stable and accurate forecasts for dynamic data.
method Leverages temporal dynamics in data, directly coupling it with diffusion steps.
result Improves computational efficiency and performs competitively on complex dynamics.
Enhanced LSTM predicts equity trends, outperforming traditional methods.
problem Nonstationary and nonlinear market regimes challenge trend forecasting.
method LSTM-based framework for forecasting equity trend differences.
result LSTM framework outperforms traditional methods in terms of overall PNL.
New models avoid lookahead bias by training on past data only.
problem Lookahead bias in language models.
method Chronologically consistent training on data before a knowledge-cutoff date.
result Elimination of lookahead bias in predictions.
HaKAN uses Hahn-KAN blocks to forecast multivariate time series.
problem Long-term time series forecasting challenges with high complexity and spectral bias.
method HaKAN integrates channel independence, patching, and a stack of Hahn-KAN blocks with residual connections. It uses Hahn polynomial-based learnable activation functions.
result HaKAN consistently outperforms state-of-the-art methods on various forecasting benchmarks.
The paper proposes a method to improve sales forecasts by selecting optimal reference classes.
problem Improving forecasts of sales growth exposed to behavioural bias.
method Finding optimal reference classes for each company based on specific predictors and matching forecast distributions to actual sales.
result The past operating margins are strong predictors for future sales distributions.
A new hierarchical forecasting method using machine learning improves forecast accuracy.
problem Improving forecast accuracy in hierarchical forecasting systems.
method Non-linear combination of base forecasts, focusing on both accuracy and coherence.
result The proposed method outperforms existing approaches, especially for diverse series.
New model predicts blood glucose in diabetics with improved accuracy.
problem Forecasting blood glucose in type 1 diabetics with high accuracy.
method Integrates machine learning with existing biomedical model to capture time-varying dynamics.
result Improved long-term forecasting of blood glucose up to 6 hours.
SPADE improves demand forecasting accuracy by 4.5% for post-promotion periods.
problem Overreacting to peak events in demand forecasting leads to biased forecasts.
method SPADE splits forecasting into two tasks: one for peak events and another for post-peak events, using masked convolution filters and a specialized Peak Attention module.
result Overall PPE improvement of 4.5%, 30% improvement for most affected forecasts after promotions and holidays, and 3.9% improvement in PE accuracy.
Project promoters, forecasters, and managers sometimes object to two things in measuring inaccuracy in travel demand forecasting: (1) using the forecast made at the time of making the decision to build as the basis for measuring inaccuracy and (2) using traffic during the first year of operations as the basis for measu…
The paper introduces a method for forecasting corporate sales growth using multiple reference variables.
problem Forecasting corporate sales growth with multiple reference variables.
method Reference class selection using rank-based algorithms and principal components analysis for data dimension reduction.
result Dimension reduced variables with past sales growth rates and operating margins perform well in forecasting.
With broad applications in various public services like aviation management and urban disaster warning, numerical precipitation prediction plays a crucial role in weather forecast. However, constrained by the limitation of observation and conventional meteorological models, the numerical precipitation predictions are o…
CNN improves medium-range temperature forecasts with limited resources.
problem Limited computational resources for high-resolution temperature forecasts.
method CNN post-processing with ensemble NWP models for bias correction and spatial downscaling.
result High-resolution (5-km) surface temperature forecasts with lead times up to 5.5 days.
REGAIN learns optimal auxiliary directions for forecast reconciliation.
problem Forecast reconciliation from fixed systems; identifying useful auxiliary directions.
method REGAIN learns normalized auxiliary directions, forecasts induced series, and selects directions by loss reduction.
result Gain-selected auxiliary directions improve forecast quality, especially for residual uncertainty.
This paper explores good practices for AI explainability in finance.
problem Complex financial models lack transparency and interpretability.
method Exploring good practices for deploying explainability in AI-based financial systems.
result Developing effective XAI tools for the financial industry.
MC-LSTM extends LSTM to conserve mass in neural networks.
problem Conservation laws in real-world systems.
method Extending LSTM's inductive bias to conserve mass.
result MC-LSTM sets new state-of-the-art for predicting peak flows.