Study forecasts Turkish residential NGD using JITL-GPR, reducing errors.
problem Accurately predict future monthly NGD for Turkey's import contracts.
method Used historical NG consumption data, applied various time series models, and introduced JITL-GPR.
result JITL-GPR reduces forecast errors compared to traditional methods.
New framework for tracking varying bounds in time series forecasting.
problem Forecasting bounded time series with varying bounds.
method Extended log-likelihood estimation, online maximum likelihood estimation, Normalized Gradient Descent (NGD) for quasiconvex optimization.
result Derive an Online Normalized Gradient Descent algorithm for online bound tracking.
NGD method uses decentralized network for federated learning.
problem Privacy and reliability in decentralized federated learning.
method NGD method executes gradient descent on a communication-based network.
result NGD estimator can be as efficient as global estimator under certain conditions.
This study explains why approximate NGD works well in wide neural networks.
problem Understanding why NGD with approximate Fisher information converges fast in wide neural networks.
method Analyzing asymptotic training dynamics in function space via the neural tangent kernel.
result NGD with approximate Fisher information achieves the same fast convergence as exact NGD under specific conditions.
Recent work has argued that stochastic gradient descent can approximate the Bayesian uncertainty in model parameters near local minima. In this work we develop a similar correspondence for minibatch natural gradient descent (NGD). We prove that for sufficiently small learning rates, if the model predictions on the trai…
SP-NGD improves deep learning models' generalization with large mini-batch sizes.
problem Worse generalization performance with large mini-batch sizes in deep learning.
method SP-NGD, a natural gradient descent approach for large-scale deep learning.
result SP-NGD achieves similar generalization performance to first-order methods with accelerated convergence and negligible overhead.
Proves FR-NGD optimally approximates evolutionary dynamics and continuous Bayesian inference.
problem Optimizing continuous time replicator equations and continuous Bayesian inference.
method Fisher-Rao natural gradient descent (FR-NGD) and its correspondence with evolutionary dynamics.
result FR-NGD optimally approximates continuous time replicator equations and continuous Bayesian inference.
This work analyzes how preconditioning affects generalization in machine learning models.
problem The impact of preconditioning on the generalization of machine learning models.
method An asymptotic bias-variance decomposition of the generalization error for ridgeless regression under various preconditioners.
result The optimal preconditioner depends on label noise, model specification, and signal alignment, with NGD potentially better under certain conditions.
NGD models have higher effective dimension than SGD models.
problem Measuring model complexity accurately.
method Comparison of NGD and SGD models using effective dimension measures.
result NGD models have a higher effective dimension than SGD models.
We develop a more efficient NGD method for structured parameters.
problem Computational challenges in NGD for structured parameter spaces.
method Local-parameter coordinates to simplify Fisher-matrix computations.
result New structured second-order algorithms and learning methods.
This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there ar…
A commonly used heuristic in non-convex optimization is Normalized Gradient Descent (NGD) - a variant of gradient descent in which only the direction of the gradient is taken into account and its magnitude ignored. We analyze this heuristic and show that with carefully chosen parameters and noise injection, this method…
Paper proposes ZO-NGD for more efficient black-box attacks.
problem Vulnerability of state-of-the-art DNNs to adversarial attacks.
method Zeroth-order natural gradient descent (ZO-NGD) for black-box attacks.
result ZO-NGD achieves significantly lower model query complexities.
A new optimizer for deep learning improves accuracy and reduces training time.
problem Training deep neural networks for classification tasks.
method Hybrid Newton/Gradient Descent (NGD) method exploiting convexity of cross-entropy loss.
result Improves validation error and provides qualitative differences in hidden layer basis functions.
Optimal reconciliation keeps some forecasts unchanged in hierarchical forecasting.
problem Keeping some forecasts unchanged in hierarchical forecasting.
method Formulates a method to keep some forecasts unchanged in a hierarchical forecasting system.
result Preserves unbiasedness and non-negativity of forecasts.
Short-term load forecasting is a critical element of power systems energy management systems. In recent years, probabilistic load forecasting (PLF) has gained increased attention for its ability to provide uncertainty information that helps to improve the reliability and economics of system operation performances. This…
Combining forecasts of 16 ED causes improves accuracy and stability.
problem Forecasting accuracy and stability for ED admissions is poor due to model uncertainty and limited data.
method High-dimensional forecast combinations of 16 cause-specific ED forecasts using extensive covariates.
result Forecast combinations yield forecast accuracies of 3.81%-23.54% across causes, outperforming individual models in 50% of scenarios.
Conditional forecasts improve performative prediction accuracy.
problem Performative predictions undermine standard forecasting methods.
method Condition forecasts on covariates to make them forecast-invariant.
result Proper scoring rules fail under conditioning, but two solutions are identified.
Study improves seasonal forecasts using deep learning.
problem Challenges in generating large forecast ensembles and limited observations for verification.
method Developed a probabilistic deep neural network model.
result Demonstrated favorable skill compared to state-of-the-art dynamical forecast systems.
Two new methods improve forecasting of functional time series data.
problem Forecasting of functional time-dependent data.
method Functional Singular Spectrum Analysis (FSFA) based forecasting methods.
result Our methods outperform existing algorithms for periodic stochastic processes.
Deep learning improves time series forecasting, outperforming other methods.
problem Improving time series forecasting accuracy.
method Deep learning models for time series prediction.
result Deep learning models consistently outperform other methods in forecasting competitions.
Nowadays, with the unprecedented penetration of renewable distributed energy resources (DERs), the necessity of an efficient energy forecasting model is more demanding than before. Generally, forecasting models are trained using observed weather data while the trained models are applied for energy forecasting using for…
MPANF improves naive forecast by incorporating directional information.
problem Challenging to surpass naive forecast in financial time series.
method Combines naive forecast with movement prediction and accuracy.
result MPANF generally outperforms common benchmarks.
Simplifies forecast combination by using diversity of out-of-sample forecasts.
problem Estimating optimal weights for forecast combinations is challenging.
method Use out-of-sample forecasts to extract features and calculate weights for forecast combination.
result Achieves superior forecasting performance in point forecasts and prediction intervals.
Develops forecast hedging for improved calibration of forecasts.
problem Improving the accuracy of forecasted frequencies.
method Combines deterministic and stochastic approaches to forecast hedging.
result Ensures expected track record can only improve.
Microdata improves inflation forecasts after major shocks, study finds.
problem Forecasting inflation in a non-stationary environment with microeconomic data.
method Developed a scan test to detect periods of micro forecast outperformance, combined with adaptive machine learning.
result Micro forecasts improve inflation predictions after major shocks, especially after 2020.
Improved forecast accuracy for Knitwear by 20% using adaptive AI/ML model.
problem Low accuracy in demand forecasts for Knitwear product category.
method Dynamic selection of the best algorithm from an algorithm rack based on performance and context.
result Increased forecast accuracy from 60% to 80% for Knitwear.
Given a nonlinear model, a probabilistic forecast may be obtained by Monte Carlo simulations. At a given forecast horizon, Monte Carlo simulations yield sets of discrete forecasts, which can be converted to density forecasts. The resulting density forecasts will inevitably be downgraded by model mis-specification. In o…
Paper proposes a new method for selecting the best hierarchical forecasting approach.
problem Selecting the best method for reconciling base forecasts in hierarchical time series.
method Conditional hierarchical forecasting using machine learning and time series features.
result Conditional hierarchical forecasting leads to significantly more accurate forecasts, especially at lower levels.
Proposes a neural network for accurate and reconciled hierarchical time series forecasting.
problem Forecasting and reconciling hierarchical time series data.
method Uses a deep neural network to directly produce accurate and reconciled forecasts, minimizing a customized loss function at training time.
result Our approach outperforms state-of-the-art competitors in hierarchical forecasting on real-world datasets.
The key contribution of this paper is to propose a classification into two dimensions of the load forecasting studies to decide which forecasting tools to use in which case. This classification aims to provide a synthetic view of the relevant forecasting techniques and methodologies by forecasting problem. In addition,…
This paper reviews forecast combinations over 50 years, highlighting their evolution and utility.
problem Improving forecast accuracy through combining multiple forecasts.
method Evolution of forecast combination methods, from simple to sophisticated.
result Forecast combinations have become a mainstream approach in forecasting.
This paper improves forecast stability without sacrificing accuracy using dynamic loss weighting.
problem Rolling origin forecast instability in time series forecasting.
method Dynamic loss weighting algorithms applied to the N-BEATS model.
result Dynamic loss weighting can further improve forecast stability without compromising accuracy.
A new framework detects forecast model inadequacies using online monitoring of forecast errors.
problem Inaccurate forecasts lead to poor decision-making in complex models.
method Sequential changepoint techniques on forecast errors for real-time identification of process changes.
result The framework identifies shifts in forecast errors faster than in the original models, indicating process changes.
Archive of 20 time series datasets for forecasting evaluation.
problem Lack of comprehensive time series forecasting datasets.
method Compilation and characterisation of 20 datasets from various domains.
result Characterisation and performance evaluation of datasets.
As renewable distributed energy resources (DERs) penetrate the power grid at an accelerating speed, it is essential for operators to have accurate solar photovoltaic (PV) energy forecasting for efficient operations and planning. Generally, observed weather data are applied in the solar PV generation forecasting model w…
Develops a method to ensure accurate quantile forecasts across multiple levels.
problem Ensuring accurate quantile forecasts at multiple levels, even under distribution shifts.
method Multi-level quantile tracker (MultiQT) wraps around any forecaster to produce calibrated forecasts.
result Guaranteed calibration of quantile forecasts at multiple levels, even against adversarial shifts.
A new hierarchical forecasting method using machine learning improves forecast accuracy.
problem Improving forecast accuracy in hierarchical forecasting systems.
method Non-linear combination of base forecasts, focusing on both accuracy and coherence.
result The proposed method outperforms existing approaches, especially for diverse series.
New method evaluates language model forecasters by checking consistency of predictions.
problem Evaluating the performance of language model forecasters is difficult due to lack of ground truth.
method Developed a consistency check framework based on arbitrage to evaluate forecasters.
result Consistency metrics correlate with ground truth performance of LLM forecasters.
The paper proposes a method to improve forecast combination accuracy using portfolio theory.
problem Improving forecast accuracy by combining multiple forecasts.
method Generates forecast combinations using a portfolio analogy, allowing negative weights for hedging.
result Demonstrates improved performance in weighted random forest forecasts.
EasyTime simplifies time series forecasting for researchers and practitioners.
problem Ease of use and accuracy in time series forecasting.
method One-click evaluation, automated ensemble, natural language Q&A.
result Superior forecasting accuracy compared to individual methods.
New method predicts wave height exceedance probabilities.
problem Forecasting significant wave height to prevent coastal disasters.
method Point forecasting approach using cumulative distribution function.
result Proposed method outperforms existing approaches.
Deep learning improves PV generation quantile forecasting.
problem Accurate probabilistic forecasting of PV generation.
method Developed an encoder-decoder deep learning model for multi-output quantile PV forecasting.
result The model improves forecast quality and computational efficiency.
Framework improves ETF volatility forecasting by adapting to market conditions.
problem Challenges in volatility forecasting due to shifting market conditions and varying model performance.
method Risk-sensitive specialist routing using online risk-sensitive evaluation and state-dependent gating.
result Reduces forecast loss by 24% and underprediction loss by 22% compared to rolling-best baseline.
Study compares forecasting methods for logistics time series.
problem Improving forecasting accuracy in logistics.
method Compared statistical and machine learning methods on simulated time series.
result Statistical methods outperformed machine learning in one-step forecasts.
New method forecasts values and timing in irregular time series.
problem Forecasting values and timing in sparse, irregularly sampled multivariate time series.
method Proposes a novel approach for forecasting values and timing in irregular time series.
result Successfully forecasts values and timing in irregular time series.
Forecast reconciliation improves portfolio risk forecasts, especially when true covariance is known.
problem Improving portfolio risk forecasts using multivariate GARCH models.
method Combining univariate and multivariate forecasts with forecast reconciliation techniques.
result Forecast reconciliation improves over standard multivariate approaches, especially when true covariance is known.
Less frequent retraining improves forecast accuracy in retail demand forecasting.
problem Balancing forecast accuracy and computational efficiency in global models.
method Analysis of ten machine learning and deep learning models across two large retail datasets with various retraining scenarios.
result Less frequent retraining strategies maintain forecast accuracy while reducing computational costs.