Improved averaging method for noisy observations converges strongly.
problem Noisy observations from random dynamical systems require stable estimates.
method Introduced p-EMA, a modified exponential moving average with subharmonic weight decay. result Stochastic convergence guarantees for p-EMA under mild assumptions. Paper analyzes how EMA improves SGD in linear regression.
problem Understanding the effectiveness of EMA in training deep learning models.
method Established risk bound for online SGD with EMA in linear regression.
result SGD with EMA has smaller variance error and exponentially decaying bias error.
This work provides a scaling rule for model EMA optimization across batch sizes.
problem Training dynamics and performance differences across batch sizes when using model EMA.
method Developed a scaling rule for model EMA optimization, demonstrating its validity across various architectures and data modalities.
result Enabled SSL methods like BYOL to train at larger batch sizes without performance degradation.
Optimal trend-following strategy uses simple EMA, avoiding complex cherry-picked signals.
problem Cherry-picking signals for trend-following strategies.
method Simple EMA for trend capture, avoiding complex indicators.
result Simple EMA is optimal for capturing trend, complex indicators are risky.
BEMA reduces bias in EMA, leading to faster convergence and better performance.
problem Stochasticity in language model fine-tuning destabilizes training.
method Bias-Corrected Exponential Moving Average (BEMA) augmentation of EMA.
result BEMA leads to significantly improved convergence rates and final performance.
AdEMAMix optimizer improves model performance and convergence speed.
problem Suboptimal use of single EMA in momentum-based optimizers.
method Proposes AdEMAMix, a modified Adam optimizer with a mixture of two EMAs.
result Empirically shows gradients can remain relevant for tens of thousands of steps, leading to faster convergence and lower minima.
Paper reviews and compares NMF, PLSA, LBA, EMA, and LCA models.
problem Identifiability of latent models.
method Comparison and proof of identifiability.
result Identifiability of LBA, EMA, LCA, PLSA is unique if and only if NMF is unique.
The paper optimizes portfolios using MACD signals derived from price history.
problem Optimizing risky asset portfolios with latent mean-reverting and momentum factors.
method Derives optimal strategies based on MACD signals from EMA processes.
result Establishes admissibility and verification of optimal strategies.
Study assesses environmental management accounting practices in Bangladesh.
problem Low environmental management accounting practices in Bangladeshi manufacturing companies.
method Developed a compliance checklist and evaluated practices using binary scoring.
result Environmental management accounting practices are poor in Bangladeshi manufacturing companies.
We examine two different techniques for parameter averaging in GAN training. Moving Average (MA) computes the time-average of parameters, whereas Exponential Moving Average (EMA) computes an exponentially discounted sum. Whilst MA is known to lead to convergence in bilinear settings, we provide the -- to our knowledge …
Neural SDEs model suicide risk with compact state space constraints.
problem Modeling suicide risk with irregular, noisy, and partially observed data.
method Developed neural SDEs confined to compact state spaces, addressing domain constraints and numerical stability.
result Improved forecasts and optimization dynamics over standard models on EMA datasets.
Optimizes LightGBM for stock market forecasting with novel feature engineering and transformation methods.
problem Accurately forecasting stock market fluctuations to mitigate risks.
method Feature engineering and transformation methods for LightGBM optimization.
result Log Returns, Returns and EMA Difference Ratio are the most effective target variable transformations.
Omega method mitigates noise in stochastic game optimization.
problem Noise sensitivity and convergence issues in stochastic game optimization.
method Omega method incorporates EMA of historic gradients in its update rule.
result Omega method outperforms optimistic gradient method in stochastic games.
Interactive EMA combines multiple explainability methods to improve model understanding.
problem Isolated explanations of machine learning models lead to misunderstandings and wrong reasoning.
method Interactive EMA (IEMA) combines multiple explainability methods sequentially.
result Interactive EMA increases the performance and confidence of human decision making.
This article presents a novel method for predicting suicidal ideation from Electronic Health Records (EHR) and Ecological Momentary Assessment (EMA) data using deep sequential models. Both EHR longitudinal data and EMA question forms are defined by asynchronous, variable length, randomly-sampled data sequences. In our …
Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
Machine learning models outperform traditional technical analysis in Bitcoin trading.
problem Maximizing profits in the Bitcoin market using trading signals.
method Comparison of machine learning models (LightGBM, LSTM) and technical analysis strategies (EMA, MACD+ADX).
result LSTM model achieved a 65.23% cumulative return over a year, significantly outperforming other strategies.
A new method Expectigrad improves on Adam and RMSProp by reducing divergence and improving performance.
problem Improving the convergence properties of adaptive gradient methods like Adam and RMSProp.
method Adjusts stepsizes using a per-component unweighted mean of all historical gradients and a bias-corrected momentum term.
result Cannot diverge on convex optimization problems that cause Adam to diverge.
In this note we discuss the mathematical tools to define trend indicators which are used to describe market trends. We explain the relation between averages and moving averages on the one hand and the so called exponential moving average (EMA) on the other hand. We present a lot of examples and give the definition of t…
Recently, consistency-based methods have achieved state-of-the-art results in semi-supervised learning (SSL). These methods always involve two roles, an explicit or implicit teacher model and a student model, and penalize predictions under different perturbations by a consistency constraint. However, the weights of the…
Improved diffusion models for image synthesis with better training dynamics.
problem Uneven and ineffective training in diffusion models.
method Redesigned network layers to preserve activation, weight, and update magnitudes.
result Significantly better networks at equal computational complexity, improving FID to 1.81.
In this paper, we propose a minimal model beyond geometric Brownian motion that aims to describe price actions with market inefficiency. From simple financial theory considerations, we arrive at a simple two-variable hidden Markovian time series model, with one of the variable entirely unobserved. Then, we analyze the …
Improves deep learning models by blending gradients from training loss and auxiliary objective.
problem Minimizing a single training loss while encouraging desirable model properties.
method Solves a bilevel optimization problem by combining training loss gradients and orthogonal projections of auxiliary gradients.
result Bloop method leads to better performance than other gradient surgery methods without EMA.
This paper analyzes the skewness of momentum trading strategies.
problem Understanding the skewness of momentum trading strategies.
method Examined linear and nonlinear momentum trading strategies, focusing on skewness.
result Skewness is generally positive and has a term structure.
Study predicts customer data sharing in Open Banking and explains key factors.
problem Predicting and explaining customer data sharing in Open Banking environments.
method Hybrid data balancing strategy with ADASYN and NEARMISS, XGBoost models, SHAP, CART.
result 91.39% accuracy for inflow and 91.53% for outflow predictions, revealing influential features.
Study compares volatility models for Bitcoin, finds GARCH and EGARCH outperform.
problem Evaluating which volatility models best predict Bitcoin spot and option prices.
method Used HIST, EMA ARCH, GARCH, and EGARCH models on Bitcoin spot price series.
result GARCH and EGARCH models outperform other models in both in-sample and out-of-sample forecasts.
Infinite hierarchical contrastive clustering identifies personal environments linked to health outcomes.
problem Identifying meaningful relationships between environmental features and health outcomes on an individual level.
method Contrastive clustering framework with stick-breaking prior and participant-specific prediction loss.
result Model effectively identifies distinct personal environments and groups them into meaningful types linked to health outcomes.
Proposes φ-balancing for more balanced expert utilization in MoE models.
problem Balanced expert utilization in MoE models to avoid bias.
method Directly targets population-level balance by minimizing a convex potential function.
result Consistently outperforms prior methods in stability and effectiveness.
mcanalysis quantifies menstrual cycle effects in health data.
problem Lack of standardised statistical methods for menstrual cycle research.
method Fourier-basis generalised additive model (GAM) pipeline.
result Nine out of 15 health outcomes showed significant association with menstrual cycle.
LassoFlexNet improves deep learning performance on tabular data.
problem Deep learning underperforms tree-based models on tabular data.
method Incorporates five inductive biases and uses Tied Group Lasso for variable selection.
result LassoFlexNet matches or outperforms leading tree-based models on 52 datasets.
High fidelity behavior prediction of intelligent agents is critical in many applications. However, the prediction model trained on the training set may not generalize to the testing set due to domain shift and time variance. The challenge motivates the adoption of online adaptation algorithms to update prediction model…
Hybrid AI system combines technical, sentiment analysis for adaptive equity trading.
problem Traditional trading strategies fail during high volatility and regime shifts.
method Combines trend-following, mean-reversion, sentiment analysis, machine learning, and market regime filtering.
result Hybrid model achieved 135.49% return on investment over 24 months.
A TTA framework improves forecasting accuracy in non-stationary time series.
problem Improving forecasting accuracy in non-stationary time series.
method Normalization-based test-time adaptation for causal timeseries forecasting and direction classification.
result Normalization-based TTA improves forecasting error in synthetic gradual drift and can even hurt in aggressive norm-only adaptation in financial markets.
This paper proposes an approach to detect emotion from human speech employing majority voting technique over several machine learning techniques. The contribution of this work is in two folds: firstly it selects those features of speech which is most promising for classification and secondly it uses the majority voting…
Adaptive t-distribution estimates nonstationary time series using moving moments.
problem Nonstationary time series with varying dependence structure.
method Moving estimator optimizing a weighted log-likelihood, using exponential moving averages for moments.
result Evolution of ν parameter in Student's t-distribution, capturing tail behavior and extreme events.
Adaptive estimation of alpha-Stable distribution and Hurst exponent for nonstationary time series.
problem Nonstationary time series require adaptive models to avoid bias.
method Moving estimator with exponentially weakening weights of old values, optimized using EMA of absolute central moments.
result Continuous adaptive estimation of alpha-Stable distribution and Hurst exponent for market stability evaluation.
Improving optimization for iterate-averaged language models
problem How to optimize the averaged model returned by Language Model pipelines
method Formulating optimizer design as an optimal-control problem
result Proven convergence rate and strict improvement in squared error
PACE optimizes training for averaged language models, improving performance.
problem How to optimize training for averaged language model iterates.
method Formulated as an optimal-control problem, solved for minimizing error of the average with a penalty on intervention size.
result PACE improves the limiting squared error of the iterate-average estimator by an arbitrarily large factor on some instances.
AdaBelief optimizes deep learning models with faster convergence and better stability.
problem Combining fast convergence and stability in deep learning models.
method Adapts stepsize based on the belief in observed gradients using exponential moving average (EMA) of noisy gradients.
result AdaBelief outperforms other methods in image classification and GAN training, achieving comparable accuracy to SGD on ImageNet.
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
Phone sensors could be useful in assessing changes in gait that occur with alcohol consumption. This study determined (1) feasibility of collecting gait-related data during drinking occasions in the natural environment, and (2) how gait-related features measured by phone sensors relate to estimated blood alcohol concen…
SiamJEPA uses Siamese student encoders to improve JEPA-based representation learning.
problem Improving self-supervised representation learning in JEPA models.
method Proposes SiamJEPA with masked Siamese student encoders and EMA teacher network.
result Siamese student encoders improve representation separability and learning speed.
Unified analysis of finite weight averaging methods in deep learning.
problem Theoretical explanation of finite weight averaging methods in deep learning.
method Generalized SGD and LAWA, establishing convergence and generalization bounds.
result FWA achieves faster convergence and better generalization compared to SGD.
New method improves Bayesian optimization by considering budget and prior quality.
problem Bayesian optimization lacks conditional treatment effects for specific contexts.
method Portable Regime Score (PRS) and RegimePlanner approach.
result Changing budget reverses optimization outcomes, and PRS predicts winners.
A dealer manages quotes and rejection rules to control slippage risk in FX markets.
problem Managing inventory risk and latency risk in OTC FX market making.
method Dynamic programming and adiabatic-quadratic approximation to optimize quotes and rejection rules.
result Developed a method to optimize quotes and rejection rules for managing slippage risk.
This study optimizes trading strategy parameters using walk-forward techniques and finds robust performance.
problem Optimizing trading strategy performance through parameter optimization.
method Walk-forward optimization with varying window lengths, tested on Bitcoin, Binance Coin, and Ethereum.
result The strategy outperforms Buy-and-Hold with lower drawdown and higher Information Ratio.
This chapter is an attempt to present a mathematical theory of compound fractional Poisson processes. The chapter begins with the characterization of a well-known Lévy process: The compound Poisson process. The semi-Markov extension of the compound Poisson process naturally leads to the compound fractional Poisson proc…
We introduce a class of hybrid marked point processes, which encompasses and extends continuous-time Markov chains and Hawkes processes. While this flexible class amalgamates such existing processes, it also contains novel processes with complex dynamics. These processes are defined implicitly via their intensity and a…