Paper analyzes normal approximation for two-timescale stochastic algorithms, revealing interaction between fast and slow timescales.
problem Non-asymptotic bounds for accuracy of normal approximation in linear two-timescale stochastic approximation algorithms.
method Established bounds for normal approximation in terms of convex distance, focusing on last iterate and Polyak-Ruppert averaging.
result Normal approximation rate for the last iterate improves with increased timescale separation, while it decreases in the averaged setting.
This study improves convergence of two-timescale SA under Markovian noise in reinforcement learning.
problem Stability and convergence of two-timescale stochastic approximations under Markovian noise.
method Introduced a new control strategy for the fast timescale parameter.
result Established almost sure convergence of TDC with eligibility traces under off-policy learning with linear function approximation.
Paper separates financial time series into fast and slow components.
problem Multiscale behavior in financial time series data.
method Uses variance and tail stationarity criteria as generalized eigenvalue problems.
result Identifies slow and fast components in asset returns and prices.
Paper develops a fast method to find near-optimal power solutions.
problem Solving AC OPF on fast timescales for large networks.
method Leverages machine learning to map system loading to optimal generation values.
result Near-optimal and feasible solutions found on milliseconds timescales.
New deep learning model estimates scattering timescale of FRBs efficiently.
problem Estimating scattering timescale of fast radio bursts (FRBs) is a bottleneck.
method Multimodal Transformer Based Generic Mixture Density Network (MT-GMDN) that ingests dynamic spectrum and timeseries profile.
result Achieves 94% R2 on expected value of τ for measurable scattering. Authors improve accuracy analysis for portfolio optimization with multiple timescale factors.
problem Asymptotic accuracy of portfolio optimization approximations for general utility functions and two timescale factors.
method Construct sub- and super-solutions to fully nonlinear problem.
result Rigorous justification of accuracy for portfolio optimization with general utility functions and two timescale factors.
Hierarchical pretraining with slow-fast ODEs
problem Causal self-attention vs. slow-fast ODEs
method Instantiating fast-slow ODE formalism as a concrete neural network
result Equilibrium manifold x=φ(y) is exactly the master-equation (ME) stationary distribution HiPPO framework optimizes memory compression for sequential data.
problem Incremental representation of cumulative history in sequential data.
method Optimal polynomial projections for online function approximation.
result HiPPO-LegS achieves state-of-the-art accuracy on MNIST.
Synthesizes computational approaches to understand neural timescales.
problem Varying definitions and measurements of neural timescales across studies.
method Reviews data analysis methods, biophysical models, and machine learning models.
result Complements experimental studies with a holistic view of neural timescales.
Predicting unobserved bifurcations in time series with unsupervised parameter extraction.
problem Predicting system behavior with unknown parameters from time series data.
method Reservoir computing framework for unsupervised extraction of slowly varying system parameters.
result Model predicts unknown bifurcations not present in training data.
We study the distributions of event-time returns and clock-time returns at different microscopic timescales using ultra-high-frequency data extracted from the limit-order books of 23 stocks traded in the Chinese stock market in 2003. We find that the returns at the one-trade timescale obey the inverse cubic law. For la…
We introduce a framework to infer lead-lag networks between the states of elements of complex systems, determined at different timescales. As such networks encode the causal structure of a system, infering lead-lag networks for many pairs of timescales provides a global picture of the mutual influence between timescale…
The report evaluates heuristics for learning timescale graphical event models.
problem Lack of heuristics for determining hyper-parameters in timescale graphical event models.
method Proposed and evaluated different heuristics for hyper-parameter determination and refined an existing distance measure.
result Conclusions about the applicability of different heuristics on synthetic data.
We investigate the daily correlation present among market indices of stock exchanges located all over the world in the time period Jan 1996 - Jul 2009. We discover that the correlation among market indices presents both a fast and a slow dynamics. The slow dynamics reflects the development and consolidation of globaliz…
New analysis shows actor-critic method converges efficiently in practical settings.
problem Understanding finite-time convergence of single-timescale actor-critic methods.
method Investigated online single-timescale actor-critic algorithm with linear function approximation and Markovian sampling.
result Proved convergence to ε-approximate stationary point with sample complexity of O(ε^(-2)).
GraPhyR uses GNNs to optimize power grid reconfiguration in real-time.
problem Optimizing power grid reconfiguration for reliability and efficiency.
method Physics-informed Graph Neural Network (GNN) framework.
result GraPhyR learns to optimize DyR tasks efficiently.
In this paper we present a framework to analyze the asymptotic behavior of two timescale stochastic approximation algorithms including those with set-valued mean fields. This paper builds on the works of Borkar and Perkins & Leslie. The framework presented herein is more general as compared to the synchronous two times…
Method detects multi-timescale consumer spending patterns from receipts.
problem Understanding and managing consumer behavior in high-dimensional data.
method Non-negative tensor factorization (NTF) to extract multi-timescale expenditure patterns.
result Consumption patterns are characterized based on spending behavior over different timescales.
SPF uses a hierarchical approach to efficiently emulate climate changes.
problem Slow and unstable climate emulation for long horizons.
method Spatiotemporal Pyramid Flows (SPF) model data hierarchically across spatial and temporal scales.
result SPF outperforms flow matching baselines and pre-trained models on ClimateBench.
A machine learning framework predicts self-induced stochastic resonance in neurons.
problem Predicting coherent oscillations in slow-fast excitable systems driven by noise.
method Physics-informed machine learning with a Noise-Augmented State Predictor architecture and Kramers' escape theory constraints.
result Trained PINN accurately predicts spike-train coherence on noise intensity, excitability, and timescale separation.
Proves convergence of neural networks in a two-timescale regime.
problem Training dynamics of shallow neural networks.
method Two-timescale regime analysis of gradient flow.
result Gradient flow converges to global optimum in non-convex optimization.
Two single-timescale algorithms improve TD learning with nonlinear approximations.
problem Optimizing TD learning with nonlinear smooth function approximation.
method Proposes two single-timescale single-loop algorithms with momentum and variance reduction.
result Achieves O(ε−4) sample complexity for the first algorithm and O(ε−3) for the second. Paper analyzes Greedy-GQ for reinforcement learning with Markovian noise.
problem Analyzing Greedy-GQ for reinforcement learning with Markovian noise.
method Develops finite-sample analysis for Greedy-GQ with linear function approximation under Markovian noise.
result Provides theoretical justification for choosing stepsizes for faster convergence.
A new buffer system improves continual learning in RL agents by adapting to changing environments.
problem Improving RL agents' ability to learn from changing environments over time.
method Multi-timescale replay buffer combined with invariant risk minimization.
result The method shows improvement over baselines in continual learning settings.
Novel approach to learning models based on subjective timescales for better exploration and decision-making.
problem Learning models over multi-step timescales in environments with intermediate states.
method Developed a subjective-timescale model (STM) based on episodic memories, enabling systematic variation of temporal extent of predictions.
result STM produces more informative action-conditioned roll-outs, leading to better decision-making and exploration.
A new approach is presented to describe the change in the statistics of the log return distribution of financial data as a function of the timescale. To this purpose a measure is introduced, which quantifies the distance of a considered distribution to a reference distribution. The existence of a small timescale regime…
Single-timescale actor-critic finds globally optimal policy.
problem Finding globally optimal policy in reinforcement learning.
method Simultaneous actor and critic updates with linear or deep neural network approximations.
result Actor sequence converges to globally optimal policy at O(K−1/2) rate. Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …
Gradient descent-ascent converges to strict local minmax equilibria with a finite timescale separation.
problem Analyzing the convergence of gradient descent-ascent in non-convex, non-concave games with a finite timescale separation.
method Investigates the role of a finite timescale separation parameter τ on gradient descent-ascent in two-player zero-sum games, providing convergence rates and non-convergence results.
result Gradient descent-ascent converges to strict local minmax equilibria for a finite timescale separation parameter τ*.
New method disentangles sources of different timescales in planetary seismic data.
problem Unsupervised source separation of multi-scale seismic data from planetary missions.
method Wavelet scattering spectra for multi-scale clustering and variational autoencoder for source separation.
result Disentangles sources with different timescales in InSight mission seismic data.
Study Whittle index learning algorithms for restless bandits with constant stepsizes.
problem Optimizing decisions in restless multi-armed bandits with constant stepsizes.
method Developed Q-learning algorithms with constant stepsizes for index learning in restless bandits, extending to DQN and function approximations.
result The algorithms learn the Whittle index effectively.
Bayesian inference and superstatistics model financial volatility dynamics across different timescales.
problem Modeling correlated volatility in financial time series with heavy tails and long memory.
method Superstatistical dynamics, Bayesian Inference, Metropolis-Hasting sampling.
result The log-Normal model is reliable for short timescales, while inverse-Gamma is preferred for long timescales.
This paper analyzes the sample complexity of two timescale reinforcement learning algorithms.
problem Analyzing the sample complexity of two timescale reinforcement learning algorithms.
method Non-asymptotic analysis of linear and nonlinear TDC and Greedy-GQ algorithms under Markovian sampling with constant stepsize.
result The paper provides non-asymptotic convergence results for two timescale linear and nonlinear TDC and Greedy-GQ algorithms.
The paper analyzes dynamics of momentum in high dimensions with sparse updates.
problem Theoretical analysis of momentum dynamics in high-dimensional sparse settings.
method Theoretical analysis of two models: least squares with sparse inputs and logistic regression with a rare class.
result Characterization of high-dimensional limits of momentum dynamics and phase structure.
Single-timescale analysis improves convergence in multi-sequence stochastic approximation.
problem Finite-time convergence of nonlinear stochastic approximation with multiple coupled sequences.
method Smoothness property of fixed points and analysis of fine-grained single-timescale SA.
result Improved iteration complexity for achieving ε-accuracy in multi-sequence single-timescale SA.
Develops a new method for efficient stochastic bilevel optimization.
problem Stochastic bilevel optimization problems in machine learning applications.
method Single-Timescale stochAstic BiLevEl optimization (STABLE) method.
result Achieves the same order of sample complexity as stochastic gradient descent for single-level optimization.
A novel multi-resolution Gaussian process model for efficient time traversal.
problem Inference for long sequences with fast and slow transitions is difficult.
method A novel Gaussian process state-space architecture composed of multiple components, each trained on a different resolution.
result The combined model allows efficient inference for arbitrarily long sequences with complex dynamics.
A fast method for decentralized non-convex optimization over networks.
problem Decentralized non-convex optimization problems over a network of nodes.
method GT-SAGA, a randomized incremental gradient method that evaluates one component gradient per node per iteration.
result GT-SAGA achieves almost sure and mean-squared convergence to a first-order stationary point for general smooth non-convex problems.
COF-PAC converges with novel critic and learning method.
problem Convergent off-policy actor-critic with function approximation.
method Two-timescale approach with Gradient Emphasis Learning (GEM).
result First provably convergent COF-PAC with linear critics and nonlinear actor.
Neuromorphic Computing is a nascent research field in which models and devices are designed to process information by emulating biological neural systems. Thanks to their superior energy efficiency, analog neuromorphic systems are highly promising for embedded, wearable, and implantable systems. However, optimizing neu…
GRACE-C improves causal learning from time series data.
problem Misleading causal information due to mismatched timescales.
method Combines constraint programming with theoretical insights and prior information.
result Significantly faster and scalable causal learning for large datasets.
New methods improve neural connectivity analysis at submillisecond timescales.
problem Limitations of standard spike train analysis methods in terms of temporal resolution and scalability.
method Developed Monte Carlo and polynomial approximation methods for continuous-time neural spike train analysis.
result Superior accuracy and scalability compared to traditional binned GLMs, enabling precise connectivity inference.
Paper analyzes finite-time performance of SA in RL with Markovian noise.
problem Finite-time analysis of linear two-timescale stochastic approximation with Markovian noise.
method Finite-time analysis of linear two-timescale SA with Markovian noise, considering both transient and steady-state terms.
result No discrepancy in convergence rate between Markovian and martingale noise; transient term is o(1/kc) and steady-state term is O(1/k). Study on Adam-family methods for nonsmooth optimization with convergence guarantees.
problem Training nonsmooth neural networks with convergence guarantees.
method Two-timescale updating scheme and stochastic subgradient methods with gradient clipping.
result Convergence guarantees for various Adam-family methods in training nonsmooth neural networks.
Study analyzes a new algorithm for complex optimization problems.
problem Stochastic bilevel optimisation problems in continuous-time models.
method Continuous-time, two-timescale stochastic approximation algorithm.
result Obtained weak convergence rate using central limit theorem.
Study finds physical momentum portfolios in Indian stock market yield higher returns than benchmarks.
problem Determining abnormal returns for physical momentum portfolios in the Indian stock market.
method Constructed physical momentum portfolios for daily, weekly, monthly, and yearly timescales, evaluated historical returns and risk profiles.
result Daily time scale physical momentum portfolios showed the strongest reversal with a 16-fold profit.
In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in rewards in addition to maximizing a standard criterion. Variance related risk measures are among the most common risk-sensitive criteria in finance and operations research. However, optimizing many such c…
Molecular Dynamics (MD) simulation is widely used to analyze the properties of molecules and materials. Most practical applications, such as comparison with experimental measurements, designing drug molecules, or optimizing materials, rely on statistical quantities, which may be prohibitively expensive to compute from …