Synthesizes computational approaches to understand neural timescales.
problem Varying definitions and measurements of neural timescales across studies.
method Reviews data analysis methods, biophysical models, and machine learning models.
result Complements experimental studies with a holistic view of neural timescales.
Proves convergence of neural networks in a two-timescale regime.
problem Training dynamics of shallow neural networks.
method Two-timescale regime analysis of gradient flow.
result Gradient flow converges to global optimum in non-convex optimization.
New methods improve neural connectivity analysis at submillisecond timescales.
problem Limitations of standard spike train analysis methods in terms of temporal resolution and scalability.
method Developed Monte Carlo and polynomial approximation methods for continuous-time neural spike train analysis.
result Superior accuracy and scalability compared to traditional binned GLMs, enabling precise connectivity inference.
Single-timescale actor-critic finds globally optimal policy.
problem Finding globally optimal policy in reinforcement learning.
method Simultaneous actor and critic updates with linear or deep neural network approximations.
result Actor sequence converges to globally optimal policy at O(K−1/2) rate. Study on Adam-family methods for nonsmooth optimization with convergence guarantees.
problem Training nonsmooth neural networks with convergence guarantees.
method Two-timescale updating scheme and stochastic subgradient methods with gradient clipping.
result Convergence guarantees for various Adam-family methods in training nonsmooth neural networks.
Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …
HiPPO framework optimizes memory compression for sequential data.
problem Incremental representation of cumulative history in sequential data.
method Optimal polynomial projections for online function approximation.
result HiPPO-LegS achieves state-of-the-art accuracy on MNIST.
This work analyzes how neural networks learn representations in actor-critic algorithms.
problem Theoretical support for neural AC algorithms is limited to linear function approximations.
method Mean-field analysis of a two-timescale learning AC algorithm with overparameterized networks.
result Neural AC finds the globally optimal policy at a sublinear rate in the continuous-time and infinite-width limiting regime.
This study improves convergence of two-timescale SA under Markovian noise in reinforcement learning.
problem Stability and convergence of two-timescale stochastic approximations under Markovian noise.
method Introduced a new control strategy for the fast timescale parameter.
result Established almost sure convergence of TDC with eligibility traces under off-policy learning with linear function approximation.
Paper analyzes normal approximation for two-timescale stochastic algorithms, revealing interaction between fast and slow timescales.
problem Non-asymptotic bounds for accuracy of normal approximation in linear two-timescale stochastic approximation algorithms.
method Established bounds for normal approximation in terms of convex distance, focusing on last iterate and Polyak-Ruppert averaging.
result Normal approximation rate for the last iterate improves with increased timescale separation, while it decreases in the averaged setting.
We study the distributions of event-time returns and clock-time returns at different microscopic timescales using ultra-high-frequency data extracted from the limit-order books of 23 stocks traded in the Chinese stock market in 2003. We find that the returns at the one-trade timescale obey the inverse cubic law. For la…
Gradient descent on neural nets often operates at the Edge of Stability, where loss behavior is complex but loss decreases over time.
problem Understanding the optimization dynamics of neural networks at the Edge of Stability.
method Empirical demonstration of gradient descent behavior in neural network training.
result Gradient descent on neural networks typically occurs at the Edge of Stability, where loss behavior is non-monotonic but loss decreases over time.
We introduce a framework to infer lead-lag networks between the states of elements of complex systems, determined at different timescales. As such networks encode the causal structure of a system, infering lead-lag networks for many pairs of timescales provides a global picture of the mutual influence between timescale…
The report evaluates heuristics for learning timescale graphical event models.
problem Lack of heuristics for determining hyper-parameters in timescale graphical event models.
method Proposed and evaluated different heuristics for hyper-parameter determination and refined an existing distance measure.
result Conclusions about the applicability of different heuristics on synthetic data.
New analysis shows actor-critic method converges efficiently in practical settings.
problem Understanding finite-time convergence of single-timescale actor-critic methods.
method Investigated online single-timescale actor-critic algorithm with linear function approximation and Markovian sampling.
result Proved convergence to ε-approximate stationary point with sample complexity of O(ε^(-2)).
In this paper we present a framework to analyze the asymptotic behavior of two timescale stochastic approximation algorithms including those with set-valued mean fields. This paper builds on the works of Borkar and Perkins & Leslie. The framework presented herein is more general as compared to the synchronous two times…
Method detects multi-timescale consumer spending patterns from receipts.
problem Understanding and managing consumer behavior in high-dimensional data.
method Non-negative tensor factorization (NTF) to extract multi-timescale expenditure patterns.
result Consumption patterns are characterized based on spending behavior over different timescales.
Two single-timescale algorithms improve TD learning with nonlinear approximations.
problem Optimizing TD learning with nonlinear smooth function approximation.
method Proposes two single-timescale single-loop algorithms with momentum and variance reduction.
result Achieves O(ε−4) sample complexity for the first algorithm and O(ε−3) for the second. Paper analyzes Greedy-GQ for reinforcement learning with Markovian noise.
problem Analyzing Greedy-GQ for reinforcement learning with Markovian noise.
method Develops finite-sample analysis for Greedy-GQ with linear function approximation under Markovian noise.
result Provides theoretical justification for choosing stepsizes for faster convergence.
A new buffer system improves continual learning in RL agents by adapting to changing environments.
problem Improving RL agents' ability to learn from changing environments over time.
method Multi-timescale replay buffer combined with invariant risk minimization.
result The method shows improvement over baselines in continual learning settings.
Novel approach to learning models based on subjective timescales for better exploration and decision-making.
problem Learning models over multi-step timescales in environments with intermediate states.
method Developed a subjective-timescale model (STM) based on episodic memories, enabling systematic variation of temporal extent of predictions.
result STM produces more informative action-conditioned roll-outs, leading to better decision-making and exploration.
A new approach is presented to describe the change in the statistics of the log return distribution of financial data as a function of the timescale. To this purpose a measure is introduced, which quantifies the distance of a considered distribution to a reference distribution. The existence of a small timescale regime…
Generating musical audio directly with neural networks is notoriously difficult because it requires coherently modeling structure at many different timescales. Fortunately, most music is also highly structured and can be represented as discrete note events played on musical instruments. Herein, we show that by using no…
Gradient descent-ascent converges to strict local minmax equilibria with a finite timescale separation.
problem Analyzing the convergence of gradient descent-ascent in non-convex, non-concave games with a finite timescale separation.
method Investigates the role of a finite timescale separation parameter τ on gradient descent-ascent in two-player zero-sum games, providing convergence rates and non-convergence results.
result Gradient descent-ascent converges to strict local minmax equilibria for a finite timescale separation parameter τ*.
New method disentangles sources of different timescales in planetary seismic data.
problem Unsupervised source separation of multi-scale seismic data from planetary missions.
method Wavelet scattering spectra for multi-scale clustering and variational autoencoder for source separation.
result Disentangles sources with different timescales in InSight mission seismic data.
Study Whittle index learning algorithms for restless bandits with constant stepsizes.
problem Optimizing decisions in restless multi-armed bandits with constant stepsizes.
method Developed Q-learning algorithms with constant stepsizes for index learning in restless bandits, extending to DQN and function approximations.
result The algorithms learn the Whittle index effectively.
Bayesian inference and superstatistics model financial volatility dynamics across different timescales.
problem Modeling correlated volatility in financial time series with heavy tails and long memory.
method Superstatistical dynamics, Bayesian Inference, Metropolis-Hasting sampling.
result The log-Normal model is reliable for short timescales, while inverse-Gamma is preferred for long timescales.
This paper analyzes the sample complexity of two timescale reinforcement learning algorithms.
problem Analyzing the sample complexity of two timescale reinforcement learning algorithms.
method Non-asymptotic analysis of linear and nonlinear TDC and Greedy-GQ algorithms under Markovian sampling with constant stepsize.
result The paper provides non-asymptotic convergence results for two timescale linear and nonlinear TDC and Greedy-GQ algorithms.
The paper analyzes dynamics of momentum in high dimensions with sparse updates.
problem Theoretical analysis of momentum dynamics in high-dimensional sparse settings.
method Theoretical analysis of two models: least squares with sparse inputs and logistic regression with a rare class.
result Characterization of high-dimensional limits of momentum dynamics and phase structure.
Authors improve accuracy analysis for portfolio optimization with multiple timescale factors.
problem Asymptotic accuracy of portfolio optimization approximations for general utility functions and two timescale factors.
method Construct sub- and super-solutions to fully nonlinear problem.
result Rigorous justification of accuracy for portfolio optimization with general utility functions and two timescale factors.
Single-timescale analysis improves convergence in multi-sequence stochastic approximation.
problem Finite-time convergence of nonlinear stochastic approximation with multiple coupled sequences.
method Smoothness property of fixed points and analysis of fine-grained single-timescale SA.
result Improved iteration complexity for achieving ε-accuracy in multi-sequence single-timescale SA.
Develops a new method for efficient stochastic bilevel optimization.
problem Stochastic bilevel optimization problems in machine learning applications.
method Single-Timescale stochAstic BiLevEl optimization (STABLE) method.
result Achieves the same order of sample complexity as stochastic gradient descent for single-level optimization.
Paper separates financial time series into fast and slow components.
problem Multiscale behavior in financial time series data.
method Uses variance and tail stationarity criteria as generalized eigenvalue problems.
result Identifies slow and fast components in asset returns and prices.
Deep RL improves power control and scheduling for wireless multicast systems.
problem Scalable power control and scheduling for wireless multicast networks.
method Deep reinforcement learning with function approximation using a deep neural network.
result Deep RL can learn optimal power control policies for large systems.
COF-PAC converges with novel critic and learning method.
problem Convergent off-policy actor-critic with function approximation.
method Two-timescale approach with Gradient Emphasis Learning (GEM).
result First provably convergent COF-PAC with linear critics and nonlinear actor.
A machine learning framework predicts self-induced stochastic resonance in neurons.
problem Predicting coherent oscillations in slow-fast excitable systems driven by noise.
method Physics-informed machine learning with a Noise-Augmented State Predictor architecture and Kramers' escape theory constraints.
result Trained PINN accurately predicts spike-train coherence on noise intensity, excitability, and timescale separation.
GRACE-C improves causal learning from time series data.
problem Misleading causal information due to mismatched timescales.
method Combines constraint programming with theoretical insights and prior information.
result Significantly faster and scalable causal learning for large datasets.
Mathematical study of learning long-term integration in linear RNNs.
problem How do linear recurrent neural networks learn to integrate over long timescales?
method Analytical study of linear RNNs trained to integrate white noise and damped oscillatory filters.
result Learning dynamics are described by low-dimensional effective equations for outlier eigenvalues.
HHT feature generation enhances financial time series forecasting.
problem Forecasting nonstationary financial time series.
method CEEMD and HHT for decomposition, machine learning integration.
result HHT-enhanced models outperform traditional models in forecasting.
Paper analyzes finite-time performance of SA in RL with Markovian noise.
problem Finite-time analysis of linear two-timescale stochastic approximation with Markovian noise.
method Finite-time analysis of linear two-timescale SA with Markovian noise, considering both transient and steady-state terms.
result No discrepancy in convergence rate between Markovian and martingale noise; transient term is o(1/kc) and steady-state term is O(1/k). Neuromorphic Computing is a nascent research field in which models and devices are designed to process information by emulating biological neural systems. Thanks to their superior energy efficiency, analog neuromorphic systems are highly promising for embedded, wearable, and implantable systems. However, optimizing neu…
Study analyzes a new algorithm for complex optimization problems.
problem Stochastic bilevel optimisation problems in continuous-time models.
method Continuous-time, two-timescale stochastic approximation algorithm.
result Obtained weak convergence rate using central limit theorem.
Study finds physical momentum portfolios in Indian stock market yield higher returns than benchmarks.
problem Determining abnormal returns for physical momentum portfolios in the Indian stock market.
method Constructed physical momentum portfolios for daily, weekly, monthly, and yearly timescales, evaluated historical returns and risk profiles.
result Daily time scale physical momentum portfolios showed the strongest reversal with a 16-fold profit.
In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in rewards in addition to maximizing a standard criterion. Variance related risk measures are among the most common risk-sensitive criteria in finance and operations research. However, optimizing many such c…
Paper develops a fast method to find near-optimal power solutions.
problem Solving AC OPF on fast timescales for large networks.
method Leverages machine learning to map system loading to optimal generation values.
result Near-optimal and feasible solutions found on milliseconds timescales.
Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex task…
GraPhyR uses GNNs to optimize power grid reconfiguration in real-time.
problem Optimizing power grid reconfiguration for reliability and efficiency.
method Physics-informed Graph Neural Network (GNN) framework.
result GraPhyR learns to optimize DyR tasks efficiently.
Recurrent neural networks (RNNs) for reinforcement learning (RL) have shown distinct advantages, e.g., solving memory-dependent tasks and meta-learning. However, little effort has been spent on improving RNN architectures and on understanding the underlying neural mechanisms for performance gain. In this paper, we prop…