New magnetic memory effects found in gravitational waves and memory.
problem Understanding new effects in gravitational waves and memory.
method Analyzing asymptotically-flat spacetimes with slow decay.
result Diverging magnetic memory sourced by curvature and neutrino cloud.
Improved RNNs reduce memory decay and enhance language tasks.
problem Memory decay in RNNs affects performance in sequence prediction tasks.
method Introduced trainable scaling factors and a dependent bidirectional RNN to mitigate memory decay and improve performance.
result The proposed ELSTM and DBRNN models achieved up to 30% improvement in LAS compared to LSTM and GRU in dependency parsing.
Paper proposes efficient inference for hidden Markov models with memory decay.
problem Challenges in scalability due to dependencies in hidden Markov model observation data.
method Utilizes memory decay to carry out forward and backward probabilities with subsequences, enabling efficient inference over long sequences.
result Developed an efficient algorithm to numerically estimate the gap of top Lyapunov exponents, which determines the length of subsequences.
Paper questions RNN and LSTM's long-term memory and introduces a new definition.
problem Whether RNN and LSTM have long-term memory.
method Introduced a new definition of long-term memory and modified RNN and LSTM to test it.
result RNN and LSTM do not meet the new definition of long-term memory.
A technique identifies memoryless algorithms approximating memory-dependent optimization methods.
problem Understanding how memory in optimization algorithms affects loss and generalization.
method Introducing a general technique to replace past iterates with the current one and adding a correction term.
result Lion does not have the same implicit anti-regularization as AdamW, explaining its better generalization performance.
Improved online FDR control with decaying memory.
problem Online multiple testing with temporal data.
method Generalized alpha-investing algorithms (GAI) with decaying memory FDR (mem-FDR).
result New algorithms reduce false discovery rate and improve power.
The study analyzes prediction errors in systems with memory kernels, providing bounds and stability results.
problem Prediction errors in stochastic dynamical systems with memory kernels.
method Analysis of generalized Langevin equations (GLEs) with Volterra equations, integrating synchronized noise coupling and weighted norms.
result Prediction discrepancies decay at a rate determined by the memory kernel's decay, quantitatively bounded by kernel estimation errors.
New memory effect discovered in gravitational wave behavior.
problem Understanding gravitational wave behavior in spacetimes with angular momentum.
method Mathematical analysis of Minkowski spacetime and Kerr black holes.
result Angular momentum memory effect observed at future null infinity.
The paper explores how the probability of default estimation changes with temporal correlation decay.
problem Difficulty in estimating the probability of default due to correlations between borrowers.
method Hierarchical Bayesian estimation using beta binomial distribution with temporal correlation.
result A phase transition occurs in the PD estimator, with convergence depending on the power decay index of temporal correlation.
New method SF-AdamW trains large models without decay phases or memory overhead.
problem Inadequate fixed compute budgets for large-scale training.
method Schedule-Free (SF) method revisited and refined.
result SF-AdamW effectively navigates loss landscape without decay phases or memory overhead.
A new MLSS model approximates high-order Markov chains efficiently.
problem Efficiently approximating high-order Markov chains.
method Decaying mixture over past states, simple sampling algorithm.
result Approximates high-order Markov chains with fixed time and memory costs.
NovoGrad improves deep learning training with adaptive moments and layer-wise normalization.
problem Training deep neural networks efficiently and effectively.
method Layer-wise adaptive moments with gradient normalization and decoupled weight decay.
result NovoGrad outperforms well-tuned SGD with momentum and Adam/AdamW in various tasks.
Simplicial persistence measures financial market dynamics, revealing long-term structure evolution.
problem Understanding the long-term structure evolution of financial markets.
method Simplicial persistence, null models, TMFG filtering, thresholding, generative process analysis.
result More liquid markets exhibit slower persistence decay, suggesting higher fragility to systemic shocks.
We propose a stochastic process driven by memory effect with novel distributions including both exponential and leptokurtic heavy-tailed distributions. A class of distribution is analytically derived from the continuum limit of the discrete binary process with the renormalized auto-correlation and the closed form momen…
Quantum systems with scrambling improve temporal information processing, but scaling requires exponential overhead.
problem Scalability and memory retention of quantum reservoirs in temporal information processing.
method Examined a quantum reservoir processing framework with scrambling reservoirs modeled by high-order unitary designs, analyzed in noiseless and noisy settings.
result Memory retention improves exponentially with reservoir size but worsens with reservoir iterations, requiring exponential shot overhead for scaling.
Financial market dynamics is rigorously studied via the exact generalized Langevin equation. Assuming market Brownian self-similarity, the market return rate memory and autocorrelation functions are derived, which exhibit an oscillatory-decaying behavior with a long-time tail, similar to empirical observations. Individ…
Adafactor optimizes neural networks with less memory and similar performance.
problem Memory constraints in adaptive optimization methods.
method Adafactor uses row and column sums of moving averages to estimate per-parameter second moments, reducing memory usage.
result Adafactor achieves similar performance to Adam with minimal auxiliary storage.
It is generally accepted that many time series of practical interest exhibit strong dependence, i.e., long memory. For such series, the sample autocorrelations decay slowly and log-log periodogram plots indicate a straight-line relationship. This necessitates a class of models for describing such behavior. A popular cl…
Lion optimizer performs well in training AI models with memory efficiency.
problem Lion optimizer's theoretical basis is unclear.
method Continuous-time and discrete-time analysis of Lion updates with a new Lyapunov function.
result Lion is a novel and principled approach for constrained optimization.
We focus on emergence of the power-law cross-correlations from processes with both short and long term memory properties. In the case of correlated error-terms, the power-law decay of the cross-correlation function comes automatically with the characteristics of separate processes. Bivariate Hurst exponent is then equa…
For the London Stock Exchange we demonstrate that the signs of orders obey a long-memory process. The autocorrelation function decays roughly as τ−α with α≈0.6, corresponding to a Hurst exponent H≈0.7. This implies that the signs of future orders are quite predictable from the signs of past orde…
Paper proposes JEDI teaching framework for adaptive crowd teaching.
problem Adaptive crowd teaching in crowdsourcing applications.
method Exponentially decayed memory model for teaching and balancing diversity and accuracy.
result JEDI teaching framework outperforms state-of-the-art techniques.
This paper improves RNN memory capacity for long sequences through learning associative memory update rules.
problem Challenges in RNNs remembering long sequences.
method Jointly learns memory update rule with task objective and uses multiple associative memories.
result Improves memory capacity for long sequence encoding.
Develops a new deep learning formulation using Mori-Zwanzig formalism.
problem Improves deep learning by introducing a new concept of memory.
method Uses Mori-Zwanzig formalism to propagate quantities of interest through neural networks.
result Rigorously transforms deep networks into shallow ones using decay property of memory operator.
Bayesian inference and superstatistics model financial volatility dynamics across different timescales.
problem Modeling correlated volatility in financial time series with heavy tails and long memory.
method Superstatistical dynamics, Bayesian Inference, Metropolis-Hasting sampling.
result The log-Normal model is reliable for short timescales, while inverse-Gamma is preferred for long timescales.
New framework analyzes temporal features in state space models.
problem Understanding temporal dependencies in data streams.
method Proposes a framework for rigorous analysis of state representations in ESNs, using temporal feature spaces and kernel machines.
result Phase transition in kernel richness for cycle reservoir topology.
We propose a stochastic process driven by the memory effect with novel distributions which include both exponential and leptokurtic heavy-tailed distributions. A class of the distributions is analytically derived from the continuum limit of the discrete binary process with the renormalized auto-correlation. The moment …
A new method selects PCA components based on residual memory, outperforming existing techniques.
problem Selecting the optimal number of components in PCA for data with long memory effects.
method Sequentially removes components, stopping when maximum memory accounted for.
result Our method outperforms existing techniques in computational efficiency and accuracy.
The study identifies persistent motifs in stock correlations for sector-neutral portfolio diversification.
problem Forecasting and diversification of sector-neutral portfolios using long-term correlations.
method Analysis of Triangulated Maximally Filtered Graphs (TMFG) generated from rolling windows of stock price log-returns, identifying persistent motifs.
result Persistent motifs in stock correlations can be used to forecast and diversify sector-neutral portfolios, reducing volatility.
A new GARCH model uses a two-dimensional Markov chain to capture long memory in volatility.
problem Capturing long-term volatility persistence in financial data.
method A GARCH-type model with state-dependent decay of past shocks using a two-dimensional Markov chain.
result The model successfully captures substantial volatility persistence and outperforms forecasts using only a two-dimensional state.
We analyze the memory in volatility by studying volatility return intervals, defined as the time between two consecutive fluctuations larger than a given threshold, in time periods following stock market crashes. Such an aftercrash period is characterized by the Omori law, which describes the decay in the rate of after…
Recent empirical studies have demonstrated long-memory in the signs of orders to buy or sell in financial markets [2, 19]. We show how this can be caused by delays in market clearing. Under the common practice of order splitting, large orders are broken up into pieces and executed incrementally. If the size of such lar…
The paper explores how score-driven models can approximate rough volatility.
problem Modeling rough volatility with long memory structures.
method Extending score-driven models to include infinite-lag structures and heavy-tailed decay.
result Score-driven models converge to fractional Ornstein-Uhlenbeck processes under appropriate scaling.
Improved performance of factorized neural layers through spectral initialization and Frobenius decay.
problem Improving the performance of factorized neural layers in various deep learning contexts.
method Spectral initialization and Frobenius decay for initialization and regularization.
result Spectral initialization and Frobenius decay lead to improved performance across multiple deep learning settings.
We investigate the probability distribution of the volatility return intervals τ for the Chinese stock market. We rescale both the probability distribution Pq(τ) and the volatility return intervals τ as Pq(τ)=1/τˉf(τ/τˉ) to obtain a uniform scaling curve for different threshold value q. The scali…
New method combines long-memory reservoirs for accurate dengue forecasting from short data.
problem Accurate dengue forecasting from short, noisy, non-stationary, and nonlinear data.
method Fractional ESN and Wavelet ESN frameworks integrating long-term memory.
result fESN and wESN outperform baselines in multiple dengue datasets and forecasting horizons.
Extended model accounts for finite memory effects in financial markets.
problem Modeling latent liquidity and its impact in financial markets.
method Continuous reaction-diffusion setup with finite cancellation and deposition rates.
result Square root impact law with finite memory corrections and linear permanent impact.
Stock prices are observed to be random walks in time despite a strong, long term memory in the signs of trades (buys or sells). Lillo and Farmer have recently suggested that these correlations are compensated by opposite long ranged fluctuations in liquidity, with an otherwise permanent market impact, challenging the s…
Enhanced financial trading system using multi-agent LLMs with layered memory.
problem Inefficient prioritization of tasks in LLMs due to their memory processing.
method Introducing a multi-agent framework with layered memories and inter-agent debate.
result Superior automated trading accuracy and decision robustness.
New CTRW model explains volatility clustering in stock markets.
problem Missing models for long-term memory in time intervals between observations.
method Introduced a new family of CTRWs with correlated waiting times.
result Successfully describes the decay of nonlinear autocorrelation function in stock market returns.
Using a relationship between the moments of the probability distribution of times between the two consecutive trades (intertrade time distribution) and the moments of the distribution of a daily number of trades we show, that the underlying point process is essentially non-markovian. A detailed analysis of all trades i…
DAS3H optimizes skill-based spaced repetition schedules.
problem Optimizing adaptive and personalized spaced repetition schedules for skill-based learning.
method Developed a new student learning and forgetting model (DAS3H) that considers memory decay and multiple skills.
result DAS3H outperforms other models on real-world educational datasets.
GraphGP: Scalable Gaussian Processes with Vecchia's Approximation
problem Naive Gaussian Process computation limits practical use
method GPU algorithm for Vecchia's approximation
result Linear time and memory requirements for nearly a billion parameters
We consider the supOU stochastic volatility model which is able to exhibit long-range dependence. For this model we give conditions for the discounted stock price to be a martingale, calculate the characteristic function, give a strip where it is analytic and discuss the use of Fourier pricing techniques. Finally, we p…
This work improves normalization methods in deep networks, enhancing stability and performance.
problem Shortcomings of Batch-Normalization hinder its use for certain tasks.
method Presented a novel view on normalization methods and weight-decay, suggesting alternatives like L1 and L∞. result Improved normalization methods enable first batch-norm alternative for half-precision implementations.
New method trains deep networks robustly without adaptive methods.
problem Training deep networks with robustness and efficiency.
method Scale invariant architecture + SGD + weight decay + gradient clipping.
result SGD can achieve similar performance to adaptive methods like Adam.
Logarithmic-time schedules boost large-scale language model training efficiency.
problem Improving performance and efficiency in large-scale language model training.
method Designing time-varying hyperparameters (β1,β2,λ) for AdamW, specifically logarithmic-time scheduling with damping mechanisms. result ADANA optimizer achieves up to 40% compute efficiency compared to tuned AdamW, with gains persisting as model scale increases.
We address the problem of long-range memory in the financial markets. There are two conceptually different ways to reproduce power-law decay of auto-correlation function: using fractional Brownian motion as well as non-linear stochastic differential equations. In this contribution we address this problem by analyzing e…