A new type of neural network variable called 'fast weights' improves sequence models by storing recent past memories.
problem Sequence models benefit from attention to the past, but current neural networks lack a mechanism to store recent past information efficiently.
method Introduce 'fast weights' that change faster than neural activities but slower than standard weights, allowing for temporary memory storage of recent past.
result Fast weights enable efficient implementation of past attention in sequence models without needing to store neural activity patterns.
Modeling a temporal process as if it is Markovian assumes the present encodes all of the process's history. When this occurs, the present captures all of the dependency between past and future. We recently showed that if one randomly samples in the space of structured processes, this is almost never the case. So, how d…
New method prevents deep learning forgetting past by remembering key examples.
problem Catastrophic forgetting in continual learning.
method Functional regularisation using Gaussian Process formulation.
result Achieves state-of-the-art performance on benchmarks.
New method recovers classical cut and paste groups for manifolds.
problem Cut and paste invariants of manifolds.
method Algebraic K-theory framework.
result Constructs a K-space recovering classical SK groups.
Survey of recent metric geometry in Kähler metrics space.
problem Understanding the metric geometry of Kähler metrics space.
method Survey and highlighting of recent results.
result Highlighting of open problems in the field.
A method to improve sequential learning by keeping past data errors in check.
problem Challenges in sequential learning with Gaussian processes due to accumulating errors.
method Memory-based dual sparse variational Gaussian processes.
result Improves accuracy in inference and learning for various applications.
Enhances SAC for better sample efficiency in continuous-action tasks.
problem Improving sample efficiency in soft actor-critic algorithms.
method Integrating Emphasizing Recent Experience (ERE) with Soft Actor-Critic (SAC) and Priority Experience Replay (PER).
result ERE significantly improves sample efficiency compared to vanilla SAC, especially for continuous-action tasks.
Algorithm learns actions from past states in complex tasks.
problem Learning policies from human feedback is expensive.
method Combining learned feature encoder with inverse models to simulate past actions.
result Algorithm can infer specific skills from single state.
We show that recent stock market fluctuations are characterized by the cumulative distributions whose tails on short, minute time scales exhibit power scaling with the scaling index alpha > 3 and this index tends to increase quickly with decreasing sampling frequency. Our study is based on high-frequency recordings of …
This work improves reinforcement learning with sparse rewards by following diverse past trajectories.
problem Challenges in reinforcement learning with sparse rewards and myopic behavior.
method Proposes a trajectory-conditioned policy to learn from a memory buffer of diverse past trajectories.
result Significantly outperforms existing methods on complex tasks with local optima.
Regularization and Bayesian methods for system identification have been repopularized in the recent years, and proved to be competitive w.r.t. classical parametric approaches. In this paper we shall make an attempt to illustrate how the use of regularization in system identification has evolved over the years, starting…
A simple algorithm predicts well with recent observations and a few summary stats.
problem Predicting future observations from past data with complex dependencies.
method A simple Markov model using recent observations and summary stats.
result A simple algorithm achieves optimal prediction error with minimal memory.
New method generates portfolios using market weights and past data.
problem Creating efficient trading strategies based on market weights.
method Pathwise generation of portfolios using market weights and past data.
result Improved conditions for outperforming the market over time.
The study examines how investor protection and past information affect stock returns and interest rates.
problem Empirical regularities related to investor protection and past information in asset pricing models.
method Developed a dynamic asset pricing model with a controlling shareholder and good/bad memory in budget dynamics.
result Good/bad memory of investors on historical market information affects stock returns and interest rates, strengthening investor protection in high ownership concentration.
VEST automates feature engineering for time series forecasting.
problem Challenges in time series forecasting with improved performance.
method VEST combines auto-regression with statistical summarization of recent past dynamics.
result VEST significantly improves forecasting performance.
Smooth Kahler-Einstein metrics have been studied for the past 80 years. More recently, singular Kahler-Einstein metrics have emerged as objects of intrinsic interest, both in differential and algebraic geometry, as well as a powerful tool in better understanding their smooth counterparts. This article is mostly a surve…
New analysis shows ROI's predictive power for stock returns weakens significantly.
problem The predictive power of retail order imbalance (ROI) for future stock returns.
method Replicated Boehmer et al. (2021) using a more recent period and analyzed the effect of using alternative quote midpoint (QMP) method.
result Past ROI can no longer predict weekly returns on large-cap stocks, and the long-short strategy based on past ROI is no longer profitable.
Nostalgic Adam improves long-term memory in adaptive learning rate algorithms.
problem Lack of long-term memory in Adam-like algorithms can lead to performance issues and divergence.
method Proposes Nostalgic Adam (NosAdam), which weights more past gradients, theoretically ensuring convergence.
result NosAdam can be a fix for Adam's non-convergence issues and is a promising alternative.
Modeling financial crises and cryptocurrency shocks using copulae clustering.
problem Detecting financial crises and shock events in stock and cryptocurrency markets.
method Copulae clustering based on probability distribution distances.
result Successfully detected all past crises and shock events in stock and cryptocurrency markets.
Motivated by recent proposals for a de Sitter version of the AdS/CFT correspondence, we give some topological restrictions on spacetimes of de Sitter type, i.e., spacetimes with Λ>0, which admit a regular past and/or future conformal boundary. For example we show that if Mn+1, n≥2, is a globally hyperbolic…
The first part of these notes is a self-contained introduction to generalized complex geometry. It is intended as a `user manual' for tools used in the study of supersymmetric backgrounds of supergravity. In the second part we review some past and recent results on the generalized complex structure of supersymmetric ty…
A technique identifies memoryless algorithms approximating memory-dependent optimization methods.
problem Understanding how memory in optimization algorithms affects loss and generalization.
method Introducing a general technique to replace past iterates with the current one and adding a correction term.
result Lion does not have the same implicit anti-regularization as AdamW, explaining its better generalization performance.
Automated Bayesian coreset construction for scalable inference.
problem Designing scalable and theoretically sound automated inference algorithms.
method Sparsity-constrained variational inference within an exponential family.
result Automated Bayesian coreset construction with improved KL divergence to the exact posterior.
Sparse Attentive Backtracking selectively backpropagates long-term dependencies in recurrent networks.
problem Difficulty in learning long-term dependencies in BPTT due to computational impracticality and biased gradient estimates.
method Sparse Attentive Backtracking learns an attention mechanism over past hidden states and selectively backpropagates through high-weight paths.
result Model learns long-term dependencies with fewer backpropagation steps, addressing biased gradient issues.
A simple approach improves performance on both past and future tasks in lifelong learning.
problem Forgetting in lifelong learning, where performance on past tasks degrades when learning new tasks.
method Representation ensembling to improve performance on both future and past tasks.
result Representation ensembling demonstrates both forward and backward transfer across various datasets.
We investigate the use of the Hurst exponent, dynamically computed over a moving time-window, to evaluate the level of stability/instability of financial firms. Financial firms bailed-out as a consequence of the 2007-2010 credit crisis show a neat increase with time of the generalized Hurst exponent in the period prece…
Until recently, Ricci flow was viewed almost exclusively as a way of deforming Riemannian metrics of bounded curvature. Unfortunately, the bounded curvature hypothesis is unnatural for many applications, but is hard to drop because so many new phenomena can occur in the general case. This article surveys some of the th…
A novel memory mechanism for reinforcement learning agents that stores past events in human-readable language.
problem Lack of interpretability in reinforcement learning agent's memory mechanisms.
method Uses CLIP to associate visual inputs with language tokens, then feeds these tokens to a pretrained language model.
result Significantly faster convergence on challenging continuous recognition tasks.
This survey aims to cover the motivation for and history of the study of local rigidity of group actions. There is a particularly detailed discussion of recent results, including outlines of some proofs. The article ends with a large number of conjectures and open questions and aims to point to interesting directions f…
Guyon-Lekeufack model accurately predicts market volatility.
problem Modeling and predicting market volatility accurately.
method Path-dependent volatility model with weighted past price returns and squared volatility.
result Wellposedness of the coupled system of stochastic differential equations for all parameter values.
Agents learn to outperform in trading by using past and current prices.
problem Optimal trading performance beyond theoretical limits.
method Two-agent Almgren-Chriss liquidation game, schedule-learning, DDQN architectures.
result Agents with access to past and current prices achieve supra-competitive outcomes.
New displacement technique vanishes bounded cohomology in all degrees.
problem Vanishing of bounded cohomology in all positive degrees and dual separable coefficients.
method Introducing the property of commuting cyclic conjugates as a new displacement technique.
result Vanishes bounded cohomology in all positive degrees and all dual separable coefficients.
We introduce a model-independent approximation for the branching ratio of Hawkes self-exciting point processes. Our estimator requires knowing only the mean and variance of the event count in a sufficiently large time window, statistics that are readily obtained from empirical data. The method we propose greatly simpli…
EvoRate metric assesses learnability of sequential data by measuring predictive information.
problem Model misspecification due to misinterpreting patterns in sequential data.
method Predictive information framework based on mutual information between past and future.
result Temporal patterns fundamentally constrain learnability; optimal predictors cannot outperform intrinsic information limit.
A new spectrum recovers cobordism cut and paste groups of manifolds with boundary.
problem Defining and studying cobordism cut and paste groups of manifolds with boundary.
method Constructing a spectrum that recovers the cobordism cut and paste groups of manifolds with boundary.
result Construction of a spectrum that recovers the cobordism cut and paste groups of manifolds with boundary.
New method reconstructs past foehn occurrences using unsupervised and supervised learning.
problem Reconstructing past foehn occurrences due to lack of direct measurement.
method Combining unsupervised and supervised learning methods to infer foehn occurrences from reanalysis data.
result Accurate hourly reconstructions of past foehn occurrences for 83 years.
New method selects recent similar periods for better electricity price forecasting.
problem Improving accuracy in forecasting electricity prices.
method Change-point detection (NOT method) to select calibration periods; estimating autoregressive models only for selected data.
result Significant improvement in forecasting accuracy compared to existing methods.
Deep learning brings major advances in AI, especially in computer vision.
problem Keeping track of regular advances in deep learning is challenging for new researchers.
method Briefly discusses recent advances in deep learning over the past few years.
result Revolutionary advances in computer vision and machine learning.
New method learns policies from past data.
problem Learning decision policies from experience.
method Fully Probabilistic Design (FPD) formalism for stochastic policy finding.
result Proposes a new approach for policy learning.
Paper connects TQFTs and SKK invariants through a group homomorphism.
problem Relating TQFTs and SKK invariants for smooth manifolds.
method Investigates a group homomorphism between invertible TQFTs and SKK invariants.
result All positive real-valued SKK invariants can be realized as TQFT restrictions.
Efficiently stores and retrieves past states for faster learning in reinforcement learning.
problem Data inefficiency and memory limitations in reinforcement learning.
method Dynamic online k-means for state clustering and prioritization.
result Dynamic online k-means improves performance with smaller memory sizes.
Analyzes non-Markovian environments in stochastic approximation.
problem Understanding learning mechanisms in non-ergodic, non-Markovian settings.
method Analytic framework for transformer learning and continual learning.
result Proposes a new approach to transformer and continual learning.
Develops effective adversarial attacks on probabilistic forecasting models.
problem Adversarial attacks on neural models outputting probability distributions.
method Effective generation of adversarial attacks through Monte-Carlo estimation and Bayesian conditioning.
result Demonstrates successful generation of attacks with small input perturbations.
In this paper, we extend the results of Klainerman and Rodnianski in \cite{KR:Trapped}, which were obtained for a finite region, by showing similar results from past null infinity. This allows us to recover and extend the results from past null infinity in the work of Christodoulou \cite{Chr:book}.
Study helps identify which ANN parameters cause forgetting.
problem Catastrophic forgetting in neural networks.
method Determines individual parameter contributions to forgetting.
result Identifies specific ANN parameters causing forgetting.
The paper develops a method to learn robust decision policies from observational data, reducing high-cost outcomes.
problem Learning safe decision policies from observational data with high-risk outcomes.
method Develops a method to learn policies that reduce high-cost outcomes, valid under finite samples and uneven feature overlap.
result Validates the method with real and synthetic data, providing statistical bounds on decision costs.
The paper shows failure of smooth pasting principle in time-inconsistent stopping problems.
problem Time-inconsistent stopping problems with non-constant time preference rates.
method Analysis of the smooth pasting principle within the intra-personal game theoretic framework.
result The smooth pasting principle fails under time-inconsistency and does not guarantee equilibrium solutions.
Past lightcones of certain points in globally hyperbolic spacetimes determine the entire spacetime.
problem Determining the entire spacetime from the past lightcone of a point.
method Analyzing properties of globally hyperbolic spacetimes and using null lines and observer horizons.
result Past lightcones of certain points in globally hyperbolic spacetimes determine the entire spacetime (up to isometry).