A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
ReOPD uses pre-collected teacher trajectories to distill knowledge from multi-turn interactions.
A new robust prefix-tuning framework improves model robustness against adversarial attacks.
CROP verifies clean prefixes in reasoning traces, improving downstream repair accuracy.
Prefix consistency improves model reliability by weighting answers based on their reproducibility.
Improved text generation with discourse-aware soft prompts.
Experience replay enables reinforcement learning agents to memorize and reuse past experiences, just as humans replay memories for the situation at hand. Contemporary off-policy algorithms either replay past experiences uniformly or utilize a rule-based replay strategy, which may be sub-optimal. In this work, we consid…
New modifiers improve noisy RNN replay in hippocampal networks.
New insights into experience replay in RL algorithms.
This project combines recent advances in experience replay techniques, namely, Combined Experience Replay (CER), Prioritized Experience Replay (PER), and Hindsight Experience Replay (HER). We show the results of combinations of these techniques with DDPG and DQN methods. CER always adds the most recent experience to th…
We present a new replay-based method of continual classification learning that we term "conditional replay" which generates samples and labels together by sampling from a distribution conditioned on the class. We compare conditional replay to another replay-based continual learning paradigm (which we term "marginal rep…
This paper describes an improvement in Deep Q-learning called Reverse Experience Replay (also RER) that solves the problem of sparse rewards and helps to deal with reward maximizing tasks by sampling transitions successively in reverse order. On tasks with enough experience for training and enough Experience Replay mem…
Curious Replay improves model-based reinforcement learning agents' adaptability.
Exposure bias has been regarded as a central problem for auto-regressive language models (LM). It claims that teacher forcing would cause the test-time generation to be incrementally distorted due to the training-generation discrepancy. Although a lot of algorithms have been proposed to avoid teacher forcing and theref…
A new method prioritizes and recycles experiences for better reinforcement learning.
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
ReaPER improves learning efficiency by prioritizing reliable experiences.
Unified study of stateful replay for streaming learning, reducing forgetting by 2-3x.
In this paper, we propose a replay attack spoofing detection system for automatic speaker verification using multitask learning of noise classes. We define the noise that is caused by the replay attack as replay noise. We explore the effectiveness of training a deep neural network simultaneously for replay attack spoof…
Generative replay improves continual learning by using generated data as negative examples.
The paper improves reinforcement learning stability and efficiency with a new theoretical framework.
A new method improves continual learning by replaying pseudo data and using orthogonal weight modification.
Autostackability for finitely generated groups is defined via a topological property of the associated Cayley graph which can be encoded in a finite state automaton. Autostackable groups have solvable word problem and an effective inductive procedure for constructing van Kampen diagrams with respect to a canonical fini…
Study risk-constrained Kelly optimization for mutually exclusive outcomes, proving support invariance and developing a structured algorithm.
Our work is a simple extension of the paper "Exploration by Random Network Distillation". More in detail, we show how to efficiently combine Intrinsic Rewards with Experience Replay in order to achieve more efficient and robust exploration (with respect to PPO/RND) and consequently better results in terms of agent perf…
Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized experience replay (PER) which samples important transitions more frequently. In this paper, we propo…
Prototype-based generative replay framework for online continual regression.
New framework models echo chamber learning, proving tight bounds on algorithm performance.
We investigate the combination of actor-critic reinforcement learning algorithms with uniform large-scale experience replay and propose solutions for two challenges: (a) efficient actor-critic learning with experience replay (b) stability of off-policy learning where agents learn from other agents behaviour. We employ …
Experience replay is a key technique behind many recent advances in deep reinforcement learning. Allowing the agent to learn from earlier memories can speed up learning and break undesirable temporal correlations. Despite its wide-spread application, very little is understood about the properties of experience replay. …
An important component of many Deep Reinforcement Learning algorithms is the Experience Replay which serves as a storage mechanism or memory of made experiences. These experiences are used for training and help the agent to stably find the perfect trajectory through the problem space. The classic Experience Replay howe…
Experience replay (ER) is a fundamental component of off-policy deep reinforcement learning (RL). ER recalls experiences from past iterations to compute gradient estimates for the current policy, increasing data-efficiency. However, the accuracy of such updates may deteriorate when the policy diverges from past behavio…
This paper explains how to optimize prompts for model adaptation.
LiDER refreshes past experiences in RL by dreaming about them.
RER improves sample complexity by updating in reverse order.
Modern Hopfield networks help prevent forgetting in generative models after task changes.
Continual learning, the setting where a learning agent is faced with a never ending stream of data, continues to be a great challenge for modern machine learning systems. In particular the online or "single-pass through the data" setting has gained attention recently as a natural setting that is difficult to tackle. Me…
Improves sparse reinforcement learning efficiency with OYMB.
ER-GNN uses experience replay to prevent GNNs from forgetting previous tasks.
We examine the question of when and how parametric models are most useful in reinforcement learning. In particular, we look at commonalities and differences between parametric models and experience replay. Replay-based learning algorithms share important traits with model-based approaches, including the ability to plan…
New algorithm reduces regret in private online learning with optimal gap-dependent rate.
Improved RA detection with SNN outperforming baseline by 26.8% EER.
VRER selectively reuses past observations to reduce variance in policy optimization.
Improved Politex algorithm reduces regret bound to O(√T) with experience replay.
New method improves deep RL by combining emphatic weightings with replay data.
Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…
Online reinforcement learning agents are currently able to process an increasing amount of data by converting it into a higher order value functions. This expansion of the information collected from the environment increases the agent's state space enabling it to scale up to a more complex problems but also increases t…
A new buffer system improves continual learning in RL agents by adapting to changing environments.
An attacker may use a variety of techniques to fool an automatic speaker verification system into accepting them as a genuine user. Anti-spoofing methods meanwhile aim to make the system robust against such attacks. The ASVspoof 2017 Challenge focused specifically on replay attacks, with the intention of measuring the …