A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new buffer system improves continual learning in RL agents by adapting to changing environments.
The paper proposes a method to learn from both simulation and real-world data.
MER algorithm speeds up VI solving with Markovian data.
LiDER refreshes past experiences in RL by dreaming about them.
In Multi-Goal Reinforcement Learning, an agent learns to achieve multiple goals with a goal-conditioned policy. During learning, the agent first collects the trajectories into a replay buffer, and later these trajectories are selected randomly for replay. However, the achieved goals in the replay buffer are often biase…
A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the p…
ReaPER improves learning efficiency by prioritizing reliable experiences.
Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…
DAC enhances exploration in reinforcement learning with entropy regularization.
A distributed system identification method for LTI systems using reverse experience replay.
A critical and challenging problem in reinforcement learning is how to learn the state-action value function from the experience replay buffer and simultaneously keep sample efficiency and faster convergence to a high quality solution. In prior works, transitions are uniformly sampled at random from the replay buffer o…
ETGL-DDPG improves DDPG for sparse reward control with new exploration and replay techniques.
Improved diffusion models for sampling from given distributions.
The paper improves reinforcement learning stability and efficiency with a new theoretical framework.
Unified study of stateful replay for streaming learning, reducing forgetting by 2-3x.
Proposes a new policy gradient algorithm to improve reinforcement learning efficiency and stability.
Experience replay is a key technique behind many recent advances in deep reinforcement learning. Allowing the agent to learn from earlier memories can speed up learning and break undesirable temporal correlations. Despite its wide-spread application, very little is understood about the properties of experience replay. …
New algorithm optimizes linear system estimation from single trajectory.
Paper improves online time series forecasting by combining natural gradient and robust t-distribution.
Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster. Neural networks trained by stochastic gradient descent often degrade on old tasks when trained successively on new tasks with different data distrib…
Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies. On-policy methods bring many benefits, such as ability to evaluate each resulting policy. However, they usually d…
Improved Politex algorithm reduces regret bound to O(√T) with experience replay.
New method improves deep RL by combining emphatic weightings with replay data.
Online reinforcement learning agents are currently able to process an increasing amount of data by converting it into a higher order value functions. This expansion of the information collected from the environment increases the agent's state space enabling it to scale up to a more complex problems but also increases t…
Study improves neural network performance in sequential learning for image classification.
In continual learning, the learner faces a stream of data whose distribution changes over time. Modern neural networks are known to suffer under this setting, as they quickly forget previously acquired knowledge. To address such catastrophic forgetting, many continual learning methods implement different types of exper…
A new method for generating replay samples on the fly, optimizing for not forgetting.
NERS improves RL by sampling diverse transitions considering local and global contexts.
Paper proposes a new method to select memory data for online class-incremental learning.
A simple technique improves continual learning by 50% on image datasets.
We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems. Our agents explore via Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop neural network. Our algorithm learns much faster than common exploration strategies such as …
In this paper, a novel racing environment for OpenAI Gym is introduced. This environment operates with continuous action- and state-spaces and requires agents to learn to control the acceleration and steering of a car while navigating a randomly generated racetrack. Different versions of two actor-critic learning algor…
New method improves ABI for sequential data, reducing forgetting and improving accuracy.
Although reinforcement learning has made great strides recently, a continuing limitation is that it requires an extremely high number of interactions with the environment. In this paper, we explore the effectiveness of reusing experience from the experience replay buffer in the Deep Q-Learning algorithm. We test the ef…
Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.
Paper proposes a new method for sampling from complex distributions.
This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-Reinforcement Learning (meta-RL). MQL builds upon three simple ideas. First, we show that Q-learning is competitive with state-of-the-art meta-RL algorithms if given access to a context variable that is a representation of the past traject…
Traditional model-based RL relies on hand-specified or learned models of transition dynamics of the environment. These methods are sample efficient and facilitate learning in the real world but fail to generalize to subtle variations in the underlying dynamics, e.g., due to differences in mass, friction, or actuators a…
Importance sampling (IS) is a common reweighting strategy for off-policy prediction in reinforcement learning. While it is consistent and unbiased, it can result in high variance updates to the weights for the value function. In this work, we explore a resampling strategy as an alternative to reweighting. We propose Im…
In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achieved goal states. The problem of learning from imbalanced data is a well-known problem in supervised…
Secure aggregation for buffered asynchronous federated learning without TEEs.
Experience replay enables reinforcement learning agents to memorize and reuse past experiences, just as humans replay memories for the situation at hand. Contemporary off-policy algorithms either replay past experiences uniformly or utilize a rule-based replay strategy, which may be sub-optimal. In this work, we consid…
Simplifies RL training with fewer techniques, reducing bias and instability.
We introduce a Markovian single point process model, with random intensity regulated through a buffer mechanism and a self-exciting effect controlling the arrival stream to the buffer. The model applies the principle of the Hawkes process in which point process jumps generate a shot-noise intensity field. Unlike the Ha…
New modifiers improve noisy RNN replay in hippocampal networks.
An agent learning through interactions should balance its action selection process between probing the environment to discover new rewards and using the information acquired in the past to adopt useful behaviour. This trade-off is usually obtained by perturbing either the agent's actions (e.g., e-greedy or Gibbs sampli…
New insights into experience replay in RL algorithms.
Neural network tackles continual learning with neuromodulation and local error signals.