Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently,…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Episodic Memory”
We investigate the robustness properties of image recognition models equipped with two features inspired by human vision, an explicit episodic memory and a shape bias, at the ImageNet scale. As reported in previous work, we show that an explicit episodic memory improves the robustness of image recognition models agains…
We discuss memory models which are based on tensor decompositions using latent representations of entities and events. We show how episodic memory and semantic memory can be realized and discuss how new memory traces can be generated from sensory input: Existing memories are the basis for perception and new memories ar…
We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classificatio…
In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform …
Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.
problem Sample inefficiency in AI learning action policies.
method Incorporates hippocampus-inspired sequential memory system into DAC theory of mind.
result DAC-ML rapidly converges to effective action policies maximizing reward.
Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memorycs.NE
New memory allocation scheme improves image generation performance.
problem Improving episodic and semantic memory representation in neural networks.
method Developed a hierarchical latent variable model with differentiable, locally block allocated latent memory.
result Improved conditional likelihood values on various datasets.
Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field c…
Current generation of memory-augmented neural networks has limited scalability as they cannot efficiently process data that are too large to fit in the external memory storage. One example of this is lifelong learning scenario where the model receives unlimited length of data stream as an input which contains vast majo…
Proposes a CL technique to improve accuracy and reduce forgetting.
problem Sequential task learners struggle with forgetting information from previous tasks.
method Extracts modular parts of neural networks and estimates task relatedness.
result Remarkable performance gain in robustness to forgetting for EWC and GEM methods.
New method improves continual learning by anchoring past knowledge.
problem Catastrophic forgetting in continual learning.
method Bilevel optimization to update current task knowledge while keeping past task predictions.
result Improves accuracy and forgetting metrics compared to experience replay.
A novel approach stores encoded images as centroids and covariance matrices to improve classification accuracy with less memory.
problem Catastrophic forgetting and memory limitations in continual learning.
method Trains autoencoders with Neural Style Transfer to encode images, replay encoded episodes to avoid forgetting, and use centroids and covariance matrices for pseudo-images when memory is full.
result Increases classification accuracy by 13-17% over state-of-the-art methods on benchmark datasets, while requiring 78% less storage space.
A new method learns from multi-modal sequences with external memory.
problem Learning new modes in a dynamic environment without prior knowledge.
method Maintains a neural episodic memory with a Dirichlet Process prior to store mode descriptors and transfers knowledge through retrieval.
result Performs continual learning favorably compared to mainstream approaches.
Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the sam…
Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discover…
Optimal CL requires perfect memory and is NP-hard.
problem Designing CL algorithms that perform reliably and avoid catastrophic forgetting.
method Theoretical approach to derive computational properties of optimal CL algorithms.
result Optimal CL algorithms generally solve an NP-hard problem and require perfect memory.
Episodic memory is a psychology term which refers to the ability to recall specific events from the past. We suggest one advantage of this particular type of memory is the ability to easily assign credit to a specific state when remembered information is found to be useful. Inspired by this idea, and the increasing pop…
We consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, which is difficult to solve with existing QA methods due to their lack of scalability. To tackle this problem, we propose a novel end-to…
Recently, neuro-inspired episodic control (EC) methods have been developed to overcome the data-inefficiency of standard deep reinforcement learning approaches. Using non-/semi-parametric models to estimate the value function, they learn rapidly, retrieving cached values from similar past states. In realistic scenarios…
Agents learn to outperform in trading by using past and current prices.
problem Optimal trading performance beyond theoretical limits.
method Two-agent Almgren-Chriss liquidation game, schedule-learning, DDQN architectures.
result Agents with access to past and current prices achieve supra-competitive outcomes.
Model-Free Episodic Controlstat.ML
State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and…
Tiny Eats GRU detects eating episodes on a microcontroller.
problem Automatic dietary monitoring on low-power devices.
method Shallow gated recurrent unit (GRU) architecture on Arm Cortex M0+.
result Tiny Eats GRU achieves 95.15% accuracy with 4% memory usage and 6 ms latency.
Personalized predictive medicine necessitates the modeling of patient illness and care processes, which inherently have long-term temporal dependencies. Healthcare observations, recorded in electronic medical records, are episodic and irregular in time. We introduce DeepCare, an end-to-end deep dynamic neural network t…
Typical neural networks with external memory do not effectively separate capacity for episodic and working memory as is required for reasoning in humans. Applying knowledge gained from psychological studies, we designed a new model called Differentiable Working Memory (DWM) in order to specifically emulate human workin…
Novel approach to learning models based on subjective timescales for better exploration and decision-making.
problem Learning models over multi-step timescales in environments with intermediate states.
method Developed a subjective-timescale model (STM) based on episodic memories, enabling systematic variation of temporal extent of predictions.
result STM produces more informative action-conditioned roll-outs, leading to better decision-making and exploration.
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we pro…
This paper presents a novel framework for predicting shot location and type in tennis. Inspired by recent neuroscience discoveries we incorporate neural memory modules to model the episodic and semantic memory components of a tennis player. We propose a Semi Supervised Generative Adversarial Network architecture that c…
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
problem Memory models struggle in partially observable reinforcement learning environments.
method Introduces a novel memory model using the Hadamard product for efficient memory management and updates.
result Significantly outperforms state-of-the-art memory-based methods on challenging benchmarks.
The study learns neural update rules by remembering past experiences.
problem Developing efficient online learning rules for neural networks.
method Representing neurons with vectors, using meta-neural networks for updates, and training for remembering past experiences.
result The approach reveals insights into learning rules and could be used for complex tasks like episodic memory.
Paper proposes SDRL to improve continual learning with less computational cost.
problem Catastrophic forgetting in continual learning.
method SDRL method that refines gradients from memorized samples to reduce gradient diversity.
result SDRL shows better performance than state-of-the-art methods on multiple benchmark tasks.
Deep learning detects microsleep episodes in EEG data.
problem Automatic detection of microsleep episodes (MSEs) in EEG data.
method Convolutional neural networks (CNNs) and LSTM networks were implemented to analyze MWT data.
result Deep learning algorithms showed good performance close to human experts in detecting MSEs.
Improves memory efficiency for meta-learning with large images.
problem High memory usage in meta-learning for few-shot classification.
method LITE: episodic training scheme that decomposes task gradients and back-propagates only a random subset of images.
result Achieves state-of-the-art accuracy on real-world and challenging benchmarks.
Freezing of gait (FoG) is a common gait disability in Parkinson's disease, that usually appears in its advanced stage. Freeze episodes are associated with falls, injuries, and psychological consequences, negatively affecting the patients' quality of life. For detecting FoG episodes automatically, a highly accurate dete…
Memory is an important aspect of intelligence and plays a role in many deep reinforcement learning models. However, little progress has been made in understanding when specific memory systems help more than others and how well they generalize. The field also has yet to see a prevalent consistent and rigorous approach f…
Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, obs…
La-MAML improves fast online continual learning with a look-ahead approach.
problem Fast online continual learning with limited model capacity.
method Optimisation-based meta-learning with look-ahead and episodic memory.
result Superior performance on visual classification benchmarks.
We first pose the Unsupervised Progressive Learning (UPL) problem: an online representation learning problem in which the learner observes a non-stationary and unlabeled data stream, learning a growing number of features that persist over time even though the data is not stored or replayed. To solve the UPL problem we …
Graph-based rehearsal improves continual learning performance.
problem Catastrophic forgetting in continual learning models.
method Augmenting a memory array with a learnable random graph to capture pairwise similarities.
result Our model consistently outperforms baselines on task-free continual learning benchmarks.
Model Selection for Average Reward RL with Application to Utility Maximization in Repeated Gamescs.LG
Algorithm improves RL model selection for repeated games with utility maximization.
problem Optimal policy learning in repeated games with unknown opponent strategy.
method Proposes MRBEAR for average reward RL, applying to utility maximization in repeated games.
result Regret bound shows linear dependence on number of model classes in average reward RL.
Bilevel Continual Learning improves continual learning by transferring knowledge effectively.
problem Catastrophic forgetting and poor generalization in continual learning.
method Bilevel optimization and dual memory management strategies.
result BCL achieves effective knowledge transfer and alleviates catastrophic forgetting.
Study on neural networks with non-normal interactions reveals unique spectral properties.
problem Understanding episodic memory encoding in the brain.
method Developed a neural network model with non-Hermitian couplings and applied random matrix theory.
result Spectral density of the model is non-uniform and can transition to chaos, providing computational benefits.
V-HMN integrates memory mechanisms for improved image recognition.
problem Limited interpretability and high data requirements of existing vision backbones.
method Brain-inspired hierarchical memory modules with iterative refinement.
result V-HMN achieves strong performance on image classification benchmarks.
We analyse perception and memory, using mathematical models for knowledge graphs and tensors, to gain insights into the corresponding functionalities of the human mind. Our discussion is based on the concept of propositional sentences consisting of \textit{subject-predicate-object} (SPO) triples for expressing elementa…
NIPA aims to translate brain learning mechanisms into scalable Bayesian inference.
problem Scalable Bayesian inference for large-scale statistical machine learning problems.
method Neural-inspired algorithm combining model-based, model-free, and episodic-control modules.
result Advances Bayesian methods and facilitates their application to deep learning.
The majority of ML research concerns slow, statistical learning of i.i.d. samples from large, labelled datasets. Animals do not learn this way. An enviable characteristic of animal learning is `episodic' learning - the ability to memorise a specific experience as a composition of existing concepts, after just one exper…
Agents struggle with solving tasks in new environments, but new models improve performance.
problem Rapid task-solving in novel environments.
method Developed EPNs to enable deep RL agents to plan over gathered knowledge.
result EPNs enable deep RL agents to excel at RTS, outperforming baselines by factors of 2-3.
New online few-shot learning model for context-aware recognition.
problem Few-shot learning in online, continuous settings with spatiotemporal context.
method Proposed new dataset and online versions of existing few-shot learning approaches.
result Contextual prototypical memory model improves performance.
Improved accuracy in neural networks using CTF memory with RPU.
problem Challenges in achieving high accuracy in neural networks using in-memory computing.
method Exploring the trade-off between conductance change range and linearity in CTF memory for RPU-based matrix operations.
result Achieved accuracy of 97.9% on MNIST dataset, 89.1% and 70.5% on CIFAR-10 and CIFAR-100 datasets.