A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Hybrid framework prevents forgetting in continual learning.
Overparameterization helps prevent forgetting in sequential learning tasks.
In this paper, we show that Generative Adversarial Networks (GANs) suffer from catastrophic forgetting even when they are trained to approximate a single target distribution. We show that GAN training is a continual learning problem in which the sequence of changing model distributions is the sequence of tasks to the d…
Synthetic data helps prevent forgetting when learning sequentially.
Pseudo-rehearsal allows neural networks to learn a sequence of tasks without forgetting how to perform in earlier tasks. Preventing forgetting is achieved by introducing a generative network which can produce data from previously seen tasks so that it can be rehearsed along side learning the new task. This has been fou…
Residual Continual Learning prevents forgetting in sequential tasks.
EWC helps prevent forgetting in neural networks by adjusting weights dynamically.
Paper proposes ensemble methods to prevent forgetting in neural networks.
A new method prevents forgetting during knowledge transfer.
Proposes a new method to prevent forgetting in neural networks.
EVCL combines VCL and EWC to prevent forgetting new tasks.
Paper tackles catastrophic forgetting in sequential learning.
Dynamic memory prevents forgetting in continuous learning of medical images.
New method prevents deep learning forgetting past by remembering key examples.
Deep neural networks are known to suffer the catastrophic forgetting problem, where they tend to forget the knowledge from the previous tasks when sequentially learning new tasks. Such failure hinders the application of deep learning based vision system in continual learning settings. In this work, we present a simple …
AI learns to learn sequentially without forgetting.
Study examines how training regime affects neural networks' forgetting.
New method prevents forgetting in learning new tasks.
LaLoRA prevents forgetting in LoRA fine-tuning.
Continual learning is the ability to sequentially learn over time by accommodating knowledge while retaining previously learned experiences. Neural networks can learn multiple tasks when trained on them jointly, but cannot maintain performance on previously learned tasks when tasks are presented one at a time. This pro…
ER-GNN uses experience replay to prevent GNNs from forgetting previous tasks.
New method prevents forgetting in learning new tasks.
Sparse routing networks with co-training prevent catastrophic forgetting in continual learning.
Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes…
Incremental learning suffers from two challenging problems; forgetting of old knowledge and intransigence on learning new knowledge. Prediction by the model incrementally learned with a subset of the dataset are thus uncertain and the uncertainty accumulates through the tasks by knowledge transfer. To prevent overfitti…
Most artificial intelligence models have limiting ability to solve new tasks faster, without forgetting previously acquired knowledge. The recently emerging paradigm of continual learning aims to solve this issue, in which the model learns various tasks in a sequential fashion. In this work, a novel approach for contin…
Though neural networks have achieved much progress in various applications, it is still highly challenging for them to learn from a continuous stream of tasks without forgetting. Continual learning, a new learning paradigm, aims to solve this issue. In this work, we propose a new model for continual learning, called Ba…
We quantify forgetting in post-training models, distinguishing mass and drift.
Despite all the success that deep neural networks have seen in classifying certain datasets, the challenge of finding optimal solutions that generalize still remains. In this paper, we propose the Boundary Optimizing Network (BON), a new approach to generalization for deep neural networks when used for supervised learn…
Modern Hopfield networks help prevent forgetting in generative models after task changes.
Humans can learn in a continuous manner. Old rarely utilized knowledge can be overwritten by new incoming information while important, frequently used knowledge is prevented from being erased. In artificial learning systems, lifelong learning so far has focused mainly on accumulating knowledge over tasks and overcoming…
Paper analyzes convergence of continual learning with adaptive methods.
Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if e…
VAR-GPs solve continual learning by updating posteriors sequentially.
Improved rates for continual learning using SGD and last-iterate analysis.
A new method combines generative models and regularization to prevent forgetting in continual learning.
In supervised machine learning, an agent is typically trained once and then deployed. While this works well for static settings, robots often operate in changing environments and must quickly learn new things from data streams. In this paradigm, known as streaming learning, a learner is trained online, in a single pass…
New method prevents forgetting in LLMs by dynamically identifying task-specific subspaces.
Meta-learning approach prevents forgetting across tasks.
Human beings are able to master a variety of knowledge and skills with ongoing learning. By contrast, dramatic performance degradation is observed when new tasks are added to an existing neural network model. This phenomenon, termed as \emph{Catastrophic Forgetting}, is one of the major roadblocks that prevent deep neu…
New method prevents deep learning models from forgetting past tasks.
SPACE algorithm prevents forgetting in neural networks by partitioning learned knowledge.
Paper tackles continual learning with Transformers, maintaining performance and speed.
This paper introduces a new lifelong learning solution where a single model is trained for a sequence of tasks. The main challenge that vision systems face in this context is catastrophic forgetting: as they tend to adapt to the most recently seen task, they lose performance on the tasks that were learned previously. O…
Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster. Neural networks trained by stochastic gradient descent often degrade on old tasks when trained successively on new tasks with different data distrib…
We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on neuronal model sparsification. In this method, subsequent tasks are trained using the inactive neurons and filters of the sparsified network and cause zero deterioration to the performanc…
AGS-CL selectively updates penalties based on node importance for continual learning.
New framework prevents forgetting in sequential learning tasks.