A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Optimal task order improves continual learning performance.
Proposes Mv-TCNN for continual learning without known tasks in advance.
Bilevel Continual Learning improves continual learning by transferring knowledge effectively.
Paper analyzes convergence of continual learning with adaptive methods.
In order to mimic the human ability of continual acquisition and transfer of knowledge across various tasks, a learning system needs the capability for continual learning, effectively utilizing the previously acquired skills. As such, the key challenge is to transfer and generalize the knowledge learned from one task t…
CAM-GAN improves GANs for continual learning with efficient feature map transformations.
Hybrid framework prevents forgetting in continual learning.
COCOA can learn new tasks without forgetting previously learned ones in a distributed setting.
EVCL combines VCL and EWC to prevent forgetting new tasks.
Most artificial intelligence models have limiting ability to solve new tasks faster, without forgetting previously acquired knowledge. The recently emerging paradigm of continual learning aims to solve this issue, in which the model learns various tasks in a sequential fashion. In this work, a novel approach for contin…
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in compl…
Continual learning aims to enable machine learning models to learn a general solution space for past and future tasks in a sequential manner. Conventional models tend to forget the knowledge of previous tasks while learning a new task, a phenomenon known as catastrophic forgetting. When using Bayesian models in continu…
Study shows memory needs grow with task sequence length in continual learning.
New insights into continual learning with task similarity.
Continual Learning is a learning paradigm where learning systems are trained with sequential or streaming tasks. Two notable directions among the recent advances in continual learning with neural networks are () variational Bayes based regularization by learning priors from previous tasks, and, () learning the s…
Despite the growing interest in continual learning, most of its contemporary works have been studied in a rather restricted setting where tasks are clearly distinguishable, and task boundaries are known during training. However, if our goal is to develop an algorithm that learns as humans do, this setting is far from r…
Improved rates for continual learning using SGD and last-iterate analysis.
A new framework for federated continual learning reduces interference and improves performance.
Sparse routing networks with co-training prevent catastrophic forgetting in continual learning.
Though neural networks have achieved much progress in various applications, it is still highly challenging for them to learn from a continuous stream of tasks without forgetting. Continual learning, a new learning paradigm, aims to solve this issue. In this work, we propose a new model for continual learning, called Ba…
Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes…
Proposes a framework for semi-supervised continual learning from sequentially arriving data.
Improves continual learning with theoretical guarantees and a new algorithm.
Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. The resulting network resembles a static entity of knowledge, with endeavours to extend this knowledge without targeting the ori…
Reliable and effective multi-task learning is a prerequisite for the development of robotic agents that can quickly learn to accomplish related, everyday tasks. However, in the reinforcement learning domain, multi-task learning has not exhibited the same level of success as in other domains, such as computer vision. In…
Study on neural networks' performance in sequential task learning.
New insights into continual learning for deep models, showing convergence issues but local linear solutions.
New method prevents forgetting in LLMs by dynamically identifying task-specific subspaces.
Hybrid model avoids forgetting across tasks and classes.
Continual lifelong learning is essential to many applications. In this paper, we propose a simple but effective approach to continual deep learning. Our approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. By enforcing their integration in an itera…
Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms need an external representation and extra computation …
Neural network tackles continual learning with neuromodulation and local error signals.
Artificial neural networks have exceeded human-level performance in accomplishing several individual tasks (e.g. voice recognition, object recognition, and video games). However, such success remains modest compared to human intelligence that can learn and perform an unlimited number of tasks. Humans' ability of learni…
New method generates continuous Pareto sets for multi-task learning.
Paper tackles continual learning with single-index models, proving regret bounds.
We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidati…
TAME learns new tasks without knowing them, outperforming existing methods.
Extends neural network approximations to guarantee continuity of real-world learning tasks.
This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.
New method handles unknown task boundaries in continual learning.
CLOPS improves deep learning for continuous physiological data.
Residual Continual Learning prevents forgetting in sequential tasks.
New approach to continual learning prioritizes adaptation over retention.
While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, some issues remain to be tackled to apply them to real-world problem domains. First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number o…
Learning a set of tasks over time, also known as continual learning (CL), is one of the most challenging problems in artificial intelligence. While recent approaches achieve some degree of CL in deep neural networks, they either (1) grow the network parameters linearly with the number of tasks, (2) require storing trai…
AGS-CL selectively updates penalties based on node importance for continual learning.
Continual learning is the ability to sequentially learn over time by accommodating knowledge while retaining previously learned experiences. Neural networks can learn multiple tasks when trained on them jointly, but cannot maintain performance on previously learned tasks when tasks are presented one at a time. This pro…
New method prevents forgetting in learning new tasks.