Paper discusses flaws in traditional RL for lifelong learning.
problem Traditional RL fails to model lifelong learning systems.
method Simplified prototype of lifelong RL system.
result Insights into lifelong RL, showing traditional RL's limitations.
New method for RL tasks transfer using Lipschitz continuity.
problem Knowledge transfer in RL tasks over time.
method Established Lipschitz continuity between MDPs and applied it to RL.
result Improved convergence rate and no negative transfer with high probability.
Modulating masks improve lifelong reinforcement learning.
problem Catastrophic forgetting and task interference in lifelong reinforcement learning.
method Adapted modulating masks for deep lifelong reinforcement learning (LRL) with PPO and IMPALA agents.
result Superior performance in both discrete and continuous RL tasks compared to LRL baselines.
Develops RL algorithm for lifelong non-stationary environments.
problem Challenges of reinforcement learning in environments with persistent change.
method Formalizes lifelong non-stationarity, uses latent variable models, and leverages online learning and probabilistic inference.
result Substantial improvement in performance over non-reasoning approaches in lifelong non-stationary environments.
Algorithm learns multiple tasks with minimal planning, achieving near-optimal performance.
problem Learning multiple tasks efficiently in a reinforcement learning setting.
method UCB Lifelong Value Distillation (UCBlvd) algorithm with structural assumption for shared exploration.
result Sublinear regret bound of i l d e O ( ( d 3 + d ′ d ) H 4 K ) ilde{\mathcal{O}}(\sqrt{(d^3+d^\prime d)H^4K}) i l d e O ( ( d 3 + d ′ d ) H 4 K ) with O ( d H log ( K ) ) \mathcal{O}(dH\log(K)) O ( d H log ( K )) planning calls. Generative memory helps AI learn new tasks without forgetting old ones.
problem Catastrophic forgetting in lifelong reinforcement learning.
method Generative memory model for task-agnostic latent space.
result Generative memory can improve lifelong reinforcement learning performance.
Meta-MDP approach optimizes exploration for lifelong reinforcement learning.
problem Improving reinforcement learning performance across multiple tasks.
method Formulates exploration strategy as a reinforcement learning problem, leveraging patterns from related tasks.
result Optimized exploration strategy improves performance on new tasks.
A lifelong learning architecture improves reinforcement learning policies using simulations and a DNC model.
problem Improving reinforcement learning policies in dynamic environments.
method Iterative training of a Reinforcement Learning agent and a DNC model in conjunction.
result DNC models can continually learn from pixels alone to simulate new tasks.
Dynamic information balancing reduces catastrophic forgetting in modular neural networks.
problem Catastrophic forgetting in neural networks when learning multiple tasks.
method Dynamic Information Balancing (DIB) using reinforcement learning to adaptively route inputs based on module information load.
result DIB combined with EWC regularization outperforms models with similar capacity and EWC regularization.
An important property for lifelong-learning agents is the ability to combine existing skills to solve unseen tasks. In general, however, it is unclear how to compose skills in a principled way. We provide a "recipe" for optimal value function composition in entropy-regularised reinforcement learning (RL) and then exten…
Framework for lifelong learning using eigentasks to avoid forgetting and transfer knowledge.
problem Avoiding forgetting and transferring knowledge in lifelong learning.
method Eigentask framework: skills paired with generative models, wake-sleep cycle for learning and consolidation.
result Improved performance in supervised continual learning, evidence of forward knowledge transfer.
PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.
problem Improving lifelong learning in multi-armed bandits.
method PAC-Bayesian analysis for deriving lower bounds and proposing lifelong learning algorithms.
result Proposed algorithms outperform baseline methods in lifelong multi-armed bandit problems.
New method speeds up lifelong learning of complex tasks.
problem Catastrophic forgetting in lifelong learning.
method Directly trains lifelong function approximators via policy gradients.
result Faster convergence and better policies achieved.
Top 8 robotic vision systems tackled lifelong object recognition challenges.
problem Lifelong learning in robotic vision for varied, dynamic environments.
method Design of a dataset with diverse conditions and rules for evaluation.
result Robotic vision systems improved over time with dynamic object appearances.
Pre-trained language model boosts RL efficiency.
problem Low sample efficiency in RL, especially in lifelong learning.
method Use a pre-trained task-independent language model for transfer learning.
result Goal-conditional RL agents become more sample efficient.
Unified approach for lifelong learning in recurrent neural networks.
problem Catastrophic forgetting and capacity saturation in lifelong learning.
method Proposed a curriculum-based benchmark and unified model combining Gradient Episodic Memory and Net2Net.
result Unified model performs better than constituent models in lifelong learning setting.
Paper proposes an algorithm for lifelong learning with shared structure.
problem Lifelong learning with shared structure in an online setting.
method Proposes a simple algorithm using multi-task empirical risk minimization.
result Establishes a sample complexity bound based on task-eluder dimension.
Flatland is a simple, lightweight environment for fast prototyping and testing of reinforcement learning agents. It is of lower complexity compared to similar 3D platforms (e.g. DeepMind Lab or VizDoom), but emulates physical properties of the real world, such as continuity, multi-modal partially-observable states with…
Paper converts graph learning to lifelong learning.
problem Learning graphs in a streaming fashion.
method Feature graph topology, converting node classification to graph classification.
result FGN achieves superior performance in lifelong human action recognition and feature matching.
FCL3 learns from multiple representative models to adapt to new tasks.
problem Degradation of performance in lifelong learning models for new task environments.
method Two knowledge libraries: feature learning and model knowledge. Self-selection of representative models for new tasks.
result FCL3 achieves better performance than most lifelong learning frameworks.
Algorithm captures and refines features for efficient lifelong learning.
problem Efficiently learning multiple tasks sequentially with limited data.
method Proposes a lifelong learning algorithm that maintains and refines internal feature representation.
result Proves sample complexity improvement and matches lower bound for efficiency.
Builds a novel educational recommender for lifelong learners.
problem Challenges in creating scalable and transparent models for lifelong learning.
method Integrative approach combining content novelty, background knowledge, and learner engagement.
result TrueLearn achieves promising performance while retaining a human interpretable learner model.
A simple approach improves performance on both past and future tasks in lifelong learning.
problem Forgetting in lifelong learning, where performance on past tasks degrades when learning new tasks.
method Representation ensembling to improve performance on both future and past tasks.
result Representation ensembling demonstrates both forward and backward transfer across various datasets.
AutoML challenge solved lifelong learning problems without i.i.d. data.
problem Designing lifelong learning systems without independent and identically distributed data.
method Developed and evaluated machine learning programs using CodaLab platform.
result More than 300 participants competed and solved complex lifelong learning problems.
New dataset and benchmarks for lifelong robotic vision tasks.
problem Challenges in applying computer vision to robots, especially lifelong learning.
method Provided a new lifelong robotic vision dataset and benchmarks.
result Demonstrated the complexity and bottlenecks in lifelong object recognition.
Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute to the development and specialization of our sensorimotor sk…
DBULL learns new clusters without forgetting past knowledge in streaming unlabelled data.
problem Challenges in Unsupervised Lifelong Learning with evolving data distributions and class labels.
method Bayesian framework for incremental learning, Deep Bayesian Unsupervised Lifelong Learning (DBULL) algorithm, knowledge preservation mechanism, automatic cluster discovery.
result DBULL can progressively discover new clusters without forgetting past knowledge in unlabelled data.
Transfer learning has received a lot of attention in the machine learning community over the last years, and several effective algorithms have been developed. However, relatively little is known about their theoretical properties, especially in the setting of lifelong learning, where the goal is to transfer information…
A new lifelong online learning framework combining current and accumulated knowledge.
problem Continuous learning over tasks with limited data and unknown number of instances.
method Interactive learning algorithm combining current task information and accumulated knowledge.
result Algorithm can benefit from small cumulative error even with few interactions.
Framework for lifelong learning of compositional structures.
problem Learning to reuse self-contained chunks of knowledge for novel problems.
method Separates learning into combining existing components and adapting them.
result Framework handles trade-off between stability and flexibility.
A-GEM improves lifelong learning efficiency with minimal computational and memory cost.
problem Efficiency in lifelong learning with incremental task exposure.
method Introducing A-GEM, a new version of GEM, with a novel evaluation protocol and metric.
result A-GEM achieves the best trade-off between accuracy and efficiency in lifelong learning benchmarks.
New framework SEU solves lifelong learning's catastrophic forgetting issue.
problem Catastrophic forgetting in lifelong learning.
method Introduces Neural Architecture Search into lifelong learning to dynamically adapt model structures for different tasks.
result Achieves higher accuracy with significantly smaller model size (25-33% of state-of-the-art methods).
New algorithm adapts to changing action sets in sequential decision making.
problem Adapting to a changing action set in sequential decision making problems.
method Breaks the problem into inferring action structure and optimizing policy.
result Demonstrates efficiency on real-world lifelong learning problems.
New schemes improve lifelong learning by balancing old and new tasks.
problem Catastrophic forgetting in deep neural networks when learning multiple tasks.
method Unified optimization perspective of episodic memory based approaches, introducing MEGA-I and MEGA-II schemes.
result Significant improvement in lifelong learning benchmarks, reducing error by up to 18%.
A Boolean algebra formalizes task composition for reinforcement learning.
problem Formalizing task composition for efficient learning and problem-solving.
method Formalized tasks as a Boolean algebra, learning goal-oriented value functions, and composing them to solve new tasks.
result Agents can solve new tasks without additional learning by composing value functions in specific ways.
A two-stream reinforcement learning model improves decision-making across human and neuropsychiatric studies.
problem Improving reinforcement learning models to better simulate human decision-making and neuropsychiatric conditions.
method Proposes a two-stream reinforcement learning model that processes positive and negative rewards and incorporates reward-processing biases.
result The two-stream model outperforms standard Q-learning and SARSA methods on various tasks and datasets.
Unified framework supports lifelong learning properties.
problem Capturing desirable properties of human learning in machine learning.
method Unified framework with one central mechanism.
result Supports almost all desirable properties of human learning.
Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of the inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong …
Graphs evolve; GNNs learn from past tasks.
problem Learning from evolving graphs and new classes.
method Incremental training method for lifelong learning on graphs.
result Less than 50% of the GNN's receptive field is needed for 95% accuracy.
Adapts Lifelong Learning for Federated Learning to prevent model drift.
problem Federated Learning on non-i.i.d. data leads to model divergence and forgetting.
method Adds a penalty term to the loss function to encourage convergence of local models to a shared optimum.
result The method improves performance in image recognition tasks compared to existing approaches.
AOP combines model-based planning with model-free learning to handle lifelong learning challenges.
problem Learning control in an online reset-free lifelong learning scenario where mistakes can compound and dynamics change.
method Adaptive Online Planning (AOP) that combines model-based planning with model-free learning, approximating uncertainty to call upon planning only when necessary.
result Achieves strong performance in lifelong learning challenges, gracefully adapting behaviors in the face of unpredictable changes.
A method for continual learning using world models in reinforcement learning.
problem Catastrophic forgetting in lifelong learning with neural networks.
method Interleaving internally generated episodes of past experiences (pseudo-rehearsal) with external environment's observations.
result Consistent reduction in temporal prediction loss compared to non-interleaved learning.
Model learns continuously, starting from a blank slate.
problem Lifelong learning starting from zero.
method Inspired by neuroplasticity, the model develops nodes based on four rules: expansion, generalization, forgetting, and backpropagation.
result Better performance in accuracy, energy efficiency, and versatility compared to other models.
Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may significantly improve the learning performance, most of the literature on transfer is focused on ba…
PETAL adapts models to changing target domains over time.
problem Lifelong test-time adaptation in changing target domains.
method Probabilistic framework with student-teacher model and data-driven parameter restoration.
result PETAL achieves better results than state-of-the-art for online lifelong test-time adaptation.
Model learns continuously from text without forgetting.
problem Catastrophic forgetting in lifelong language learning.
method Episodic memory with sparse experience replay and local adaptation.
result Model can continuously learn from new datasets.
New neural network avoids forgetting old knowledge as new data comes in.
problem Catastrophic forgetting in lifelong learning systems.
method Sequential Neural Coding Network, biologically-plausible synapse adaptation, task-executive control mimicry.
result Significantly less forgetting compared to standard neural models in experiments.
LBO optimizes model selection for evolving datasets.
problem Model selection for datasets arriving and evolving over time.
method Online, multitask Bayesian optimization exploiting function correlation.
result LBO outperforms standard BO algorithms on real and synthetic data.