MOCA enables meta-learning without task segmentation for online learning.
problem Meta-learning without task segmentation for online learning.
method MOCA combines meta-learning with online changepoint analysis.
result MOCA enables efficient meta-learning in time-varying tasks.
New algorithm for online meta-learning with task boundary detection.
problem Adapting to new tasks in a non-stationary environment.
method Two detection mechanisms for task switches and distribution shift; online model updates based on current data.
result Achieves sublinear task-averaged regret under mild conditions.
Bayesian online meta-learning framework tackles catastrophic forgetting in few-shot classification.
problem Catastrophic forgetting in few-shot classification problems.
method Bayesian online learning, meta-learning, Laplace approximation, variational inference.
result Framework effectively achieves goal of overcoming catastrophic forgetting in few-shot classification.
Algorithm improves online learning in adversarial bandits.
problem Online learning in adversarial multi-armed bandits with non-uniform best arm distribution.
method Online-within-online setup, inner and outer learners, leveraging non-uniform empirical distribution of best arms.
result Improves regret bounds for non-uniform best arm distributions.
Meta-strategy learns tuning parameters for online learning methods.
problem Difficulty in setting tuning parameters for online learning methods.
method Meta-learning approach to learn parameters from past tasks.
result Meta-strategy improves on learning each task in isolation.
New method for online meta-learning reduces dynamic regret in changing environments.
problem Learning new tasks quickly from limited data in dynamic settings.
method Established dynamic regret analysis using generalized adaptive gradient methods.
result Logarithmic local dynamic regret with dependence on total iterations and learner parameters.
This paper surveys meta-learning, online, and continual learning.
problem Combining and understanding meta-learning, online, and continual learning.
method Organizing various problem settings using consistent terminology and formal descriptions.
result Fosters further advancements in meta-learning, online, and continual learning.
The online meta-learning framework is designed for the continual lifelong learning setting. It bridges two fields: meta-learning which tries to extract prior knowledge from past tasks for fast learning of future tasks, and online-learning which deals with the sequential setting where problems are revealed one by one. I…
Meta-learning improves performance across similar tasks in adversarial bandit settings.
problem Improving performance across multiple similar tasks in adversarial bandit scenarios.
method Designing meta-algorithms that combine outer learners to tune hyperparameters of inner learners for MAB and BLO.
result Meta-algorithms improve task-averaged regret for MAB and BLO, showing direct relationship with action space-dependent measures.
Proposes efficient user cold start recommendation via meta parameter partition.
problem User cold start in recommendation systems.
method Divides model parameters into fixed and adaptive parts, learning them separately offline and online.
result Significant improvement in AUC (2.48% absolute improvement).
A central capability of intelligent systems is the ability to continuously build upon previous experiences to speed up and enhance learning of new tasks. Two distinct research paradigms have studied this question. Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adapt…
La-MAML improves fast online continual learning with a look-ahead approach.
problem Fast online continual learning with limited model capacity.
method Optimisation-based meta-learning with look-ahead and episodic memory.
result Superior performance on visual classification benchmarks.
POLA adapts learning rates for online time series prediction.
problem Adapting to changing data distributions in dynamic environments.
method Adaptive learning rate regulation for recurrent neural networks.
result POLA outperforms other online prediction methods in real-world datasets.
Humans and animals can learn complex predictive models that allow them to accurately and reliably reason about real-world phenomena, and they can adapt such models extremely quickly in the face of unexpected changes. Deep neural network models allow us to represent very complex functions, but lack this capacity for rap…
VSML unifies meta learning concepts and enables simple backpropagation.
problem Improving and unifying meta learning concepts for neural networks.
method Unified approach using variable shared meta learning and simple weight-sharing.
result Simple backpropagation can be implemented and meta learned without gradient calculation.
Meta-learning bandits by reducing dimensionality with PCA.
problem Learning multiple bandit tasks with shared structure.
method Online Principal Component Analysis (PCA) for dimensionality reduction, combined with optimistic and Thompson sampling strategies.
result Significant reduction in expected regret compared to existing methods.
Meta-learning control algorithm with finite-time guarantees for unknown systems.
problem Online control of unknown linear systems with constraints.
method Provable regret guarantees for an iterative control algorithm.
result Regret bounds of O(T3/4) for controller cost and constraint violation. Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinf…
We build a theoretical framework for designing and understanding practical meta-learning methods that integrates sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms. Our approach enables the task-similarity to be learned adapti…
MetaTS learns to explore better by meta-learning prior from bandit instances.
problem Efficient exploration in bandit problems.
method MetaThompson Sampling (MetaTS) that meta-learns the prior from bandit instances.
result MetaTS quickly adapts to unknown prior and achieves better exploration.
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when tasks are considerably dissimilar or change over time. We use the connection between gradient-based me…
The constant introduction of standardized benchmarks in the literature has helped accelerating the recent advances in meta-learning research. They offer a way to get a fair comparison between different algorithms, and the wide range of datasets available allows full control over the complexity of this evaluation. Howev…
This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.
problem Real-world tasks violate assumptions of task distributions, independence, and clear task delineations.
method A mixture of Gaussian Processes models different dynamics, and a transition prior handles temporal dependencies.
result The approach reliably handles task distribution shifts and outperforms alternatives in non-stationary tasks.
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
We study the problem of meta-learning through the lens of online convex optimization, developing a meta-algorithm bridging the gap between popular gradient-based meta-learning and classical regularization-based multi-task transfer methods. Our method is the first to simultaneously satisfy good sample efficiency guarant…
Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but on the agent's uncertainty about the environment. Computing a Bayes-optimal policy is how…
Meta-learning adjusts TD learning's eligibility trace parameter for more efficient reinforcement learning.
problem Efficiently tuning the eligibility trace parameter for temporal difference learning.
method Meta-learning method to adjust eligibility trace parameter state-dependently.
result Improves overall quality of update targets, minimizing target error.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
Unified meta-algorithm improves average performance across similar tasks in adversarial bandits.
problem Improving performance across multiple similar tasks in adversarial bandit settings.
method Unified meta-algorithm for multi-armed bandits and bandit linear optimization, tuning initialization, step-size, and entropy parameters.
result Unified meta-algorithm yields setting-specific guarantees for MAB and BLO, improving task-averaged regret.
The study learns neural update rules by remembering past experiences.
problem Developing efficient online learning rules for neural networks.
method Representing neurons with vectors, using meta-neural networks for updates, and training for remembering past experiences.
result The approach reveals insights into learning rules and could be used for complex tasks like episodic memory.
MetaSelector learns to choose the best model for each user.
problem Heterogeneous datasets and user-specific historical data make it hard to find the best model for each user.
method Meta-learning framework to train a model selector that chooses the best model for each user based on their historical data.
result MetaSelector outperforms single model and sample-level model selector in AUC and LogLoss.
MetaTNE tackles few-shot novel labels in graphs, improving node classification.
problem Node classification on graphs with novel labels and limited training data.
method MetaTNE framework with structural, meta-learning, and optimization modules.
result MetaTNE significantly improves node classification over state-of-the-art methods.
In [1, 2], we have explored the theoretical aspects of feature extraction optimization processes for solving largescale problems and overcoming machine learning limitations. Majority of optimization algorithms that have been introduced in [1, 2] guarantee the optimal performance of supervised learning, given offline an…
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
problem Improving sample efficiency in continuous control tasks.
method Meta-critic that meta-learns an additional loss for the actor.
result Online meta-critic learning leads to improved performance in various continuous control environments.
A method learns common bias for multiple low-variance tasks without hyper-parameter tuning.
problem Learning common bias for multiple low-variance tasks without manual tuning.
method Two variants of online learning methods (aggressive and lazy) that update bias after each datapoint or at the end of each task.
result Across-tasks regret bound derived for the method, showing faster rates for aggressive variant and standard rates for lazy variant.
Cold-start problems are long-standing challenges for practical recommendations. Most existing recommendation algorithms rely on extensive observed data and are brittle to recommendation scenarios with few interactions. This paper addresses such problems using few-shot learning and meta learning. Our approach is based o…
Framework detects fake news using weak social signals from multiple sources.
problem Lack of annotated data for early fake news detection.
method Jointly uses weak social signals and clean data to train deep neural networks in a meta-learning framework.
result Framework outperforms state-of-the-art baselines for early fake news detection.
The security of Deep Reinforcement Learning (Deep RL) algorithms deployed in real life applications are of a primary concern. In particular, the robustness of RL agents in cyber-physical systems against adversarial attacks are especially vital since the cost of a malevolent intrusions can be extremely high. Studies hav…
One of the main obstacles to broad application of reinforcement learning methods is the parameter sensitivity of our core learning algorithms. In many large-scale applications, online computation and function approximation represent key strategies in scaling up reinforcement learning algorithms. In this setting, we hav…
Online portfolio selection is a fundamental problem in computational finance, which has been extensively studied across several research communities, including finance, statistics, artificial intelligence, machine learning, and data mining, etc. This article aims to provide a comprehensive survey and a structural under…
APO optimizes neural network parameters by amortizing proximal point methods.
problem Optimizing neural network parameters online and adaptively.
method APO framework that meta-learns proximal point parameters.
result APO can recover and outperform existing optimizers and schedules.
Framework uses RL with dynamic embedding to outperform benchmarks in volatile markets.
problem Challenges in high-dimensional, non-stationary, and noisy market information.
method Dynamic embedding of market information using generative autoencoders and online meta-learning in a reinforcement learning framework.
result Framework outperforms common portfolio benchmarks and PTO approach during market stress.
Meta-learning adapts models for unseen tasks across AI, robotics, and NLP.
problem Adapting models to unseen tasks efficiently and accurately.
method Black-box, metric-based, layered, and Bayesian approaches.
result Meta-learning enhances model generalization and adaptation to unseen tasks.
Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time. Given that it is impractical …
New algorithm reduces decision switching in dynamic environments.
problem Online learning with memory and non-stationary environments.
method Dynamic policy regret, novel ensemble approach, meta-base decomposition.
result Proves optimal dynamic policy regret for memory length, non-stationarity, and time horizon.
Diffusion models help in learning priors for Thompson Sampling in bandit problems.
problem Learning effective strategies for diverse bandit tasks.
method Training a denoising diffusion model to learn task distributions, combining with Thompson Sampling.
result The approach significantly improves performance across different bandit tasks.
A continual learning agent should be able to build on top of existing knowledge to learn on new data quickly while minimizing forgetting. Current intelligent systems based on neural network function approximators arguably do the opposite---they are highly prone to forgetting and rarely trained to facilitate future lear…
Meta-learning improves feature extraction for few-shot tasks.
problem Understanding why meta-learning models perform better on few-shot classification.
method Developed hypotheses and a regularizer to improve standard training routines.
result Meta-learned models outperform classical training routines in few-shot classification.