Bayesian MAML outperforms MAML in meta learning tasks with theoretical guarantees.
problem Theoretical understanding of Bayesian MAML's superiority over MAML.
method Comparison of meta test risks between Bayesian MAML and MAML in meta linear regression.
result Bayesian MAML has provably lower meta test risks than MAML in both distribution agnostic and linear centroid cases.
Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with non…
Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinf…
Study MAML's generalization in varying tasks, proving bounds on error.
problem Bounding MAML's generalization error across tasks.
method Characterizes MAML's generalization error from two perspectives: recurring and unseen tasks.
result MAML's generalization error depends on the number of tasks and samples per task.
The paper analyzes MAML's representation using RSA, revealing that feature reuse is not the primary reason for its success.
problem Understanding why model-agnostic meta-learning (MAML) works well in few-shot learning tasks.
method Representation similarity analysis (RSA) applied to MAML's few-shot learning instantiation.
result Feature reuse is not the primary reason for MAML's success; instead, it is the learning task itself that increases representation similarity.
New method improves convergence of RL meta-learning.
problem Improving convergence in model-agnostic meta-reinforcement learning.
method Proposes Stochastic Gradient Meta-Reinforcement Learning (SG-MRL) to find ε-first-order stationary points. result Derives iteration and sample complexity for SG-MRL.
Meta-learning has emerged as an important framework for learning new tasks from just a few examples. The success of any meta-learning model depends on (i) its fast adaptation to new tasks, as well as (ii) having a shared representation across similar tasks. Here we extend the model-agnostic meta-learning (MAML) framewo…
New MAML variant prioritizes hardest tasks, improving robustness.
problem Meta-learning's focus on average performance, ignoring worst-case.
method Reformulate MAML to minimize max loss over observed tasks.
result Task-robust model performs well across various task distributions.
We derive the ODE of MAML and propose a new BI-MAML algorithm.
problem Training efficiency and computational burden in MAML.
method Continuous-time limit view of MAML, ODE derivation, and BI-MAML algorithm.
result MAML ODE shows linear convergence rate for strongly convex task losses.
CARML uses meta-learning to avoid obstacles in 2D vehicle navigation.
problem Collision avoidance in 2D vehicle navigation.
method Model-Agnostic Meta-Learning for multi-objective reinforcement learning.
result CARML outperforms a baseline TD3 solution in obstacle avoidance.
Meta-learning for few-shot learning entails acquiring a prior over previous tasks and experiences, such that new tasks be learned from small amounts of data. However, a critical challenge in few-shot learning is task ambiguity: even when a powerful prior can be meta-learned from a large number of prior tasks, a small d…
When fitting Bayesian machine learning models on scarce data, the main challenge is to obtain suitable prior knowledge and encode it into the model. Recent advances in meta-learning offer powerful methods for extracting such prior knowledge from data acquired in related tasks. When it comes to meta-learning in Gaussian…
Sharp-MAML improves MAML by reducing saddle points in few-shot learning.
problem Challenges in optimizing MAML due to complex loss landscape.
method Sharpness-aware minimization applied to MAML.
result Sharp-MAML and its variant outperform plain MAML on few-shot learning tasks.
The study examines when MAML's objective has a benign landscape.
problem Understanding when MAML's objective landscape is benign.
method Analyzing the landscape of MAML objective on LQR tasks.
result The benign landscape of the MAML objective depends on task similarities.
End-to-end meta-learned system for image compression.
problem Reducing the gap between training and inference conditions in image compression.
method Model-Agnostic Meta-learning approach for latent tensor overfitting and updating encoder and decoder networks.
result Meta-learned system achieves better compression performance compared to traditional methods.
First-order ANIL learns shared representations even with overparametrization.
problem Lack of theoretical evidence for model-agnostic meta-learning (ANIL) learning shared representations.
method First-order ANIL with a linear two-layer network architecture, showing asymptotically low-rank solutions with overparametrization.
result First-order ANIL learns linear shared representations, even with overparametrization, and performs well in adaptation.
ANIL adapts only a subset of parameters, reducing computational cost.
problem Efficiently adapt model parameters in meta-learning.
method Adapts only a small subset of parameters in the inner loop of ANIL.
result Theoretical convergence and computational complexity analysis for ANIL.
New analysis improves generalization bounds for meta-learning.
problem Improving generalization in meta-learning algorithms.
method Information-theoretic analysis of MAML and its stochastic variant.
result Data-dependent generalization bound is tighter and non-vacuous.
New method improves meta-learning performance by task-specific initialization.
problem Difficulties in generalizing and achieving theoretical guarantees in conditional meta-learning.
method Structured prediction approach for task-specific initialization.
result TASML improves performance of existing meta-learning models.
MAML optimizes shared priors for subtasks in a nonconvex meta-objective.
problem Understanding global optimality of MAML for nonconvex meta-objectives.
method Characterizes optimality gap of MAML stationary points via first-order optimization methods.
result Establishes global optimality of MAML for both RL and supervised learning.
Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those frameworks demonstrate appealing performance on a variety of domains such as few-shot image classificat…
BOIL updates model body only, showing better few-shot learning performance.
problem Few-shot learning efficiency with model reuse vs. change.
method Proposes BOIL, updating only model body, freezing head.
result Significantly outperforms MAML on cross-domain tasks.
New framework guarantees convergence of multi-step MAML.
problem Convergence of multi-step MAML in nonconvex settings.
method Developed a theoretical framework for two types of MAML objective functions.
result Guaranteed convergence rate and computational complexity for multi-step MAML.
Meta-learning improves image segmentation performance.
problem Improving image segmentation accuracy using meta-learning.
method Extending FOMAML and Reptile to image segmentation, using EfficientLab architecture, and leveraging test error definition.
result Meta-learned initializations provide value for few-shot image segmentation but are quickly matched by conventional transfer learning.
The performance of gradient-based optimization strategies depends heavily on the initial weights of the parametric model. Recent works show that there exist weight initializations from which optimization procedures can find the task-specific parameters faster than from uniformly random initializations and that such a w…
A new meta-learning framework that assigns weights to source tasks based on target samples.
problem Learning initialization for target tasks with limited labeled examples.
method A general framework that assigns weights to the loss of different source tasks, which can depend on the target samples. Provides upper bounds and develops a learning algorithm based on minimizing the error bound with respect to an empirical IPM.
result Empirically, the weighted meta-learning algorithm finds better initializations than uniformly-weighted meta-learning algorithms.
Few-shot learning improves bearing fault diagnosis with limited data.
problem Challenges in collecting sufficient fault data for robust classifier training.
method Model-Agnostic Meta-Learning (MAML) for few-shot learning.
result Framework achieves up to 25% higher accuracy than Siamese network.
We introduce a new, rigorously-formulated Bayesian meta-learning algorithm that learns a probability distribution of model parameter prior for few-shot learning. The proposed algorithm employs a gradient-based variational inference to infer the posterior of model parameters to a new task. Our algorithm can be applied t…
Bayesian online meta-learning framework tackles catastrophic forgetting in few-shot classification.
problem Catastrophic forgetting in few-shot classification problems.
method Bayesian online learning, meta-learning, Laplace approximation, variational inference.
result Framework effectively achieves goal of overcoming catastrophic forgetting in few-shot classification.
The paper proposes a learning algorithm that improves adaptability and generalization.
problem Improving adaptability and generalization in learning models.
method Learning to meta-learn by meta-finetuning on related tasks before adapting to specific tasks.
result Learning to meta-learn improves adaptability and generalization across various tasks.
The capacity of meta-learning algorithms to quickly adapt to a variety of tasks, including ones they did not experience during meta-training, has been a key factor in the recent success of these methods on few-shot learning problems. This particular advantage of using meta-learning over standard supervised or reinforce…
Unified framework improves meta-learning generalization bounds.
problem Limited sharpness of existing meta-generalization bounds.
method Unified information-theoretic derivation for single-step bounds.
result Unified bounds exhibit tighter scaling and computational advantages.
Federated Learning (FL) refers to learning a high quality global model based on decentralized data storage, without ever copying the raw data. A natural scenario arises with data created on mobile phones by the activity of their users. Given the typical data heterogeneity in such situations, it is natural to ask how ca…
The field of few-shot learning has recently seen substantial advancements. Most of these advancements came from casting few-shot learning as a meta-learning problem. Model Agnostic Meta Learning or MAML is currently one of the best approaches for few-shot learning via meta-learning. MAML is simple, elegant and very pow…
C-ADAM is a new adaptive solver for complex nested problems.
problem Solving compositional problems involving nested expected values.
method Adaptive solver for non-linear functional nesting of expected values.
result C-ADAM converges to a stationary point in O(δ−2.25). ST-MAML tackles task ambiguity in meta-learning by encoding tasks with stochastic representations.
problem Handling tasks from multiple distributions is challenging for meta-learning due to task ambiguity.
method ST-MAML uses a stochastic neural network module to encode tasks and propagate task representations to revise input variable encoding.
result ST-MAML matches or outperforms state-of-the-art methods on various tasks.
Meta-learning improves GNN initializations for low-resource drug discovery.
problem Limited labeled data hinders deep learning in drug discovery.
method Model-Agnostic Meta-Learning (MAML) and its variants for graph neural networks initializations.
result Meta-initializations outperform multi-task pre-training baselines on 16 out of 20 tasks and all out-of-distribution tasks.
Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure that uses the knowledge learned across many tasks as an inductive bias towards better natural languag…
Meta-learning framework uses task similarity through nonparametric kernel regression.
problem Limited tasks and outliers/dissimilar tasks hinder meta-learning performance.
method Nonparametric kernel regression to quantify and use task similarity.
result Meta-learning algorithm outperforms existing methods in task-limited settings.
A new biased gradient descent method for conditional stochastic optimization.
problem Challenges in constructing unbiased gradient estimators for conditional stochastic optimization.
method Proposes a biased stochastic gradient descent (BSGD) algorithm and analyzes its sample complexities.
result Establishes sample complexities of BSGD for various objectives and shows that BSpiderBoost matches the lower bound complexity.
We introduce ES-MAML, a new framework for solving the model agnostic meta learning (MAML) problem based on Evolution Strategies (ES). Existing algorithms for MAML are based on policy gradients, and incur significant difficulties when attempting to estimate second derivatives using backpropagation on stochastic policies…
Gradient-based meta-learners such as MAML are able to learn a meta-prior from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. One important limitation of such frameworks is that they seek a common initialization shared across the entire task distribution, substantially limiti…
Meta-learning improves few-shot land cover classification across diverse regions.
problem Capturing diversity in land cover classification across different geographic regions.
method Model-agnostic meta-learning (MAML) algorithm applied to classification and segmentation tasks.
result Few-shot model adaptation outperforms traditional methods in diverse land cover classification tasks.
Model-Agnostic Meta-Learning (MAML) and its variants have achieved success in meta-learning tasks on many datasets and settings. On the other hand, we have just started to understand and analyze how they are able to adapt fast to new tasks. For example, one popular hypothesis is that the algorithms learn good represent…
Develops a new method for efficient stochastic bilevel optimization.
problem Stochastic bilevel optimization problems in machine learning applications.
method Single-Timescale stochAstic BiLevEl optimization (STABLE) method.
result Achieves the same order of sample complexity as stochastic gradient descent for single-level optimization.
Personalized federated learning adapts models to each user's data.
problem Training models across users without data exchange leads to a common model, not personalized.
method Develops a personalized federated learning approach using Model-Agnostic Meta-Learning (MAML).
result Personalized models can be adapted by a few gradient descent steps on local data.
New measure quantifies task difficulty for machine learning models.
problem Quantifying the inherent difficulty of machine learning tasks.
method Inductive bias complexity measure.
result Tasks requiring generalization over many dimensions are more difficult.
GeneraLight improves traffic signal control models' generalization ability.
problem Overfitting and lack of generalization ability in RL TSC models.
method GeneraLight uses a meta-RL framework with a traffic flow generator based on GANs.
result GeneraLight significantly boosts generalization performance across different traffic flows.