A new meta-learning framework that assigns weights to source tasks based on target samples.
problem Learning initialization for target tasks with limited labeled examples.
method A general framework that assigns weights to the loss of different source tasks, which can depend on the target samples. Provides upper bounds and develops a learning algorithm based on minimizing the error bound with respect to an empirical IPM.
result Empirically, the weighted meta-learning algorithm finds better initializations than uniformly-weighted meta-learning algorithms.
Meta-learning improves neural networks by adapting learning algorithms.
problem Conventional AI approaches solve tasks from scratch, but meta-learning aims to improve the learning algorithm.
method Meta-learning adapts a learning algorithm based on multiple learning episodes.
result Meta-learning can tackle deep learning challenges like data and computation bottlenecks.
New PAC-Bayesian framework for flexible meta-learning.
problem Limitations in transferring knowledge between tasks in meta-learning.
method PAC-Bayesian theory applied to learning the learning algorithm.
result Flexibility in meta-learning mechanisms and improved prediction quality.
Meta-learning helps models learn quickly from few samples.
problem Deep learning requires many samples, which are hard to get.
method Meta-learning optimizes models to adapt quickly to new tasks.
result Meta-learning can improve model efficiency and adaptability.
Meta-learning balances task-specific modeling and optimization complexity.
problem Balancing accurate task-specific modeling with ease of optimization in meta-learning.
method Theoretical and empirical analysis of trade-off between modeling and optimization in meta-learning.
result Explicit bounds on modeling and optimization errors for non-convex and linear regression problems.
Meta-learning algorithm improves AI efficiency by teaching itself.
problem Overcoming meta-optimization challenges in AI learning.
method Bootstraps a target from the meta-learner and optimizes the meta-learner by minimizing distance under a chosen metric.
result Achieves state-of-the-art performance on Atari ALE benchmark and demonstrates efficiency gains in multi-task learning.
Meta learning can adapt fast but is vulnerable to adversarial attacks.
problem Vulnerability of meta learning to adversarial attacks.
method Formal definition of adversarial attacks unique to meta learning, proposing an attacking algorithm.
result Meta learning is vulnerable to adversarial attacks.
Meta-learning algorithms are studied for their data efficiency in personalized federated learning.
problem Data efficiency in meta-learning algorithms.
method Algorithmic stability techniques to derive bounds on transfer risk and a new evaluation framework.
result Different meta-learning methods require varying amounts of supervision for effective generalization.
The paper provides fast, sample-efficient algorithms for learning common features in multi-task linear regression.
problem Learning transferable features from multiple related tasks.
method Provably fast, sample-efficient algorithms for multi-task linear regression.
result Theoretical guarantees on sample complexity for learning common features.
This paper improves meta-learning by developing new PAC-Bayes bounds.
problem Meta-learning generalization gap across multiple tasks.
method Upper bounding convex functions linking environment and task-level losses.
result New PAC-Bayes bounds for meta-learning with improved algorithms.
Meta-learning enables a model to learn from very limited data to undertake a new task. In this paper, we study the general meta-learning with adversarial samples. We present a meta-learning algorithm, ADML (ADversarial Meta-Learner), which leverages clean and adversarial samples to optimize the initialization of a lear…
Meta-learning framework uses task similarity through nonparametric kernel regression.
problem Limited tasks and outliers/dissimilar tasks hinder meta-learning performance.
method Nonparametric kernel regression to quantify and use task similarity.
result Meta-learning algorithm outperforms existing methods in task-limited settings.
Meta-learning improves feature extraction for few-shot tasks.
problem Understanding why meta-learning models perform better on few-shot classification.
method Developed hypotheses and a regularizer to improve standard training routines.
result Meta-learned models outperform classical training routines in few-shot classification.
Meta-learning algorithms use past experience to learn to quickly solve new tasks. In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks. The performance of meta-learning algorithms depe…
Self-referential meta learning avoids explicit optimization by modifying itself.
problem Dependency on human engineering in meta learning algorithms.
method Investigates self-referential meta learning systems that modify themselves without explicit optimization.
result Self-referential neural networks can improve their own modifications without explicit optimization.
Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks. However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task,…
learn2learn simplifies meta-learning research by providing a library and standardized interfaces.
problem Prototyping and reproducibility issues in meta-learning.
method Developed a library (learn2learn) with common routines and standardized interfaces.
result Fosters a community around standardized software for meta-learning research.
Probabilistic active meta-learning improves data efficiency in robotics.
problem Data-efficient learning in robotics where data collection is expensive.
method Conceptualizing meta-learning with a probabilistic latent variable model for sequential task selection.
result Improves data efficiency compared to baselines on simulated robotic experiments.
New analysis improves generalization bounds for meta-learning.
problem Improving generalization in meta-learning algorithms.
method Information-theoretic analysis of MAML and its stochastic variant.
result Data-dependent generalization bound is tighter and non-vacuous.
Meta learning with information theory and Gaussian processes.
problem Few-shot learning problems.
method Information bottleneck, mutual information, variational approximations, Gaussian processes.
result Competitive accuracy on few-shot classification problems.
Many IoT applications at the network edge demand intelligent decisions in a real-time manner. The edge device alone, however, often cannot achieve real-time edge intelligence due to its constrained computing resources and limited local data. To tackle these challenges, we propose a platform-aided collaborative learning…
New PAC-Bayes meta-learning method improves few-shot learning accuracy and calibration.
problem Few-shot learning with limited data.
method PAC-Bayes framework extended to meta-learning, estimating task-specific posteriors.
result State-of-the-art calibration and classification results on benchmarks.
TaskNorm improves meta-learning performance by rethinking batch normalization.
problem Challenges in batch normalization for meta-learning with deep networks.
method Developed TaskNorm, a novel approach to batch normalization for meta-learning.
result TaskNorm consistently improves meta-learning performance across various datasets and meta-learning approaches.
Meta-learning bounds derived using PAC-Bayes theory for improved generalization.
problem Uncertainty in generalization performance for meta-learning with new tasks.
method PAC-Bayes relative entropy bounds and empirical risk minimization (ERM) method.
result Competitive generalization performance and rapid convergence with data-dependent prior.
New meta-learning framework for minimizing simple regret in bandits.
problem Minimizing simple regret in a sequence of bandit tasks with unknown distributions.
method Developed Bayesian and frequentist meta-learning algorithms for bandits, analyzing their meta simple regret.
result Bayesian algorithm achieves i l d e O ( m / n ) ilde{O}(m / \sqrt{n}) i l d e O ( m / n ) meta simple regret, frequentist algorithm i l d e O ( m n + m / n ) ilde{O}(\sqrt{m} n + m/ \sqrt{n}) i l d e O ( m n + m / n ) . We extend first-order model agnostic meta-learning algorithms (including FOMAML and Reptile) to image segmentation, present a novel neural network architecture built for fast learning which we call EfficientLab, and leverage a formal definition of the test error of meta-learning algorithms to decrease error on out of d…
There is no free lunch, no single learning algorithm that will outperform other algorithms on all data. In practice different approaches are tried and the best algorithm selected. An alternative solution is to build new algorithms on demand by creating a framework that accommodates many algorithms. The best combination…
Algorithm reduces regret in non-stationary bandits and meta-learning with optimal arms.
problem Sequential decision-making with changing task boundaries and optimal arms.
method Reduction to bandit submodular maximization, meta-learning algorithms.
result Regret bounds for both non-stationary and bandit meta-learning problems.
Local nonparametric meta-learning improves meta-generalization across tasks.
problem Meta-learning struggles with global inductive biases and out-of-distribution tasks.
method Proposes a local, nonparametric meta-learning algorithm using meta-trained local learning rules.
result Improved meta-generalization and state-of-the-art results in robotics benchmarks.
Unified framework improves meta-learning generalization bounds.
problem Limited sharpness of existing meta-generalization bounds.
method Unified information-theoretic derivation for single-step bounds.
result Unified bounds exhibit tighter scaling and computational advantages.
Meta-learning priors improves safe Bayesian optimization.
problem Optimizing robot controllers under safety constraints.
method Meta-learning priors from offline data using F-PACOH.
result Meta-learned priors accelerate safe BO convergence.
Study MAML's generalization in varying tasks, proving bounds on error.
problem Bounding MAML's generalization error across tasks.
method Characterizes MAML's generalization error from two perspectives: recurring and unseen tasks.
result MAML's generalization error depends on the number of tasks and samples per task.
BOML unifies meta-learning methods into a common bilevel optimization framework.
problem Meta-learning methods with diverse modeling aspects.
method Modularized bilevel optimization library in Python.
result Unified solution for various meta-learning formulations.
New algorithm for online meta-learning with task boundary detection.
problem Adapting to new tasks in a non-stationary environment.
method Two detection mechanisms for task switches and distribution shift; online model updates based on current data.
result Achieves sublinear task-averaged regret under mild conditions.
A meta-learning approach for efficient algorithm selection in budget-limited scenarios.
problem Efficiently selecting the best-performing machine learning algorithm with limited computational resources.
method A Markov Decision Process framework where an agent decides whether to train, wake up, or start new algorithms based on partial learning curves.
result Meta-learning from learning curves improves algorithm selection, especially when learning curves do not intersect frequently.
This paper bounds meta-generalization gap using information theory.
problem Improving sample efficiency for new tasks in meta-learning.
method Information-theoretic upper bounds on meta-generalization gap for two meta-learning classes.
result Novel ITMI bounds for noisy iterative algorithms.
Meta-learning curiosity algorithms improves exploration across various tasks.
problem Generating curious behavior in reinforcement learning.
method Meta-learning approach to adapt reward signals dynamically.
result Two novel curiosity algorithms outperform human-designed ones.
Meta-learning method for accurate classifier from noisy annotators' data.
problem Accurate learning from noisy labels provided by multiple annotators.
method Meta-learning neural network to embed examples in latent space and estimate annotators' abilities, then adapt classifiers using EM algorithm.
result Meta-learning method improves classifier performance with minimal labeled data.
VSML unifies meta learning concepts and enables simple backpropagation.
problem Improving and unifying meta learning concepts for neural networks.
method Unified approach using variable shared meta learning and simple weight-sharing.
result Simple backpropagation can be implemented and meta learned without gradient calculation.
Improves meta-learning efficiency with mixed-mode differentiation.
problem Efficiently calculating complex derivatives in meta-learning.
method Mixed-Flow Meta-Gradients (MixFlow-MG) for scalable differentiation.
result Significant memory and time improvements in meta-learning tasks.
We introduce a new, rigorously-formulated Bayesian meta-learning algorithm that learns a probability distribution of model parameter prior for few-shot learning. The proposed algorithm employs a gradient-based variational inference to infer the posterior of model parameters to a new task. Our algorithm can be applied t…
The ability to learn new concepts with small amounts of data is a critical aspect of intelligence that has proven challenging for deep learning methods. Meta-learning has emerged as a promising technique for leveraging data from previous tasks to enable efficient learning of new tasks. However, most meta-learning algor…
New PAC-Bayesian bounds explain few-shot learning performance gaps.
problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.
The constant introduction of standardized benchmarks in the literature has helped accelerating the recent advances in meta-learning research. They offer a way to get a fair comparison between different algorithms, and the wide range of datasets available allows full control over the complexity of this evaluation. Howev…
Meta-learning performance improved by keeping support sets fixed.
problem Effect of support set diversity on meta-learning performance.
method Fixed support sets across tasks in meta-learning.
result Not only does not reduce performance, but almost always improves it.
Meta-learning improves performance in stochastic linear bandits.
problem Selecting a learning algorithm that performs well across multiple bandit tasks.
method Regularized OFUL algorithm with a bias vector, estimating bias within the learning-to-learn setting.
result Meta-learning strategies improve performance when the number of tasks grows and task variance is small.
Proposes a method to evaluate meta-learning performance based on task similarity.
problem Meta-learning performance evaluation ignores task similarity, leading to biased results.
method Generative approach using Latent Dirichlet Allocation to analyze task similarity.
result The proposed method provides an insightful evaluation of meta-learning algorithms, matching common intuition.
Meta-learning is increasingly used to support the recommendation of machine learning algorithms and their configurations. Such recommendations are made based on meta-data, consisting of performance evaluations of algorithms on prior datasets, as well as characterizations of these datasets. These characterizations, also…