Probabilistic active meta-learning improves data efficiency in robotics.
problem Data-efficient learning in robotics where data collection is expensive.
method Conceptualizing meta-learning with a probabilistic latent variable model for sequential task selection.
result Improves data efficiency compared to baselines on simulated robotic experiments.
A new method for efficient probabilistic meta-learning.
problem High-quality predictions with well-calibrated uncertainty estimates require large amounts of data.
method Amortised Inference in Bayesian Neural Networks (APOVI-BNN)
result The APOVI-BNN produces high-quality predictions with well-calibrated uncertainty estimates using significantly less data.
ACE models allow flexible conditioning and prediction of latent variables.
problem Lack of flexibility in conditioning and prediction of latent variables in probabilistic models.
method Introduces Amortized Conditioning Engine (ACE) that explicitly represents latent variables and allows runtime conditioning and prediction.
result ACE models outperform existing methods in diverse tasks like image completion, classification, Bayesian optimization, and simulation-based inference.
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains.…
This paper introduces a new framework for data efficient and versatile learning. Specifically: 1) We develop ML-PIP, a general framework for Meta-Learning approximate Probabilistic Inference for Prediction. ML-PIP extends existing probabilistic interpretations of meta-learning to cover a broad class of methods. 2) We i…
Meta-learn sparse Gaussian process inference for faster predictions.
problem Cubic computational cost of exact Gaussian process inference for many observations.
method Meta-learn sparse Gaussian process inference.
result Rapid prediction on new tasks with sparse Gaussian processes.
Meta-learning for few-shot learning entails acquiring a prior over previous tasks and experiences, such that new tasks be learned from small amounts of data. However, a critical challenge in few-shot learning is task ambiguity: even when a powerful prior can be meta-learned from a large number of prior tasks, a small d…
We introduce a unified probabilistic framework for solving sequential decision making problems ranging from Bayesian optimisation to contextual bandits and reinforcement learning. This is accomplished by a probabilistic model-based approach that explains observed data while capturing predictive uncertainty during the d…
F-PACOH improves meta-learners' reliability in uncertain regions.
problem Overconfident uncertainty estimates in meta-learning.
method Meta-learning priors as stochastic processes in function space, directly steering predictions towards high epistemic uncertainty.
result Significantly outperforms other meta-learners in Bayesian Optimization.
Meta-learning priors improves safe Bayesian optimization.
problem Optimizing robot controllers under safety constraints.
method Meta-learning priors from offline data using F-PACOH.
result Meta-learned priors accelerate safe BO convergence.
Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with non…
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
Meta-learning reformulated as Bayesian risk minimization.
problem Learning models to quickly adapt to new tasks from small datasets.
method Formalized meta-learning as Bayesian risk minimization, using a probabilistic framework to compute predictive distributions from posterior distributions of latent variables conditioned on contextual datasets.
result A novel Gaussian approximation for the posterior distribution that converges to maximum likelihood estimates and outperforms Neural Process on benchmark datasets.
Effective implementations of sampling-based probabilistic inference often require manually constructed, model-specific proposals. Inspired by recent progresses in meta-learning for training learning agents that can generalize to unseen environments, we propose a meta-learning approach to building effective and generali…
Meta-learning method for accurate classifier from noisy annotators' data.
problem Accurate learning from noisy labels provided by multiple annotators.
method Meta-learning neural network to embed examples in latent space and estimate annotators' abilities, then adapt classifiers using EM algorithm.
result Meta-learning method improves classifier performance with minimal labeled data.
New method learns priors for Bayesian neural networks from datasets.
problem Lack of prior beliefs in Bayesian deep learning.
method Amortised variational inference to learn priors from datasets.
result Flexible Bayesian neural networks for meta-learning and within-task minibatching.
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling. However, existing SG-MCMC schemes are not tailored to any specific probabilistic model, even a simple modification of the underlying dynamical system requires signifi…
Meta-active learning optimizes control of safety-critical systems by efficiently learning dynamics and configurations.
problem Efficiently learning system dynamics and optimal configurations for safety-critical systems like deep brain stimulation.
method Meta-learning an acquisition function using LSTM, cast as meta-learning, with a mixed-integer linear program policy.
result Achieved a 46% increase in information gain and a 20% speedup in computation time over baselines.
Meta-learning adapts models for unseen tasks across AI, robotics, and NLP.
problem Adapting models to unseen tasks efficiently and accurately.
method Black-box, metric-based, layered, and Bayesian approaches.
result Meta-learning enhances model generalization and adaptation to unseen tasks.
Unified framework for learning flexible probabilistic programs using DPP and PAC-Bayes bounds.
problem Learning and generalizing from complex probabilistic models.
method Unified DPP representation and PAC-Bayes bounds for stochastic programs.
result Improved performance and generalization prediction using flexible DPP model representations and learned complexity measures.
A concise review of recent few-shot meta-learning methods.
problem Mimicking human fast adaptation to new concepts based on prior knowledge.
method Categorized into four branches based on technical characteristics.
result Current challenges and future prospects identified.
In order to achieve state-of-the-art performance, modern machine learning techniques require careful data pre-processing and hyperparameter tuning. Moreover, given the ever increasing number of machine learning models being developed, model selection is becoming increasingly important. Automating the selection and tuni…
A new meta-learning framework that assigns weights to source tasks based on target samples.
problem Learning initialization for target tasks with limited labeled examples.
method A general framework that assigns weights to the loss of different source tasks, which can depend on the target samples. Provides upper bounds and develops a learning algorithm based on minimizing the error bound with respect to an empirical IPM.
result Empirically, the weighted meta-learning algorithm finds better initializations than uniformly-weighted meta-learning algorithms.
TaskNorm improves meta-learning performance by rethinking batch normalization.
problem Challenges in batch normalization for meta-learning with deep networks.
method Developed TaskNorm, a novel approach to batch normalization for meta-learning.
result TaskNorm consistently improves meta-learning performance across various datasets and meta-learning approaches.
Meta-learning algorithms produce feature extractors which achieve state-of-the-art performance on few-shot classification. While the literature is rich with meta-learning methods, little is known about why the resulting feature extractors perform so well. We develop a better understanding of the underlying mechanics of…
Meta-learning balances task-specific modeling and optimization complexity.
problem Balancing accurate task-specific modeling with ease of optimization in meta-learning.
method Theoretical and empirical analysis of trade-off between modeling and optimization in meta-learning.
result Explicit bounds on modeling and optimization errors for non-convex and linear regression problems.
Meta-learning bounds derived using PAC-Bayes theory for improved generalization.
problem Uncertainty in generalization performance for meta-learning with new tasks.
method PAC-Bayes relative entropy bounds and empirical risk minimization (ERM) method.
result Competitive generalization performance and rapid convergence with data-dependent prior.
Meta-learning improves neural networks by adapting learning algorithms.
problem Conventional AI approaches solve tasks from scratch, but meta-learning aims to improve the learning algorithm.
method Meta-learning adapts a learning algorithm based on multiple learning episodes.
result Meta-learning can tackle deep learning challenges like data and computation bottlenecks.
Meta learning works well with overparameterized models, a phenomenon called 'benign overfitting'.
problem Understanding why overparameterized models perform well in few-shot learning.
method Analyzed the generalization performance of gradient-based meta learning with an overparameterized meta linear regression model.
result Demonstrated that overparameterized meta learning can still generalize well, a phenomenon called 'benign overfitting'.
Meta-learning helps models learn quickly from few samples.
problem Deep learning requires many samples, which are hard to get.
method Meta-learning optimizes models to adapt quickly to new tasks.
result Meta-learning can improve model efficiency and adaptability.
When fitting Bayesian machine learning models on scarce data, the main challenge is to obtain suitable prior knowledge and encode it into the model. Recent advances in meta-learning offer powerful methods for extracting such prior knowledge from data acquired in related tasks. When it comes to meta-learning in Gaussian…
Meta learning can adapt fast but is vulnerable to adversarial attacks.
problem Vulnerability of meta learning to adversarial attacks.
method Formal definition of adversarial attacks unique to meta learning, proposing an attacking algorithm.
result Meta learning is vulnerable to adversarial attacks.
The goal of optimization-based meta-learning is to find a single initialization shared across a distribution of tasks to speed up the process of learning new tasks. Conditional meta-learning seeks task-specific initialization to better capture complex task distributions and improve performance. However, many existing c…
BOML unifies meta-learning methods into a common bilevel optimization framework.
problem Meta-learning methods with diverse modeling aspects.
method Modularized bilevel optimization library in Python.
result Unified solution for various meta-learning formulations.
learn2learn simplifies meta-learning research by providing a library and standardized interfaces.
problem Prototyping and reproducibility issues in meta-learning.
method Developed a library (learn2learn) with common routines and standardized interfaces.
result Fosters a community around standardized software for meta-learning research.
MARS meta-learns function scores for improved predictive accuracy and uncertainty.
problem Difficulty in specifying expressive priors for Bayesian meta-learning.
method Meta-learning the score function of data-generating process marginals in the function space.
result State-of-the-art predictive accuracy and improved uncertainty estimates.
New PAC-Bayesian framework for flexible meta-learning.
problem Limitations in transferring knowledge between tasks in meta-learning.
method PAC-Bayesian theory applied to learning the learning algorithm.
result Flexibility in meta-learning mechanisms and improved prediction quality.
Meta-learning algorithm improves AI efficiency by teaching itself.
problem Overcoming meta-optimization challenges in AI learning.
method Bootstraps a target from the meta-learner and optimizes the meta-learner by minimizing distance under a chosen metric.
result Achieves state-of-the-art performance on Atari ALE benchmark and demonstrates efficiency gains in multi-task learning.
Machine learning approaches have seen considerable applications in human movement modeling, but remain limited for motor learning. Motor learning requires accounting for motor variability, and poses new challenges as the algorithms need to be able to differentiate between new movements and variation of known ones. In t…
Self-referential meta learning avoids explicit optimization by modifying itself.
problem Dependency on human engineering in meta learning algorithms.
method Investigates self-referential meta learning systems that modify themselves without explicit optimization.
result Self-referential neural networks can improve their own modifications without explicit optimization.
VMGP extends Gaussian processes for Bayesian meta-learning, improving uncertainty prediction.
problem Bayesian meta-learning for few-shot tasks with non-Gaussian uncertainty.
method VMGP (Variational Meta-Gaussian Processes) extends Gaussian processes to model non-Gaussian predictive posteriors.
result VMGP significantly outperforms existing Bayesian meta-learning methods on complex tasks.
Meta-learning performance improved by keeping support sets fixed.
problem Effect of support set diversity on meta-learning performance.
method Fixed support sets across tasks in meta-learning.
result Not only does not reduce performance, but almost always improves it.
VSML unifies meta learning concepts and enables simple backpropagation.
problem Improving and unifying meta learning concepts for neural networks.
method Unified approach using variable shared meta learning and simple weight-sharing.
result Simple backpropagation can be implemented and meta learned without gradient calculation.
Proposes a model combining graph networks and variational Bayes for graph data.
problem Probabilistic modeling of graph structured data.
method Combines graph networks and variational Bayes for probabilistic modeling of graph data.
result Demonstrates effectiveness on wind farm monitoring and Gaussian Process data.
Bayesian MAML outperforms MAML in meta learning tasks with theoretical guarantees.
problem Theoretical understanding of Bayesian MAML's superiority over MAML.
method Comparison of meta test risks between Bayesian MAML and MAML in meta linear regression.
result Bayesian MAML has provably lower meta test risks than MAML in both distribution agnostic and linear centroid cases.
Meta-learning can perform well on non-convex models even with few samples, contrary to convex models.
problem Understanding the sample complexity of meta-learning for non-convex models.
method Constructing a simple meta-learning instance and analyzing the training dynamics of Reptile and multi-task representation learning.
result Meta-learning can achieve new task sample complexity of O(1) for non-convex models, unlike convex models which require Ω(d) samples. Conditional meta-learning improves meta-learning performance in diverse task environments.
problem Meta-learning struggles with tasks that have heterogeneous complexity.
method Conditional meta-learning infers a task-specific meta-parameter vector.
result Conditional meta-learning outperforms standard meta-learning in diverse task environments.
Meta-learning improves few-shot acoustic event detection.
problem Detecting new audio events with limited labeled data.
method Formulated few-shot AED problem; explored supervised and meta-learning approaches.
result Meta-learning achieves superior performance in few-shot AED.