Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

1122 · Nov 201919922001200920172026
14 results for Reptile

Meta-learning techniques like MAML and Reptile fail to generalize well to out-of-distribution tasks compared to simple finetuning.

problem Generalization of meta-learning techniques in low-data scenarios.
method Investigation of MAML, Reptile, and finetuning on various tasks.
result MAML and Reptile specialize for fast adaptation but fail to generalize to out-of-distribution tasks.

Generative Adversarial Networks (GAN) boast impressive capacity to generate realistic images. However, like much of the field of deep learning, they require an inordinate amount of data to produce results, thereby limiting their usefulness in generating novelty. In the same vein, recent advances in meta-learning have o…

2019-01-08abs ↗pdf ↗

Meta-learning can perform well on non-convex models even with few samples, contrary to convex models.

problem Understanding the sample complexity of meta-learning for non-convex models.
method Constructing a simple meta-learning instance and analyzing the training dynamics of Reptile and multi-task representation learning.
result Meta-learning can achieve new task sample complexity of O(1)\mathcal{O}(1) for non-convex models, unlike convex models which require Ω(d)Ω(d) samples.

New PAC-Bayesian bounds explain few-shot learning performance gaps.

problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.

The recent progress in neural architecture search (NAS) has allowed scaling the automated design of neural architectures to real-world domains, such as object detection and semantic segmentation. However, one prerequisite for the application of NAS are large amounts of labeled data and compute resources. This renders i…

2019-11-25abs ↗pdf ↗

This work proposes a meta-learning approach for better adaptation of source code models.

problem Adapting source code models to unseen local contexts.
method Formulated as a meta-learning problem, selecting targeted information for adaptation.
result Improved performance in code auto-completion tasks, especially for identifiers and literals.

We extend first-order model agnostic meta-learning algorithms (including FOMAML and Reptile) to image segmentation, present a novel neural network architecture built for fast learning which we call EfficientLab, and leverage a formal definition of the test error of meta-learning algorithms to decrease error on out of d…

2019-12-13abs ↗pdf ↗

Many real-world problems, including multi-speaker text-to-speech synthesis, can greatly benefit from the ability to meta-learn large models with only a few task-specific components. Updating only these task-specific modules then allows the model to be adapted to low-data tasks for as many steps as necessary without ris…

2019-09-12abs ↗pdf ↗

Meta-learning algorithms are studied for their data efficiency in personalized federated learning.

problem Data efficiency in meta-learning algorithms.
method Algorithmic stability techniques to derive bounds on transfer risk and a new evaluation framework.
result Different meta-learning methods require varying amounts of supervision for effective generalization.

This paper presents a novel optimization method for maximizing generalization over tasks in meta-learning. The goal of meta-learning is to learn a model for an agent adapting rapidly when presented with previously unseen tasks. Tasks are sampled from a specific distribution which is assumed to be similar for both seen …

2018-10-18abs ↗pdf ↗