arXiv research

Boosts share routing for multi-task learning with flexible sparse connections.

Algorithm learns which weights to share in deep multi-task learning.

Improves shared encoder representations for better multi-task learning performance.

We study the problem of distributed multi-task learning with shared representation, where each machine aims to learn a separate, but related, task in an unknown shared low-dimensional subspaces, i.e. when the predictor matrix has low rank. We consider a setting where each task is handled by a different machine, with sa…

Unified MTL framework for heterogeneous data integrates shared and task-specific encoders.

Meta-RL learns shared and task-specific information for quick adaptation.

A new method for multi-task learning by allocating parameters.

DGPs learn from multiple tasks using shared and private latent processes.

UCB-TQL learns from multiple tasks with shared dynamics and adapts to task-specific variations.

The paper extends deep multi-task learning to diverse domains, finding shared functionality.

Multitask learning (MTL) aims to learn multiple tasks simultaneously through the interdependence between different tasks. The way to measure the relatedness between tasks is always a popular issue. There are mainly two ways to measure relatedness between tasks: common parameters sharing and common features sharing acro…

Multi-task learning (MTL) allows deep neural networks to learn from related tasks by sharing parameters with other networks. In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amoun…

Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tas…

Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering. Such an organization significantly constricts the types of shared structure that can be learned. The necessity of parallel ordering for deep MTL is first tested by comparing it with permuted ordering of shared la…

New algorithms minimize regret in multi-task and lifelong linear bandits with shared representation.

FinCausal 2020 task detects financial document causality.

First-order ANIL learns shared representations even with overparametrization.

MT-HAL learns features and task associations for multiple tasks with a shared sparse structure.

In the paradigm of multi-task learning, mul- tiple related prediction tasks are learned jointly, sharing information across the tasks. We propose a framework for multi-task learn- ing that enables one to selectively share the information across the tasks. We assume that each task parameter vector is a linear combi- nat…

We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process. Specifically, we introduce an asymmetric autoencoder term that allows reliable predictors fo…

This paper considers the multi-task learning problem and in the setting where some relevant features could be shared across few related tasks. Most of the existing methods assume the extent to which the given tasks are related or share a common feature space to be known apriori. In real-world applications however, it i…

Most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where different tasks share a set of actions. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound polic…

New method learns shared structures in non-linear tasks.

Unified multitask learning framework for mixed-type outcomes.

Establishes connection between MTDNN and multitask GP, revealing weight correlation as key to task sharing.

Survey on multi-task learning for deep neural networks.

Near-optimal rates for multi-task learning with shared representations.

Spectral algorithm reduces samples needed for multitask regression.

CNNs require fewer samples than LCNs and FCNs for image-based tasks due to locality and weight sharing.

We introduce and describe the results of a novel shared task on bandit learning for machine translation. The task was organized jointly by Amazon and Heidelberg University for the first time at the Second Conference on Machine Translation (WMT 2017). The goal of the task is to encourage research on learning machine tra…

Training-free source selection for LLM families with shared vocabularies

A framework isolates and learns approximately shared features for better domain adaptation.

New method uses shared attention for multi-task time series forecasting.

Hybrid framework prevents forgetting in continual learning.

Proposes a method to balance tasks in multitask learning with a single gradient step update.

While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, some issues remain to be tackled to apply them to real-world problem domains. First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number o…

Automated multi-task learning algorithm that optimizes network topology.

Research explores the trade-off between multi-task learning and multitasking in deep neural networks.

We learn multiple hypotheses for related tasks under a latent hierarchical relationship between tasks. We exploit the intuition that for domain adaptation, we wish to share classifier structure, but for multitask learning, we wish to share covariance structure. Our hierarchical model is seen to subsume several previous…

Bayesian neural networks improve knowledge sharing across networks.

Multi-task learning shares information between related tasks, sometimes reducing the number of parameters required. State-of-the-art results across multiple natural language understanding tasks in the GLUE benchmark have previously used transfer from a single large task: unsupervised pre-training with BERT, where a sep…

We investigated the training of a shared model for both text-to-speech (TTS) and voice conversion (VC) tasks. We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks. This model can a…

Modern discriminative predictors have been shown to match natural intelligences in specific perceptual tasks in image classification, object and part detection, boundary extraction, etc. However, a major advantage that natural intelligences still have is that they work well for "all" perceptual problems together, solvi…

Enhances generative model accuracy through knowledge transfer.

Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task.…

New model extracts shared brain activity patterns from fMRI data.

GeoERM learns shared representations on Riemannian manifolds for multi-task learning.

The incorporation of prior knowledge into learning is essential in achieving good performance based on small noisy samples. Such knowledge is often incorporated through the availability of related data arising from domains and tasks similar to the one of current interest. Ideally one would like to allow both the data f…