Framework uses experience replay to prevent deep networks from forgetting past tasks.
problem Deep networks forget past tasks after learning new ones in sequential multitask learning.
method Generative model that couples current task with past learned tasks through a discriminative embedding space.
result Framework learns a shared abstract distribution across all tasks, preventing catastrophic forgetting.
Multitask learning has shown promising performance in many applications and many multitask models have been proposed. In order to identify an effective multitask model for a given multitask problem, we propose a learning framework called learning to multitask (L2MT). To achieve the goal, L2MT exploits historical multit…
Deep learning methods such as multitask neural networks have recently been applied to ligand-based virtual screening and other drug discovery applications. Using a set of industrial ADMET datasets, we compare neural networks to standard baseline models and analyze multitask learning effects with both random cross-valid…
This paper presents a new multitask learning framework that learns a shared representation among the tasks, incorporating both task and feature clusters. The jointly-induced clusters yield a shared latent subspace where task relationships are learned more effectively and more generally than in state-of-the-art multitas…
MSOL learns hierarchical policies for multitask tasks with soft options.
problem Training hierarchical policies for multiple tasks with stability and flexibility.
method MSOL uses separate variational posteriors for each task, regularized by a shared prior, to avoid instabilities and fine-tune options for new tasks.
result MSOL significantly outperforms hierarchical and flat transfer-learning baselines.
Generative multitask learning mitigates confounders causing targets.
problem Unobserved confounders causing targets but not inputs.
method Generative multitask learning (GMTL) modifies inference objective to remove joint target influence.
result Improved robustness to target shift across multitask learning methods.
Proposes a method to balance tasks in multitask learning with a single gradient step update.
problem Balancing tasks in multitask learning to avoid imbalance.
method Gradient-based meta-learning to balance tasks at the gradient level, training shared and task-specific layers separately.
result Achieves state-of-the-art performance on various multitask computer vision problems.
We discuss a general method to learn data representations from multiple tasks. We provide a justification for this method in both settings of multitask learning and learning-to-learn. The method is illustrated in detail in the special case of linear feature learning. Conditions on the theoretical advantage offered by m…
Improved molecular property prediction using multitask learning.
problem Predicting molecular properties from chemical data is challenging.
method Multitask learning applied to graph neural networks.
result Multitask learning significantly improves model performance and reduces variance.
Paper proposes a method to identify negative transfers in multitask learning using surrogate models.
problem Identifying subsets of source tasks that improve target task performance in multitask learning.
method Surrogate modeling to precompute multitask learning performances and approximate them with a linear regression model.
result The approach predicts negative transfers from multiple source tasks to target tasks more accurately than existing methods.
This paper introduces self-paced task selection to multitask learning, where instances from more closely related tasks are selected in a progression of easier-to-harder tasks, to emulate an effective human education strategy, but applied to multitask machine learning. We develop the mathematical foundation for the appr…
Research explores the trade-off between multi-task learning and multitasking in deep neural networks.
problem Trade-off between multi-task learning and multitasking in deep neural networks.
method Meta-learning algorithm to manage the trade-off between shared and separated representations.
result Agent successfully optimizes training strategy based on environment.
Establishes connection between MTDNN and multitask GP, revealing weight correlation as key to task sharing.
problem Limited theoretical understanding of information sharing in MTDNN.
method Derives multitask GP kernels for MTDNN and MTBNN, showing shared hyper-parameters and last layer weights.
result Information sharing in MTDNN is due to weight correlation, not intermediate layer weights.
Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task.…
Electronic phenotyping is the task of ascertaining whether an individual has a medical condition of interest by analyzing their medical record and is foundational in clinical informatics. Increasingly, electronic phenotyping is performed via supervised learning. We investigate the effectiveness of multitask learning fo…
The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.
problem Understanding the risk in semi-supervised multitask learning on Gaussian mixtures.
method Statistical physics methods applied to Gaussian mixture models.
result The study evaluates the performance gain of learning tasks together versus separately.
We learn multiple hypotheses for related tasks under a latent hierarchical relationship between tasks. We exploit the intuition that for domain adaptation, we wish to share classifier structure, but for multitask learning, we wish to share covariance structure. Our hierarchical model is seen to subsume several previous…
Background: Pharmacokinetic evaluation is one of the key processes in drug discovery and development. However, current absorption, distribution, metabolism, excretion prediction models still have limited accuracy. Aim: This study aims to construct an integrated transfer learning and multitask learning approach for deve…
New algorithm improves multitask learning across diverse agents.
problem Performance degradation in decentralized learning with heterogeneous objectives.
method Developed an exact subspace diffusion algorithm for multitask learning over networks.
result The algorithm outperforms alternatives in noisy gradient approximations.
Massively multitask neural architectures provide a learning framework for drug discovery that synthesizes information from many distinct biological sources. To train these architectures at scale, we gather large amounts of data from public sources to create a dataset of nearly 40 million measurements across more than 2…
This paper proposes a new multitask learning method to better measure task relationships.
problem Learning multiple tasks independently loses information about task relationships.
method Jointly learns shared parameters and shared feature representations through a new strategy.
result The proposed method better measures task relationships and optimizes shared parameters.
Introduces MLM dataset for multitask learning across multiple languages and modalities.
problem Training and evaluating multitask systems on diverse data.
method Develops a new benchmark dataset with semantic data and modalities.
result Demonstrates challenges in generalizing on diverse data.
Improved audio classification with limited labels using multitask and self-supervised learning.
problem Limited labeled data for audio classification.
method Multitask learning and self-supervised learning on unlabeled data.
result Significant improvement in performance (up to 6%) through multitask and self-supervised learning.
New algorithm REFUEL shows multitask representation learning is more sample-efficient in RL.
problem Understanding the benefit of representation learning in reinforcement learning.
method Developed REFUEL algorithm for multitask low-rank RL, analyzing both upstream and downstream tasks.
result Multitask representation learning is provably more sample-efficient than individual task learning.
The paper provides a theoretical framework for learning task similarity in multitask learning.
problem Understanding and leveraging task similarities for improving multitask learning performance.
method The paper derives theoretical bounds on multitask learning generalization error, and proposes a new training algorithm to learn task relation coefficients and neural network parameters.
result The proposed algorithm outperforms baselines and reaffirms the benefits of theoretical insight in algorithm design.
Analyzes how multitask learning improves deep neural networks' generalization.
problem Understanding how multitask learning enhances deep neural networks' generalization.
method Developed an analytic theory using statistical physics techniques for classification tasks.
result Multitask learning benefits from task alignment and noise characteristics.
Paper introduces multitask neural networks for efficient stochastic control problems.
problem Infeasibility of simulating state variables in some stochastic control problems.
method Multitask neural networks with dynamic task balancing.
result Multitask neural networks outperform state-of-the-art approaches in derivatives pricing problems.
Multitask Gaussian process regression reduces data generation costs for molecular property prediction.
problem Data bottleneck in training surrogate models for molecular properties.
method Multitask Gaussian process regression over heterogeneous data sources (CC and DFT).
result Predicts at CC-level accuracy with over an order of magnitude reduction in data generation cost.
Unified multitask learning framework for mixed-type outcomes.
problem Difficulty in formulating a unified objective for tasks with different outcomes.
method Multitask transformation framework with shared sparsity, using deep neural networks and rank-based optimization.
result Improved prediction and variable selection across continuous, binary, and mixed outcomes.
The study analyzes implicit biases in neural networks using backward error analysis.
problem Analyzing implicit biases in multitask and continual learning settings.
method Backward error analysis to compute implicit training biases, deriving modified losses with three terms.
result The conflict term, measuring gradient alignment, is a new quantity in continual learning.
A new multitask model using Hopfield Networks improves classification performance.
problem Improving classification models by sharing information across multiple tasks.
method Developed a transductive semi-supervised parametric Hopfield Network (HoMTask) that minimizes an energy function for all tasks.
result HoMTask achieves competitive performance compared to state-of-the-art semi-supervised graph-based algorithms.
Optimal multitask learning method for sparse heterogeneous datasets.
problem Efficiently learning from multiple related datasets with sparse task-specific differences.
method MOLAR estimator, combining weighted median and shrinkage.
result Improves estimation error dependence on data dimension compared to task-wise least squares.
Study on multitask learning performance factors.
problem Mixed results in multitask learning performance.
method Task simulator and symbolic regression to learn performance factors.
result Empirical formulas relating model performance to sqrt(n), sqrt(T), and sqrt(AMI).
A new approach simplifies multitask Gaussian processes without rank approximations.
problem Handling multioutput regression problems with conditionally dependent tasks.
method Introduces a novel approach to reduce multitask learning to univariate GPs, eliminating the need for rank approximations.
result Accurately recovers multitask covariance and noise matrices with fewer parameters, improving performance and reducing overfitting risk.
Scalable GP model handles functional covariates and multitasks.
problem Uncertainty quantification in complex mechanical systems with time-dependent inputs.
method Introduced a fully separable kernel structure for functional covariates and multitask problems, leveraging Kronecker structure for scalability.
result The model significantly improves over single task GPs, requiring fewer samples for accurate predictions.
Fundamental frequency (f0) estimation from polyphonic music includes the tasks of multiple-f0, melody, vocal, and bass line estimation. Historically these problems have been approached separately, and only recently, using learning-based approaches. We present a multitask deep learning architecture that jointly estimate…
Bayesian model connects KMs and ELMs for multitask regression.
problem Multitask regression with shared features and sparsity.
method Bayesian framework using RFFs and RBF kernels.
result Significant performance improvements over state-of-the-art methods.
Automated protein function prediction is a challenging problem with distinctive features, such as the hierarchical organization of protein functions and the scarcity of annotated proteins for most biological functions. We propose a multitask learning algorithm addressing both issues. Unlike standard multitask algorithm…
We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning. The central assumption of our learning method is that the tasks parameters are well approximated by sparse linear combinations of the atoms of a dictionary on a high or infinite dimensional space. This ass…
Online learning with streaming data in a distributed and collaborative manner can be useful in a wide range of applications. This topic has been receiving considerable attention in recent years with emphasis on both single-task and multitask scenarios. In single-task adaptation, agents cooperate to track an objective o…
Estimates shared parameters across related learning problems using robust statistics and LASSO.
problem Simultaneously learning related but heterogeneous problems like store demand or patient risk.
method Two-stage multitask learning estimator combining robust statistics and LASSO regression.
result Improved sample complexity bounds for multitask learning, especially beneficial for 'data-poor' instances.
The paper improves Gaussian processes by adding sum constraints, enhancing prediction accuracy.
problem Improving Gaussian process predictions with background knowledge constraints.
method Conditioning the prior distribution on sum constraints to ensure fulfillment of linear and nonlinear constraints.
result The approach fulfills constraints with high precision and improves prediction accuracy.
Study evaluates task selection policies for multitask learning.
problem How to allocate training resources among multiple tasks.
method Empirical evaluation of task selection policies in synthetic and real-world settings.
result Improved model performance through counterfactual estimation.
New methods improve autotuning of exascale applications by 1.5x.
problem Finding optimal performance parameters for exascale applications.
method Multitask and transfer learning for autotuning.
result Average 1.5x improvement in application runtime.
We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process. Specifically, we introduce an asymmetric autoencoder term that allows reliable predictors fo…
Future projection of climate is typically obtained by combining outputs from multiple Earth System Models (ESMs) for several climate variables such as temperature and precipitation. While IPCC has traditionally used a simple model output average, recent work has illustrated potential advantages of using a multitask lea…
From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived. It is applied to sharpen existing and to derive novel bounds on the empirical Rademacher complexities of unit balls in various norms appearing in the context of structured sparsity and multitask dictionary lear…
Neural networks improve Lorenz trajectory prediction.
problem Predicting the Lorenz dynamical system's future states.
method Multitask learning to improve prediction of the z trajectory.
result Neural networks enhance forecasting performance.