Pseudo-task augmentation boosts deep learning performance.
problem Improving performance on single-task learning problems.
method Formalizing pseudo-task augmentation as training with multiple decoders for each task, simulating closely-related tasks.
result Pseudo-task augmentation improves performance on single-task learning problems and when combined with multitask learning.
Multitask neural networks improve performance on industrial ADMET datasets.
problem Improving drug discovery with deep learning methods.
method Comparison of neural networks to baseline models, analysis of multitask learning effects.
result Multitask learning provides modest benefits over single-task models, especially for smaller datasets.
Research explores the trade-off between multi-task learning and multitasking in deep neural networks.
problem Trade-off between multi-task learning and multitasking in deep neural networks.
method Meta-learning algorithm to manage the trade-off between shared and separated representations.
result Agent successfully optimizes training strategy based on environment.
A method to learn data representations from multiple tasks is discussed.
problem Learning from multiple tasks and the theoretical advantage over independent task learning.
method Multitask representation learning, illustrated in linear feature learning.
result Conditions under which multitask representation learning is beneficial over independent task learning.
New method learns flexible layer ordering for deep multitask learning, improving performance.
problem Fixed parallel layer ordering limits multitask learning effectiveness.
method Develops soft layer ordering approach to learn flexible application of shared layers.
result Soft ordering outperforms parallel ordering across various domains.
Establishes connection between MTDNN and multitask GP, revealing weight correlation as key to task sharing.
problem Limited theoretical understanding of information sharing in MTDNN.
method Derives multitask GP kernels for MTDNN and MTBNN, showing shared hyper-parameters and last layer weights.
result Information sharing in MTDNN is due to weight correlation, not intermediate layer weights.
Analyzes how multitask learning improves deep neural networks' generalization.
problem Understanding how multitask learning enhances deep neural networks' generalization.
method Developed an analytic theory using statistical physics techniques for classification tasks.
result Multitask learning benefits from task alignment and noise characteristics.
Deep-AMTFL learns shared features across tasks while preventing negative transfer.
problem Preventing negative transfer in multi-task learning.
method Introduces an asymmetric autoencoder term to prevent unreliable predictors from influencing feature learning.
result Significantly outperforms existing models on benchmark datasets.
Unified multitask learning framework for mixed-type outcomes.
problem Difficulty in formulating a unified objective for tasks with different outcomes.
method Multitask transformation framework with shared sparsity, using deep neural networks and rank-based optimization.
result Improved prediction and variable selection across continuous, binary, and mixed outcomes.
Improved audio classification with limited labels using multitask and self-supervised learning.
problem Limited labeled data for audio classification.
method Multitask learning and self-supervised learning on unlabeled data.
result Significant improvement in performance (up to 6%) through multitask and self-supervised learning.
Deep learning model estimates multiple f0s, melodies, vocals, and bass lines from music.
problem Estimating f0s and other musical elements from polyphonic music.
method Multitask deep learning architecture trained on a large dataset.
result Multitask model outperforms single-task models.
Study improves drug prediction accuracy for pharmacokinetic parameters.
problem Limited accuracy in predicting pharmacokinetic parameters.
method Integrated transfer learning and multitask learning approach.
result Improved model generalization and predictive ability.
A new challenge and model for multitask NLP.
problem Improving general NLP models across multiple tasks.
method Natural Language Decathlon challenge and Multitask Question Answering Network (MQAN).
result MQAN achieves state-of-the-art performance on multiple tasks.
Paper introduces multitask neural networks for efficient stochastic control problems.
problem Infeasibility of simulating state variables in some stochastic control problems.
method Multitask neural networks with dynamic task balancing.
result Multitask neural networks outperform state-of-the-art approaches in derivatives pricing problems.
The study analyzes implicit biases in neural networks using backward error analysis.
problem Analyzing implicit biases in multitask and continual learning settings.
method Backward error analysis to compute implicit training biases, deriving modified losses with three terms.
result The conflict term, measuring gradient alignment, is a new quantity in continual learning.
L2MT learns multitask models from historical experience.
problem Identifying effective multitask models for specific problems.
method L2MT uses a graph neural network to learn task embeddings and an estimation function to predict relative test errors.
result L2MT effectively identifies suitable multitask models for new problems.
Improved emotional speech classification using deep learning.
problem Difficult to assess subtle emotional changes in speech.
method Deep Convolutional Generative Adversarial Networks (DCGAN) for unlabeled data, multitask learning.
result Speaker-independent classification performance improved to 43.88% on a 5-point scale and 49.80% on a 3-point scale.
A new method prunes deep networks in one go without specifying pruning levels.
problem Deep model compression to reduce model size and inference time.
method Learning a pruner network to identify and prune unnecessary filters from a pre-trained network.
result Pruned networks achieve comparable performance to unpruned ones, with significant reduction in model size.
The study creates benchmarks for clinical time series data to evaluate machine learning models.
problem Lack of publicly available benchmark data sets for healthcare research.
method Proposed four clinical prediction benchmarks using MIMIC-III data, evaluated various deep supervision and multitask training methods.
result Demonstrated the effectiveness of deep supervision, multitask training, and data-specific architectural modifications on neural models.
Improves deep learning optimization with a new stochastic gradient method.
problem Noisy and sparse gradients in deep learning optimization.
method Proposes a family of double adaptive stochastic gradient methods (DASGrad).
result Analyzes theoretical convergence improvements and empirical validation.
A new approach for multitask reinforcement learning improves data efficiency and stability.
problem Data inefficiency and instability in multitask reinforcement learning.
method Distral: a new approach that shares a distilled policy and constrains workers to stay close to it.
result Distral outperforms related methods in complex 3D environments and is more robust and stable.
New framework learns shared representation for multitask learning.
problem Improving multitask learning performance.
method Co-clustering for shared representation, conjugate gradient descent, generalized Sylvester equations.
result Systematically outperforms state-of-the-art multitask learning methods.
XMixup improves transfer learning accuracy by 1.9% with less training time.
problem Efficiently transfer knowledge from large source datasets to target tasks with small samples.
method Cross-domain Mixup technique that selects auxiliary samples from source datasets and augments training samples via mixup strategy.
result Improves accuracy by 1.9% on average over six real-world transfer learning datasets.
Deep CNN extracts features for text-prompted speaker verification.
problem Improving speaker verification using deep learning.
method Used a deep convolutional neural network with Max-Feature-Map activation function, trained on segmented digits of a passphrase.
result Achieved 2.85% EER on RSR2015 evaluation set, surpassing classic baseline systems.
Framework uses experience replay to prevent deep networks from forgetting past tasks.
problem Deep networks forget past tasks after learning new ones in sequential multitask learning.
method Generative model that couples current task with past learned tasks through a discriminative embedding space.
result Framework learns a shared abstract distribution across all tasks, preventing catastrophic forgetting.
Paper presents a stress prediction model for students using wearable data.
problem Predicting students' stress levels from wearable data is challenging.
method Used Auto-encoders and Multitask learning to predict stress from sensor data and covariates.
result Model improved stress prediction by 45.6% on StudentLife dataset.
Self-paced learning selects tasks in a human-like progression for better multitask machine learning.
problem Improving multitask machine learning performance through effective task selection.
method Iterative selection of most appropriate tasks, learning task parameters, and updating shared knowledge using a bi-convex loss function.
result Self-paced task selection outperforms baseline methods in various multitask learning scenarios.
MSOL learns hierarchical policies for multitask tasks with soft options.
problem Training hierarchical policies for multiple tasks with stability and flexibility.
method MSOL uses separate variational posteriors for each task, regularized by a shared prior, to avoid instabilities and fine-tune options for new tasks.
result MSOL significantly outperforms hierarchical and flat transfer-learning baselines.
Generative multitask learning mitigates confounders causing targets.
problem Unobserved confounders causing targets but not inputs.
method Generative multitask learning (GMTL) modifies inference objective to remove joint target influence.
result Improved robustness to target shift across multitask learning methods.
Multitask learning improves phenotyping in EHR data, but its benefits vary by phenotype complexity.
problem Improving phenotyping accuracy in EHR data using multitask learning.
method Investigated multitask learning for phenotyping rare and common phenotypes in EHR data using neural nets and logistic regression.
result Multitask learning with neural nets consistently outperforms single-task neural nets for rare phenotypes but underperforms for common phenotypes.
Model learns multiple tasks using visual and textual representations.
problem Training visual navigation agents for multiple tasks.
method Dual-Attention unit for task-invariant alignment of visual and textual representations.
result Model outperforms baselines on semantic goal navigation and embodied question answering.
Proposes a method to balance tasks in multitask learning with a single gradient step update.
problem Balancing tasks in multitask learning to avoid imbalance.
method Gradient-based meta-learning to balance tasks at the gradient level, training shared and task-specific layers separately.
result Achieves state-of-the-art performance on various multitask computer vision problems.
Improved molecular property prediction using multitask learning.
problem Predicting molecular properties from chemical data is challenging.
method Multitask learning applied to graph neural networks.
result Multitask learning significantly improves model performance and reduces variance.
Paper proposes a method to identify negative transfers in multitask learning using surrogate models.
problem Identifying subsets of source tasks that improve target task performance in multitask learning.
method Surrogate modeling to precompute multitask learning performances and approximate them with a linear regression model.
result The approach predicts negative transfers from multiple source tasks to target tasks more accurately than existing methods.
Diff-DAC uses deep neural networks for distributed reinforcement learning across multiple tasks.
problem Learning policies for multiple tasks with limited local data.
method Distributed actor-critic algorithm approximated by deep neural networks, with parameter diffusion.
result Diff-DAC outperforms previous distributed MRL approaches and even centralized methods.
The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.
problem Understanding the risk in semi-supervised multitask learning on Gaussian mixtures.
method Statistical physics methods applied to Gaussian mixture models.
result The study evaluates the performance gain of learning tasks together versus separately.
Proposes a deep network for causal inference from observational data.
problem Inferring individualized causal effects from observational data.
method Conceptualizes causal inference as multitask learning; uses deep multitask network with shared and outcome-specific layers; alleviates selection bias via propensity-dropout.
result Outperforms state-of-the-art methods in experiments on real-world observational data.
We learn multiple hypotheses for related tasks under a latent hierarchical relationship between tasks. We exploit the intuition that for domain adaptation, we wish to share classifier structure, but for multitask learning, we wish to share covariance structure. Our hierarchical model is seen to subsume several previous…
New algorithm improves multitask learning across diverse agents.
problem Performance degradation in decentralized learning with heterogeneous objectives.
method Developed an exact subspace diffusion algorithm for multitask learning over networks.
result The algorithm outperforms alternatives in noisy gradient approximations.
New approach for multitask learning over networks sharing a common latent feature.
problem Learning multiple tasks simultaneously in a distributed network.
method Assumes shared latent feature representation; develops distributed online algorithms.
result Unified framework for analyzing mean-square-error performance.
Solves a model for sudden problem-solving ability in deep learning.
problem Emergence of new problem-solving abilities in deep learning models.
method Solves a simple multi-linear model in a skill-basis, finding analytic expressions for emergence and scaling laws.
result Simple model captures sigmoidal emergence of multiple new skills in neural networks.
Massively multitask neural architectures provide a learning framework for drug discovery that synthesizes information from many distinct biological sources. To train these architectures at scale, we gather large amounts of data from public sources to create a dataset of nearly 40 million measurements across more than 2…
New multitask algorithm separates rare from frequent protein functions.
problem Challenging automated protein function prediction with unbalanced data.
method Uses dissimilarity information to separate rare class labels, unlike similarity-based approaches.
result Multitask label propagation algorithm performs best with dissimilarity matrix.
This paper proposes a new multitask learning method to better measure task relationships.
problem Learning multiple tasks independently loses information about task relationships.
method Jointly learns shared parameters and shared feature representations through a new strategy.
result The proposed method better measures task relationships and optimizes shared parameters.
Introduces MLM dataset for multitask learning across multiple languages and modalities.
problem Training and evaluating multitask systems on diverse data.
method Develops a new benchmark dataset with semantic data and modalities.
result Demonstrates challenges in generalizing on diverse data.
GNNs can learn multiple graph centrality measures from a single embedding.
problem Estimating network centrality measures from graph data.
method Training a GNN to refine a single set of multidimensional embeddings and decode them into multiple outputs.
result GNN achieves 89% accuracy on random instances with up to 128 vertices.
New algorithm REFUEL shows multitask representation learning is more sample-efficient in RL.
problem Understanding the benefit of representation learning in reinforcement learning.
method Developed REFUEL algorithm for multitask low-rank RL, analyzing both upstream and downstream tasks.
result Multitask representation learning is provably more sample-efficient than individual task learning.
A method for cold-start playlist recommendation using multitask learning.
problem Playlist recommendation for new users and songs.
method Flexible multitask learning method that minimizes a bipartite ranking loss and a classification loss.
result The method performs well in cold-start scenarios on real datasets.