In multi-task learning, multiple tasks are solved jointly, sharing inductive bias between them. Multi-task learning is inherently a multi-objective problem because different tasks may conflict, necessitating a trade-off. A common compromise is to optimize a proxy objective that minimizes a weighted linear combination o…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “multi-task objectives”
New research shows Multi-Task Learning does not behave like Multi-Objective Optimization.
problem Applying MOO methods to MTL problems is not equivalent.
method Comparing MTL and MOO approaches on Multi-Fashion-MNIST datasets.
result A single model can perform as well as optimizing multiple objectives.
Small parameterized towers improve multi-task learning efficiency and generalization.
problem Balancing Pareto efficiency and generalization in multi-task learning.
method Under-parameterized self-auxiliaries for multi-task models.
result Small parameterized towers enhance Pareto efficiency in various multi-task applications.
New algorithms optimize multiple tasks with shared similarities, reducing regret.
problem Optimizing multiple objectives with shared similarities in non-parametric Bayesian optimization.
method Developed two novel BO algorithms using multi-task kernels and random scalarizations.
result Derived worst-case regret bounds capturing inter-task similarities.
Proposes a simple framework to balance task difficulty in multi-task learning.
problem Varying difficulty levels among different tasks in multi-task learning.
method Introduces a Balanced Multi-Task Learning (BMTL) framework that transforms training losses to balance task difficulty.
result Empirical studies show state-of-the-art performance of the proposed BMTL framework.
New method generates continuous Pareto sets for multi-task learning.
problem Challenges in finding optimal solutions for correlated multi-task learning problems.
method Efficiently generates locally continuous Pareto sets and fronts in multi-objective optimization problems.
result Demonstrates continuous analysis of Pareto optimal solutions in machine learning problems.
Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approachcs.LG
Develops a new method to improve performance in multi-objective learning problems.
problem Gradient bias in multi-objective learning leading to degraded performance.
method Stochastic Multi-objective gradient Correction (MoCo) method that guarantees convergence without increasing batch size.
result Demonstrates effectiveness of MoCo method in simulations on multi-task learning.
The present work investigates whether different quantification mechanisms (set comparison, vague quantification, and proportional estimation) can be jointly learned from visual scenes by a multi-task computational model. The motivation is that, in humans, these processes underlie the same cognitive, non-symbolic abilit…
This paper tackles multilingual speech processing by optimizing conflicting objectives hierarchically.
problem Training models for multilingual, multi-task speech processing is hampered by conflicting objectives.
method Investigates three multi-objective MSP formulations and introduces a lightweight layer-selection mechanism.
result A bi-level recipe outperforms standard flat optimization in state-of-the-art MSP models.
Adaptive data fusion boosts efficiency in multi-task optimization.
problem Multi-task non-smooth optimization in various fields.
method Adaptive data fusion approach leveraging commonalities among objectives.
result Significant improvements in sample efficiency with sharp statistical guarantees.
Proposes new stochastic algorithms for multi-objective optimization.
problem Multi-objective optimization in machine learning problems.
method Direction-oriented multi-objective formulation and Stochastic Direction-oriented Multi-objective Gradient descent (SDMGrad).
result Stochastic algorithms converge to Pareto stationary points with improved complexities.
Currently there is no validated objective measure of pain. Recent neuroimaging studies have explored the feasibility of using functional near-infrared spectroscopy (fNIRS) to measure alterations in brain function in evoked and ongoing pain. In this study, we applied multi-task machine learning methods to derive a pract…
In important applications involving multi-task networks with multiple objectives, agents in the network need to decide between these multiple objectives and reach an agreement about which single objective to follow for the network. In this work we propose a distributed decision-making algorithm. The agents are assumed …
Bayesian approach improves network lasso for multi-task learning.
problem Improving the determination of relational coefficients in network lasso.
method Proposes a Bayesian approach to solve multi-task learning problems using network lasso.
result Objective determination of relational coefficients through Bayesian estimation.
Develops a method for fairness in multi-task learning using Wasserstein barycenters.
problem Extending fairness to multi-task learning with shared representations.
method Definition of Strong Demographic Parity extended to multi-task learning using multi-marginal Wasserstein barycenters. Closed form solution for optimal fair predictor.
result Empirical results show practical value of post-processing methodology in promoting fair decision-making.
A new framework integrates classification and regression tasks in multi-task learning.
problem Jointly solving classification and regression tasks in multi-task scenarios.
method Two-Stage Learning-to-Defer (L2D) framework with a unified deferral mechanism.
result Unified deferral mechanism ensures convergence to the Bayes-optimal rejector.
Multi-task learning is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other related tasks. In this paper, we propose a regularization formulation for learning the relationships between tasks in multi-task learning. This formulation can be viewed as a n…
SON-GOKU uses graph coloring to improve multi-task learning by partitioning tasks into compatible groups.
problem Gradient interference between conflicting multi-task learning objectives slows convergence and model performance.
method SON-GOKU computes gradient interference, constructs an interference graph, and applies greedy graph-coloring to partition tasks.
result SON-GOKU consistently outperforms baselines and state-of-the-art multi-task optimizers on six datasets.
New method uses shared attention for multi-task time series forecasting.
problem Insufficient training instances in single-task forecasting.
method Self-attention based sharing schemes across multiple tasks.
result Outperforms state-of-the-art single-task forecasting baselines and RNN-based multi-task forecasting method.
Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximizationmath.OC
Paper tackles multi-block min-max optimization with applications in deep AUC maximization.
problem Multi-block min-max bilevel optimization with non-convex strongly-concave upper level and strongly convex lower level.
method Single-loop randomized stochastic algorithm for constant number of blocks per iteration.
result Sample complexity of O(1/ε^4) for finding ε-stationary point, matching optimal complexity.
Proposes GTTN for discovering all low-rank structures in deep multi-task learning.
problem Discovering all low-rank structures among tasks in deep multi-task models.
method Introduces GTTN, a convex combination of matrix trace norms of all tensor flattenings, to automatically determine the importance of components.
result Demonstrates the effectiveness of GTTN on real-world datasets.
Method learns graph structure for multi-task learning, revealing interpretable relationships.
problem Learning relationships among tasks in multi-task learning.
method Simultaneously learns graph structure and model parameters, optimizing the graph structure with the model parameters.
result Reduces generalization error and reveals interpretable sparse graph among tasks.
Advance Prediction of Ventricular Tachyarrhythmias using Patient Metadata and Multi-Task Networkscs.LG
We describe a novel neural network architecture for the prediction of ventricular tachyarrhythmias. The model receives input features that capture the change in RR intervals and ectopic beats, along with features based on heart rate variability and frequency analysis. Patient age is also included as a trainable embeddi…
MAGMA uses a common mean process to improve multi-step-ahead time series forecasting.
problem Improving multiple-step-ahead predictions for time series data.
method Proposes a novel multi-task Gaussian process framework with a common mean process for sharing information across tasks.
result Significantly improves predictive performances, even far from observations, and reduces computational complexity.
Automated multi-task learning algorithm that optimizes network topology.
problem Over-sharing in multi-task learning leads to over-generalization and suboptimal performance.
method Tree-structured design space with gumbel-softmax sampling for differentiable network splitting.
result End-to-end trainable algorithm that optimizes network topology for multiple objectives across tasks.
The problem of domain generalization is to take knowledge acquired from a number of related domains where training data is available, and to then successfully apply it to previously unseen domains. We propose a new feature learning algorithm, Multi-Task Autoencoder (MTAE), that provides good generalization performance …
Develops a new fairness learning approach for multi-task regression models.
problem Fairness in multi-task regression models with biased datasets.
method Uses rank-based non-parametric independence test (Mann Whitney U statistic) and reformulates as non-convex optimization problem.
result Outperforms state-of-the-art methods on fairness metrics.
We consider multi-task learning, which simultaneously learns related prediction tasks, to improve generalization performance. We factorize a coefficient matrix as the product of two matrices based on a low-rank assumption. These matrices have sparsities to simultaneously perform variable selection and learn and overlap…
Conversion prediction plays an important role in online advertising since Cost-Per-Action (CPA) has become one of the primary campaign performance objectives in the industry. Unlike click prediction, conversions have different types in nature, and each type may be associated with different decisive factors. In this pap…
DMTG groups tasks for multi-task learning in one shot.
problem Efficiently grouping and training multiple tasks in machine learning.
method Formulates Multi-Task Grouping as a differentiable pruning problem, training all groups simultaneously.
result Significantly improves training efficiency and mitigates objective bias.
Log-density gradient estimation is a fundamental statistical problem and possesses various practical applications such as clustering and measuring non-Gaussianity. A naive two-step approach of first estimating the density and then taking its log-gradient is unreliable because an accurate density estimate does not neces…
Bayesian Optimization improved for high-dimensional outputs.
problem Optimizing many correlated outcomes efficiently.
method Efficient multi-task Gaussian Process sampling.
result Substantial improvements in sample efficiency.
Adversarial online multi-task RL with task separation.
problem Minimize regret in an adversarial online multi-task setting with unknown MDPs.
method Prove minimax and instance-specific lower bounds, develop a clustering algorithm with optimal sample complexity and regret.
result Tight sample complexity and regret bounds for adversarial online multi-task RL.
Meta-Q-Learningcs.LG
Meta-Q-Learning improves meta-Reinforcement Learning using past data.
problem Improving meta-Reinforcement Learning performance.
method Meta-Q-Learning combines Q-learning, multi-task objectives, and off-policy updates to adapt policies from past data.
result Meta-Q-Learning compares favorably with state-of-the-art meta-RL algorithms on benchmarks.
New method robustly discovers causal relationships from imperfect data.
problem Challenges in causal discovery from imperfect structural constraints.
method Prior alignment and conflict resolution through surrogate model and multi-task learning.
result Proposes a robust method for causal discovery under imperfect constraints.
Meta-World benchmark tests meta-reinforcement learning on 50 robotic tasks.
problem Current meta-reinforcement learning benchmarks are too narrow, limiting generalization.
method Proposes a new benchmark with 50 robotic manipulation tasks.
result State-of-the-art algorithms struggle with multiple tasks simultaneously.
DeepDIVE disentangles input into marginal and conditional distributions for multi-task learning.
problem Challenges in multi-task learning due to conflicting objectives.
method Inspired by probability theory, DeepDIVE uses a variational autoencoder with disentangled features and cross-attention mechanism.
result DeepDIVE disentangles input and improves forecast accuracy compared to baseline models.
Higher-Order Generalization Bounds: Learning Deep Probabilistic Programs via PAC-Bayes Objectivescs.LG
Unified framework for learning flexible probabilistic programs using DPP and PAC-Bayes bounds.
problem Learning and generalizing from complex probabilistic models.
method Unified DPP representation and PAC-Bayes bounds for stochastic programs.
result Improved performance and generalization prediction using flexible DPP model representations and learned complexity measures.
New algorithm reduces sample complexity for multi-task bandits.
problem Optimizing representation and predictor pairs for multi-task bandits.
method OSRL-SC algorithm with sample complexity .
result OSRL-SC algorithm approaches sample complexity lower bounds.
Continual Learning in artificial neural networks suffers from interference and forgetting when different tasks are learned sequentially. This paper introduces the Active Long Term Memory Networks (A-LTM), a model of sequential multi-task deep learning that is able to maintain previously learned association between sens…
Multi-task learning uses auxiliary data or knowledge from relevant tasks to facilitate the learning in a new task. Multi-task optimization applies multi-task learning to optimization to study how to effectively and efficiently tackle multiple optimization problems simultaneously. Evolutionary multi-tasking, or multi-fa…
Gradient surgery improves multi-task learning efficiency.
problem Challenges in sharing structure across multiple tasks.
method Gradient projection onto normal plane of conflicting gradients.
result Substantial gains in efficiency and performance.
Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecastingstat.ML
The paper uses random matrix theory for multi-task regression, improving time series forecasting.
problem Improving time series forecasting using multi-task regression.
method Applying random matrix theory to multi-task regression problems, deriving closed-form solutions for optimization.
result Provides a robust foundation for hyperparameter optimization in multi-task regression scenarios.
We study the stability properties of nonlinear multi-task regression in reproducing Hilbert spaces with operator-valued kernels. Such kernels, a.k.a. multi-task kernels, are appropriate for learning prob- lems with nonscalar outputs like multi-task learning and structured out- put prediction. We show that multi-task ke…
Reliable and effective multi-task learning is a prerequisite for the development of robotic agents that can quickly learn to accomplish related, everyday tasks. However, in the reinforcement learning domain, multi-task learning has not exhibited the same level of success as in other domains, such as computer vision. In…
Multi-task learning holds the promise of less data, parameters, and time than training of separate models. We propose a method to automatically search over multi-task architectures while taking resource constraints into consideration. We propose a search space that compactly represents different parameter sharing strat…
Disentangled representations naturally emerge in multi-task learning.
problem Finding adaptable representations for multiple tasks.
method Empirical study of neural networks trained on automatically generated supervised tasks.
result Disentanglement naturally occurs during multi-task learning.
Syndrome differentiation in Traditional Chinese Medicine (TCM) is the process of understanding and reasoning body condition, which is the essential step and premise of effective treatments. However, due to its complexity and lack of standardization, it is challenging to achieve. In this study, we consider each patient'…