Establishes connection between MTDNN and multitask GP, revealing weight correlation as key to task sharing.
problem Limited theoretical understanding of information sharing in MTDNN.
method Derives multitask GP kernels for MTDNN and MTBNN, showing shared hyper-parameters and last layer weights.
result Information sharing in MTDNN is due to weight correlation, not intermediate layer weights.
Framework identifies brain connectivity alterations for MDD patients using limited rs-fMRI data.
problem Difficult to analyze brain connectivity alterations from limited rs-fMRI data.
method Proposed a multitask Gaussian Bayesian network (MTGBN) framework to learn individual disease-induced alterations.
result Framework efficiently learns Bayesian network structures from limited data, showing improved performance.
We study learning problems in which the conditional distribution of the output given the input varies as a function of additional task variables. In varying-coefficient models with Gaussian process priors, a Gaussian process generates the functional relationship between the task variables and the parameters of this con…
New Bayesian method improves Pareto front estimation in multitask finetuning.
problem Efficiently estimating Pareto fronts for multitask finetuning.
method Variational Model Merging using non-Gaussian posteriors.
result More flexible posteriors lead to better Pareto front estimates.
Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.
problem Understanding the generalization error in concept bottleneck models.
method Mathematical analysis of Bayesian generalization error and free energy in CBM for 3-layered linear neural networks.
result CBM significantly alters the parameter region and Bayesian generalization error compared to standard models.
Study uses SABR model to create implied volatilities from sparse quotes.
problem Creating accurate implied volatility surfaces from limited market data.
method Multitask Gaussian process with SABR model embeddings and hierarchical regularization.
result Model produces more accurate volatilities than single-task methods.
A new LMC model reduces complexity from cubic to linear, making multitask Gaussian processes more practical.
problem High computational complexity in multitask Gaussian processes.
method Decoupling latent processes under mild noise model assumptions, leading to linear complexity.
result Efficient exact computation of the LMC model is possible under mild noise model assumptions.
The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.
problem Understanding the risk in semi-supervised multitask learning on Gaussian mixtures.
method Statistical physics methods applied to Gaussian mixture models.
result The study evaluates the performance gain of learning tasks together versus separately.
New framework identifies and reduces errors in machine learning under distribution shift.
problem Errors in machine learning models when distributions change.
method Developed a principled framework to characterize and eliminate epistemic errors in imperfect multitask learning.
result Provided a decompositional epistemic error bound for general settings of distribution shift.
Multitask Gaussian process regression reduces data generation costs for molecular property prediction.
problem Data bottleneck in training surrogate models for molecular properties.
method Multitask Gaussian process regression over heterogeneous data sources (CC and DFT).
result Predicts at CC-level accuracy with over an order of magnitude reduction in data generation cost.
The paper improves Gaussian processes by adding sum constraints, enhancing prediction accuracy.
problem Improving Gaussian process predictions with background knowledge constraints.
method Conditioning the prior distribution on sum constraints to ensure fulfillment of linear and nonlinear constraints.
result The approach fulfills constraints with high precision and improves prediction accuracy.
Bayesian model connects KMs and ELMs for multitask regression.
problem Multitask regression with shared features and sparsity.
method Bayesian framework using RFFs and RBF kernels.
result Significant performance improvements over state-of-the-art methods.
A new approach simplifies multitask Gaussian processes without rank approximations.
problem Handling multioutput regression problems with conditionally dependent tasks.
method Introduces a novel approach to reduce multitask learning to univariate GPs, eliminating the need for rank approximations.
result Accurately recovers multitask covariance and noise matrices with fewer parameters, improving performance and reducing overfitting risk.
We learn multiple hypotheses for related tasks under a latent hierarchical relationship between tasks. We exploit the intuition that for domain adaptation, we wish to share classifier structure, but for multitask learning, we wish to share covariance structure. Our hierarchical model is seen to subsume several previous…
Scalable GP model handles functional covariates and multitasks.
problem Uncertainty quantification in complex mechanical systems with time-dependent inputs.
method Introduced a fully separable kernel structure for functional covariates and multitask problems, leveraging Kronecker structure for scalability.
result The model significantly improves over single task GPs, requiring fewer samples for accurate predictions.
Deep learning methods such as multitask neural networks have recently been applied to ligand-based virtual screening and other drug discovery applications. Using a set of industrial ADMET datasets, we compare neural networks to standard baseline models and analyze multitask learning effects with both random cross-valid…
Massively multitask neural architectures provide a learning framework for drug discovery that synthesizes information from many distinct biological sources. To train these architectures at scale, we gather large amounts of data from public sources to create a dataset of nearly 40 million measurements across more than 2…
Improved molecular property prediction using multitask learning.
problem Predicting molecular properties from chemical data is challenging.
method Multitask learning applied to graph neural networks.
result Multitask learning significantly improves model performance and reduces variance.
Multitask learning has shown promising performance in many applications and many multitask models have been proposed. In order to identify an effective multitask model for a given multitask problem, we propose a learning framework called learning to multitask (L2MT). To achieve the goal, L2MT exploits historical multit…
Research explores the trade-off between multi-task learning and multitasking in deep neural networks.
problem Trade-off between multi-task learning and multitasking in deep neural networks.
method Meta-learning algorithm to manage the trade-off between shared and separated representations.
result Agent successfully optimizes training strategy based on environment.
Paper introduces multitask neural networks for efficient stochastic control problems.
problem Infeasibility of simulating state variables in some stochastic control problems.
method Multitask neural networks with dynamic task balancing.
result Multitask neural networks outperform state-of-the-art approaches in derivatives pricing problems.
Paper discovers differential equations from data using neural networks and Bayesian methods.
problem Discovering differential equations from datasets using machine learning.
method Integrates neural network-based surrogates with Sparse Bayesian Learning (SBL).
result Proposes a robust model discovery algorithm and a Physics Informed Normalizing Flow (PINF).
We present a scalable end-to-end classifier that uses streaming physiological and medication data to accurately predict the onset of sepsis, a life-threatening complication from infections that has high mortality and morbidity. Our proposed framework models the multivariate trajectories of continuous-valued physiologic…
The fundamental aim of clustering algorithms is to partition data points. We consider tasks where the discovered partition is allowed to vary with some covariate such as space or time. One approach would be to use fragmentation-coagulation processes, but these, being Markov processes, are restricted to linear or tree s…
We propose a novel hierarchical model for multitask bipartite ranking. The proposed approach combines a matrix-variate Gaussian process with a generative model for task-wise bipartite ranking. In addition, we employ a novel trace constrained variational inference approach to impose low rank structure on the posterior m…
From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived. It is applied to sharpen existing and to derive novel bounds on the empirical Rademacher complexities of unit balls in various norms appearing in the context of structured sparsity and multitask dictionary lear…
We discuss a general method to learn data representations from multiple tasks. We provide a justification for this method in both settings of multitask learning and learning-to-learn. The method is illustrated in detail in the special case of linear feature learning. Conditions on the theoretical advantage offered by m…
Proposes a method to balance tasks in multitask learning with a single gradient step update.
problem Balancing tasks in multitask learning to avoid imbalance.
method Gradient-based meta-learning to balance tasks at the gradient level, training shared and task-specific layers separately.
result Achieves state-of-the-art performance on various multitask computer vision problems.
The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
Analyzes how multitask learning improves deep neural networks' generalization.
problem Understanding how multitask learning enhances deep neural networks' generalization.
method Developed an analytic theory using statistical physics techniques for classification tasks.
result Multitask learning benefits from task alignment and noise characteristics.
We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply …
New algorithm improves multitask learning across diverse agents.
problem Performance degradation in decentralized learning with heterogeneous objectives.
method Developed an exact subspace diffusion algorithm for multitask learning over networks.
result The algorithm outperforms alternatives in noisy gradient approximations.
Bayesian inference for wide neural networks using Edgeworth expansion.
problem Analyzing the non-Gaussian behavior of wide neural networks in Bayesian inference.
method Proposed a non-Gaussian distribution using multivariate Edgeworth expansion for finite-width neural networks.
result Derived non-Gaussian posterior distribution in Bayesian regression tasks.
Unified multitask learning framework for mixed-type outcomes.
problem Difficulty in formulating a unified objective for tasks with different outcomes.
method Multitask transformation framework with shared sparsity, using deep neural networks and rank-based optimization.
result Improved prediction and variable selection across continuous, binary, and mixed outcomes.
New models improve stock and wind speed forecasting.
problem Lack of posterior distribution in stochastic volatility models.
method Re-cast stochastic volatility models as hierarchical Gaussian processes with specialized covariance functions.
result Volt and Magpie models significantly outperform baselines in forecasting.
The study analyzes implicit biases in neural networks using backward error analysis.
problem Analyzing implicit biases in multitask and continual learning settings.
method Backward error analysis to compute implicit training biases, deriving modified losses with three terms.
result The conflict term, measuring gradient alignment, is a new quantity in continual learning.
Proposes TAGI for efficient Gaussian inference in Bayesian neural networks.
problem Efficient inference in Bayesian neural networks with complex architectures.
method Analytical method for tractable approximate Gaussian inference (TAGI).
result Matches performance of gradient-based methods with O(n) computational complexity. Multitask learning algorithms are typically designed assuming some fixed, a priori known latent structure shared by all the tasks. However, it is usually unclear what type of latent task structure is the most appropriate for a given multitask learning problem. Ideally, the "right" latent task structure should be learne…
Wide stochastic networks show Gaussian behavior and improve training with PAC-Bayesian methods.
problem Analyzing and training over-parameterised neural networks with large width.
method Establishing Gaussian behavior for a stochastic architecture, applying PAC-Bayesian training.
result PAC-Bayesian training on large but finite-width networks outperforms standard methods.
Researchers derive exact priors for finite Bayesian neural networks.
problem Understanding non-Gaussian priors in finite Bayesian neural networks.
method Analytical derivation of function space priors for finite fully-connected feedforward networks.
result Exact solutions for priors of finite networks, including Meijer G-function for linear networks and mixtures for ReLU networks.
The paper proposes a semi-parametric Bayesian network model using Gaussian Processes and Horseshoe priors.
problem Learning semi-parametric relationships in Expert Bayesian Networks with minimal nonlinear components.
method Uses Gaussian Processes and Horseshoe priors to model relationships, prioritizes modifying expert graphs, and generates diverse graphs.
result Models outperform state-of-the-art semi-parametric Bayesian Network models in synthetic and real-world datasets.
A new method prunes deep networks in one go without specifying pruning levels.
problem Deep model compression to reduce model size and inference time.
method Learning a pruner network to identify and prune unnecessary filters from a pre-trained network.
result Pruned networks achieve comparable performance to unpruned ones, with significant reduction in model size.
Study how depth affects inference in deep Bayesian neural networks.
problem Understanding how depth impacts inference in overparameterized linear Bayesian neural networks.
method Interpreting finite deep linear Bayesian neural networks as scale mixtures of Gaussian process predictors.
result Advances analytical understanding of how depth affects inference in a simple class of Bayesian neural networks.
Background: Pharmacokinetic evaluation is one of the key processes in drug discovery and development. However, current absorption, distribution, metabolism, excretion prediction models still have limited accuracy. Aim: This study aims to construct an integrated transfer learning and multitask learning approach for deve…
Bayesian model transfers knowledge across different engineering fleets.
problem Data sparsity in predictive models for engineering infrastructure.
method Hierarchical Bayesian approach with multitask learning.
result Improves survival analysis and power prediction in truck fleets and wind farms.
A novel Laplace-approximated Bayesian Tensor Network Kernel Machine (LA-TNKM) provides principled uncertainty estimates.
problem How to provide principled uncertainty estimates for tensor network kernel machines.
method Employing a linearized Laplace approximation for Bayesian inference.
result Consistently matches or surpasses Gaussian Processes and BNNs across diverse UCI regression benchmarks.
Bayesian neural networks use ridgelet prior for uncertainty quantification.
problem Combining strong predictive performance with uncertainty quantification in Bayesian neural networks.
method Proposes a ridgelet prior that approximates a Gaussian process covariance function in the output space of the network.
result Establishes universality property allowing Bayesian neural networks to approximate any Gaussian process.
Bayesian kernel regression improves functional output prediction.
problem Functional output regression in supervised learning.
method Kernel methods, leveraging covariance structure within function values.
result Enhanced prediction accuracy and handling of high-dimensional nonlinearity.