Improved patient risk stratification with relaxed parameter sharing in clinical time-series data.
problem Learning time-varying relationships in clinical time-series data with limited training data.
method Proposed a novel RNN formulation based on a mixture model with relaxed parameter sharing over time.
result Relaxed parameter sharing leads to improved patient risk stratification performance in settings with limited data.
This work formalizes and extends parameter sharing in multi-agent reinforcement learning.
problem Parameter sharing limits multi-agent learning to a single policy, preventing different tasks or action spaces.
method Introduces agent indication and extends parameter sharing to heterogeneous observation and action spaces.
result Proves convergence to optimal policies for parameter sharing in heterogeneous environments.
A new method for multi-task learning by allocating parameters.
problem Sharing parameters between unrelated tasks can hurt performance.
method Learned binary variables to allocate components to tasks, encouraging sharing between related tasks.
result Achieves a 17% relative reduction of the error rate on Omniglot benchmark.
Proposes a method to integrate learner models robustly against misspecifications.
problem Misspecifications in learner models and parameter sharing patterns degrade prediction accuracy.
method Sequentially incorporates additional learners based on user-specified parameter sharing patterns.
result Data-adaptively selects the most suitable way of parameter sharing to enhance predictive performance.
NPAS trains neural networks with a fixed parameter budget, improving performance and compactness.
problem Training neural networks requires memory, and existing methods struggle with arbitrary parameter budgets.
method NPAS learns to share parameters automatically, covering low and high budgets.
result NPAS and SSNs improve network performance and compactness across various tasks.
A study on preventing catastrophic forgetting in neural networks using conditional computation.
problem Catastrophic forgetting in connectionist neural networks.
method Conditional computation framework where parameters are conditioned on each input example.
result Proposed conditional rehearsal to prevent forgetting of previously learned examples.
A hybrid CNN with shared parameters learns to incorporate recurrent structures.
problem Designing efficient recurrent neural networks.
method Parameter sharing scheme combining CNN and recurrent layers.
result Substantial parameter savings with competitive accuracy.
This paper proposes a new multitask learning method to better measure task relationships.
problem Learning multiple tasks independently loses information about task relationships.
method Jointly learns shared parameters and shared feature representations through a new strategy.
result The proposed method better measures task relationships and optimizes shared parameters.
New algorithm reduces multi-agent bandit regret by sharing data.
problem Designing efficient collaboration between multi-agent linear bandits.
method Bandit Adaptive Sample Sharing (BASS) algorithm, without assumptions on bandit parameters structure.
result Validated through theoretical analysis and empirical evaluations, BASS outperforms current state-of-the-art.
ABPS improves RL training efficiency by sharing policies and evolving hyper-params.
problem Data inefficiency in training deep RL models for real-world applications.
method ABPS: adaptive behavior policy sharing; ABPS-PBT: hybridizing ABPS with PBT for evolving hyper-params.
result ABPS achieves superior performance and reduced variance compared to conventional hyper-parameter tuning.
Two approaches improve parameter learning in various mixture models.
problem Parameter learning in mixture models.
method Complex-analytic and algebraic-combinatorial methods.
result Improved sample sufficiency for parameter estimation in specific mixture models.
A method for sharing synthetic data without revealing actual data or model parameters.
problem Sharing data privately between enterprises and their clients.
method privGAN architecture with local GANs and a central discriminator.
result Synthetic data leads to better utility than real small datasets.
We build a multiassets heterogeneous agents model with fundamentalists and chartists, who make investment decisions by maximizing the constant relative risk aversion utility function. We verify that the model can reproduce the main stylized facts in real markets, such as fat-tailed return distribution and long-term mem…
Dynamic Transfer Networks improve NER from clinical notes with less data.
problem Improving named entity recognition from clinical notes with limited data.
method Dynamic Transfer Networks (DTN) using parameter sharing and gating architecture.
result DTN achieves better NER performance with fewer training settings.
New method tackles catastrophic forgetting and order-sensitivity in continual learning.
problem Catastrophic forgetting and order-sensitivity in continual learning.
method Additive Parameter Decomposition (APD) to represent task parameters as a sum of shared and adaptive parts.
result Significantly outperforms state-of-the-art methods in accuracy, scalability, and order-robustness.
Method learns equivariances from data without custom architecture design.
problem Learning equivariances for tasks without manually designed architectures.
method Reparameterization to learn equivariant parameter sharing.
result Can learn equivariances for any finite group of transformations.
We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group G that acts discretely on the input and output of a standard neural network layer φW:ℜM→ℜN, we show that φW is equivariant with respect to G-action iff $\m…
Multi-task learning (MTL) allows deep neural networks to learn from related tasks by sharing parameters with other networks. In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amoun…
Approach for recovering shared structure from multiple networks with unknown noise.
problem Recovering shared structure from multiple networks with unknown edge distributions.
method Exploits shared mean structure to denoise edge-level measurements and estimate population-level parameters.
result Established a finite-sample concentration inequality for low-rank eigenvalue truncation of a random weighted adjacency matrix.
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
FedLog reduces communication in federated learning by sharing data summaries.
problem Significant communication overhead in federated learning with large model parameters.
method Shares minimal sufficient statistics via Bayesian inference and differential privacy.
result High learning accuracy with low communication overhead.
Multi-model forgetting occurs when training multiple deep networks sequentially, leading to performance degradation of previously trained models.
problem Performance degradation of previously trained models when sequentially training multiple deep networks with shared parameters.
method Introduce a weight plasticity loss that regularizes the learning of shared parameters based on their importance for previous models.
result Weight plasticity loss effectively preserves the performance of previously trained models during sequential training and neural architecture search.
Study shows automorphisms of Markov surfaces share periodic points if they share a common iterate.
problem Study of unlikely intersections for automorphisms of Markov surfaces with positive entropy.
method Arithmetic equidistribution for adelic line bundles, theory of laminar currents, quasi-Fuchsian representation theory.
result Two automorphisms with positive entropy share a Zariski dense set of periodic points if and only if they share a common iterate.
CAVIA improves meta-learning by adapting context parameters at test time.
problem Meta-learning overfits and requires complex parallelisation.
method CAVIA splits model into context and shared parameters, updating only the former.
result CAVIA outperforms MAML across regression, classification, and reinforcement learning.
Developed a new algorithm to improve dynamic treatment regimens.
problem Non-convergence of Q-learning-based Q-shared algorithm in dynamic treatment regimens.
method Penalized Q-shared algorithm to address convergence issues.
result The penalized Q-shared algorithm converges and outperforms the original in various settings.
Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tas…
New PG samplers improve inference in coupled state-space models.
problem Bayesian inference from multiple time series with shared parameters.
method Marginalized Particle Gibbs samplers for coupled state-space models.
result Improved parameter inference through shared information.
This paper analyzes the quantitative relations between stock prices and quantities of tradable stock shares in Chinese stock markets at six time points by means of Exploratory Data Analysis (EDA) method. It is found the resulting formulae have the same structure but different parameters. This paper also uses these rela…
The group membership prediction (GMP) problem involves predicting whether or not a collection of instances share a certain semantic property. For instance, in kinship verification given a collection of images, the goal is to predict whether or not they share a {\it familial} relationship. In this context we propose a n…
Meta-learning improves CATE estimation from limited data.
problem Estimating heterogeneous treatment effects from scarce observational data.
method Meta-learning framework decomposes CATE estimation into sub-problems, using neural networks with shared and specific parameters, and optimizing task-specific parameters in closed form.
result Meta-learning method outperforms existing approaches in few-shot CATE estimation.
In many distributed learning problems, the heterogeneous loading of computing machines may harm the overall performance of synchronous strategies. In this paper, we propose an effective asynchronous distributed framework for the minimization of a sum of smooth functions, where each machine performs iterations in parall…
HCL learns shared and modality-specific latent representations for multimodal data.
problem Binary shared-private decomposition inadequately represents shared information across subsets of modalities.
method Hierarchical Contrastive Learning framework combining latent-variable formulation, structural sparsity, and contrastive objective.
result HCL accurately recovers hierarchical structure and improves predictive performance on multimodal data.
PAVI speeds up VI for large-scale studies by sharing parameterization across i.i.d. variables.
problem Challenges in Bayesian inference for large population studies with many latent parameters.
method Designing plate-amortized variational inference (PAVI) to share parameterization across i.i.d. variables.
result Significant speedup in training large-scale hierarchical variational distributions.
A new method for efficient neural network fine-tuning using queryable low-rank update atoms.
problem Rigidity of static low-rank adaptation methods when input and depth-wise computation vary.
method A shared queryable memory of low-rank update atoms, allowing dynamic and context-sensitive adaptation.
result Improves final test performance and training stability compared to standard low-rank adaptation.
Boosts share routing for multi-task learning with flexible sparse connections.
problem Designing suitable sharing mechanisms among multiple tasks in multi-task learning.
method Proposes MTNAS framework to modularize sharing into sub-networks with sparse connections and gating.
result Demonstrates consistent improvement over single-task and typical multi-task methods while maintaining efficiency.
Cronus securely transfers model parameters to protect federated learning from poisoning attacks.
problem Federated learning's privacy and security issues, especially with large models.
method Cronus uses black-box knowledge transfer to reduce parameter dimensions and control information leakage.
result Cronus is the only secure federated learning method against poisoning attacks.
Neural models can realize decision trees with parameter sharing and improved performance.
problem Training and optimizing oblique decision trees.
method Locally constant networks based on ReLU gradients, parameter sharing, and neural tools.
result Locally constant networks can implicitly model oblique decision trees with fewer neurons.
New method uses fewer parameters to match state-of-the-art performance on multiple natural language tasks.
problem Efficiently adapting BERT for multiple tasks with fewer parameters.
method PALs (projected attention layers) for shared BERT model with task-specific parameters.
result Matches state-of-the-art performance on GLUE benchmark with 7 times fewer parameters.
Novel framework for data sharing and coordinated exploration in concurrent RL with non-identical environments.
problem Learning more data-efficient and better policies in concurrent RL with non-identical environments.
method Proposes a novel algorithmic framework that leverages causal inference via ANM-MM to extract model parameters and a new data sharing scheme based on similarity measures.
result Demonstrates superior learning speeds on various tasks and effectiveness of diverse action selection.
AGNN automates GNN architecture search, achieving best performance.
problem Finding optimal GNN architectures is laborious and requires human expertise.
method AGNN uses reinforcement learning to search for optimal GNN architectures within a predefined space, with a novel parameter sharing strategy.
result AGNN identifies optimal GNN architectures achieving best performance.
Transfer learning assumes classifiers of similar tasks share certain parameter structures. Unfortunately, modern classifiers uses sophisticated feature representations with huge parameter spaces which lead to costly transfer. Under the impression that changes from one classifier to another should be ``simple'', an effi…
Parameters in deep neural networks which are trained on large-scale databases can generalize across multiple domains, which is referred as "transferability". Unfortunately, the transferability is usually defined as discrete states and it differs with domains and network architectures. Existing works usually heuristical…
Mirror descent with an entropic regularizer is known to achieve shifting regret bounds that are logarithmic in the dimension. This is done using either a carefully designed projection or by a weight sharing technique. Via a novel unified analysis, we show that these two approaches deliver essentially equivalent bounds …
We consider the problem of multi-task learning in the high dimensional setting. In particular, we introduce an estimator and investigate its statistical and computational properties for the problem of multiple connected linear regressions known as Data Enrichment/Sharing. The between-tasks connections are captured by a…
Improved AutoDML estimator for causal inference using outcome-adapted shared covariate representation.
problem Efficiency in estimating treatment or policy effects in causal inference.
method Outcome-adapted AutoDML estimator that uses a shared covariate representation that is predictive of the outcome but not the Riesz representer.
result Outcome-adapted AutoDML estimator is asymptotically more efficient than baseline AutoDML.
We consider high-dimensional distribution estimation through autoregressive networks. By combining the concepts of sparsity, mixtures and parameter sharing we obtain a simple model which is fast to train and which achieves state-of-the-art or better results on several standard benchmark datasets. Specifically, we use a…
Model analyzes cooccurrence data for recommender systems and item relevance.
problem High-dimensional cooccurrence data from online platforms.
method Shared parameter Alternating Tweedie (SA-Tweedie) model with Fisher scoring and learning rate adjustment.
result SA-Tweedie model outperforms other methods in optimizing parameters.
FSNet compresses CNNs by sharing weights in 1D vectors.
problem Compressing deep CNNs while maintaining performance.
method Learning a 1D vector (Filter Summary) to share weights in convolutional filters.
result FSNet reduces parameter space and accelerates computation.