arXiv research

nGPT learns to transfer learning rates across model dimensions and token horizons.

Proof of learning rate transfer in MLPs with $μ$P parameterization.

Flexible framework for transfer learning with optimal rates.

This paper quantifies hyperparameter transfer and finds embedding layer learning rate is key.

Transfer learning improves chaotic dynamics predictions with less data.

We present a new approach for transferring knowledge from groups to individuals that comprise them. We evaluate our method in text, by inferring the ratings of individual sentences using full-review ratings. This approach, which combines ideas from transfer learning, deep learning and multi-instance learning, reduces t…

Study examines the impact of target data in transfer learning.

Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve the sample complexity of the new tasks. Considering that data collection in RL is both more time and cost consuming and Q-learning converge…

Learning from small amounts of labeled data is a challenge in the area of deep learning. This is currently addressed by Transfer Learning where one learns the small data set as a transfer task from a larger source dataset. Transfer Learning can deliver higher accuracy if the hyperparameters and source dataset are chose…

New study reveals surprising adaptive rates in model selection for transfer learning.

AdaTrans adapts to feature and sample transfer in high-dimensional regression.

Develops methods to measure and set function-space learning rates in neural networks.

New method tackles concept shifts in nonparametric regression using robust and adaptive transfer learning.

This paper improves kernel-based regression using transfer learning.

Cross-domain recommendation has long been one of the major topics in recommender systems. Recently, various deep models have been proposed to transfer the learned knowledge across domains, but most of them focus on extracting abstract transferable features from auxilliary contents, e.g., images and review texts, and th…

We propose to apply deep transfer learning from computer vision to static malware classification. In the transfer learning scheme, we borrow knowledge from natural images or objects and apply to the target domain of static malware detection. As a result, training time of deep neural networks is accelerated while high c…

Paper proposes a transfer learning method for improving matrix completion.

Study shows benefits of transfer learning with neural networks.

Estimates target GGM using auxiliary studies with false discovery rate control.

The paper uses thermodynamics to improve machine learning representation quality.

Human learners have the natural ability to use knowledge gained in one setting for learning in a different but related setting. This ability to transfer knowledge from one task to another is essential for effective learning. In this paper, we study transfer learning in the context of nonparametric classification based …

Paper tackles transfer learning for contextual multi-armed bandits under covariate shift.

Localized transfer learning improves nonparametric regression performance.

A simple strategy prevents negative transfer in transfer learning.

Transfer learning for bandits with latent Lipschitz continuity.

The paper improves high-dimensional linear regression prediction and estimation using auxiliary samples.

Paper tackles federated learning with privacy, enhancing target data analysis.

Improves functional linear regression with shape transfer learning.

Collaborative filtering (CF) aims to predict users' ratings on items according to historical user-item preference data. In many real-world applications, preference data are usually sparse, which would make models overfit and fail to give accurate predictions. Recently, several research works show that by transferring k…

We introduce a new weight-decay scaling rule to maintain sublayer gains across different widths in modern scale-invariant architectures.

New optimizers control network width scaling, improving stability and transfer across different model sizes.

TLRF improves timely COVID-19 outbreak detection with small sample size counties.

Study shows cliff-learning in transfer learning from foundation models.

New method improves adversarial transferability from Bayesian neural networks.

Adaptive transfer learning model for varying mechanisms across domains.

MLR-SNet learns flexible LR schedules for diverse tasks.

Paper tackles nonparametric classification with privacy constraints, achieving optimal accuracy.

Bullseye Polytope improves clean-label poisoning attacks in transfer learning.

We consider the problem of knowledge transfer when an agent is facing a series of Reinforcement Learning (RL) tasks. We introduce a novel metric between Markov Decision Processes (MDPs) and establish that close MDPs have close optimal value functions. Formally, the optimal value functions are Lipschitz continuous with …

SATL adapts to varying smoothness in hypothesis transfer learning.

RIFLE improves deep transfer learning by reinitializing fully-connected layers.

New algorithm improves knowledge transfer in dynamic decision-making.

The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.

Framework improves target domain prediction using quantile matching.

LGV boosts adversarial attacks by improving surrogate models.

New method uses correlation-ratio for transfer learning, improving target model inference.

Background: Pharmacokinetic evaluation is one of the key processes in drug discovery and development. However, current absorption, distribution, metabolism, excretion prediction models still have limited accuracy. Aim: This study aims to construct an integrated transfer learning and multitask learning approach for deve…

Unified approach for sample aggregation in transfer learning across various divergence measures.