arXiv research

Survey on self-supervised pre-training for neural networks using unlabeled data.

DM improves self-supervised transfer learning by matching target distributions.

SASSL improves self-supervised learning by preserving image structure.

Simplifies transfer learning with deep neural networks using ridge regression.

ProtoTransfer learns from unlabeled data to classify unseen tasks.

TX-Ray analyzes and quantifies model knowledge transfer in NLP.

Framework for lifelong learning using eigentasks to avoid forgetting and transfer knowledge.

In real-world applications, it is often expensive and time-consuming to obtain labeled examples. In such cases, knowledge transfer from related domains, where labels are abundant, could greatly reduce the need for extensive labeling efforts. In this scenario, transfer learning comes in hand. In this paper, we propose D…

Framework reuses pre-trained models for data-free transfer learning.

New method for one-shot timbre transfer in music.

Transfer learning improves sparse, interpretable probabilistic classification.

We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervised learning have shown the benefits of training on large unlabeled datasets, we find improvements in …

Single-trial classification of event-related potentials in electroencephalogram (EEG) signals is a very important paradigm of brain-computer interface (BCI). Because of individual differences, usually some subject-specific calibration data are required to tailor the classifier for each subject. Transfer learning has be…

Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet, discrepancies between the underlying distributions of the source and target data are c…

TransMatch uses transfer learning to improve few-shot learning accuracy.

PCA-based MTL improves performance with reduced computation.

New method improves transfer and robustness of supervised contrastive learning.

Sense and Learn uses self-supervised learning for IoT data.

Method transfers knowledge without label overlap, source data, or target architecture consistency.

Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may significantly improve the learning performance, most of the literature on transfer is focused on ba…

This paper evaluates a method to improve representations using incomplete external evidence across tasks.

User interaction with voice-powered agents generates large amounts of unlabeled utterances. In this paper, we explore techniques to efficiently transfer the knowledge from these unlabeled utterances to improve model performance on Spoken Language Understanding (SLU) tasks. We use Embeddings from Language Model (ELMo) t…

End-to-end analysis of SGD for STL with adaptive sub-sampling.

Unified meta-learning framework from supervised learning.

This study compares transfer learning and self-supervised learning for better model performance.

Proposes a transfer learning method for PCA studies.

Unified framework for transfer learning regression without extra cost.

With access to large datasets, deep neural networks (DNN) have achieved human-level accuracy in image and speech recognition tasks. However, in chemistry, data is inherently small and fragmented. In this work, we develop an approach of using rule-based knowledge for training ChemNet, a transferable and generalizable de…

Modular deep learning framework using pairwise labels without backpropagation.

TAP transfers knowledge from unlabeled data to improve cross-modal learning.

We propose a novel approach for estimating the difficulty and transferability of supervised classification tasks. Unlike previous work, our approach is solution agnostic and does not require or assume trained models. Instead, we estimate these values using an information theoretic approach: treating training labels as …

New model improves transfer learning and semi-supervised learning.

A new model improves semi-supervised learning for unstructured data.

Study finds CLIP's caption-based learning outperforms image-only methods under certain conditions.

Paper proposes a method to train robust neural networks without labeled data.

SupSiam and SupBYOL improve supervised representation learning with ANCL.

Advances in neural network based classifiers have transformed automatic feature learning from a pipe dream of stronger AI to a routine and expected property of practical systems. Since the emergence of AlexNet every winning submission of the ImageNet challenge has employed end-to-end representation learning, and due to…

Paper shows pre-training and transfer learning reduce sample complexity for neural networks.

New unsupervised transfer learning method for spatiotemporal tasks.

DDSTN improves breast cancer diagnosis by leveraging imbalanced ultrasound modalities.

In recent years, supervised machine learning models have demonstrated tremendous success in a variety of application domains. Despite the promising results, these successful models are data hungry and their performance relies heavily on the size of training data. However, in many healthcare applications it is difficult…

Adaptive source selection for positive transfer in linear models improves target dataset performance.

Protein modeling is an increasingly popular area of machine learning research. Semi-supervised learning has emerged as an important paradigm in protein modeling due to the high cost of acquiring supervised protein labels, but the current literature is fragmented when it comes to datasets and standardized evaluation tec…

Paper introduces a new method for improving reinforcement learning performance using transfer learning.

We consider the problem of function estimation in the case where an underlying causal model can be inferred. This has implications for popular scenarios such as covariate shift, concept drift, transfer learning and semi-supervised learning. We argue that causal knowledge may facilitate some approaches for a given probl…

This paper introduces a new transfer learning method for regression.

CLS measures dataset similarity through decision rule performance.

The task of learning a sentiment classification model that adapts well to any target domain, different from the source domain, is a challenging problem. Majority of the existing approaches focus on learning a common representation by leveraging both source and target data during training. In this paper, we introduce a …