Study measures impact of data and neural net similarity on transferability in restaurant sales data.
problem Identify indicators for successful transferability of neural nets across different data sets.
method Empirical study on sales data from six restaurants, calculating indicators based on data and neural net similarities.
result Negative correlations between transferability and indicators, allowing better model performance and fewer transfers.
Paper introduces a new method for improving reinforcement learning performance using transfer learning.
problem Improving reinforcement learning performance with limited sample sizes in dynamic decision-making scenarios.
method Developed a novel ``re-weighted targeting procedure'' and ``transfer deep Q∗-learning'' approach. result Demonstrated improved reinforcement learning performance through strategic sample construction.
Physics-informed neural networks simulate radiative transfer efficiently.
problem Simulating radiative transfer accurately and efficiently.
method Physics-informed neural networks trained to minimize radiative transfer equations.
result PINNs provide an easy-to-implement, robust, and accurate method for radiative transfer simulation.
We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural A…
New attention mechanism improves meta-transfer learning in dynamic tasks.
problem Underfitting in meta-transfer learning with dynamic tasks.
method Proposed Recurrent Memory Reconstruction (RMR) attention mechanism.
result ASNP-RMR significantly outperforms baselines in various tasks.
Paper shows pre-training and transfer learning reduce sample complexity for neural networks.
problem Training high-dimensional supervised learning with limited labeled data.
method Study of single-layer neural networks via online stochastic gradient descent, considering concept shift.
result Pre-training and transfer learning reduce sample complexity by polynomial factors under general assumptions.
Simplifies transfer learning with deep neural networks using ridge regression.
problem High computational cost of finetuning deep models for transfer learning.
method Leverage the low-rank property of deep neural networks' feature vectors in kernel ridge regression.
result Successful on supervised and semi-supervised transfer learning tasks.
Study shows benefits of transfer learning with neural networks.
problem Understanding generalization errors in transfer learning.
method Mean-field analysis applied to α-ERM and fine-tuning. result Established conditions for generalization error and convergence rates.
Measures neural network information transfer for generalization.
problem Estimating the generalizable information in neural networks.
method Proposes Information Transfer (LIT) based on prequential coding. result Consistently correlates with generalizable information in neural networks.
The paper establishes limits of transfer learning with neural networks.
problem Understanding the fundamental limits of transfer learning.
method Statistical minimax framework for regression with linear and neural network models.
result Lower bounds for target generalization error.
Paper proposes transferring neural network learning trajectories.
problem Training deep neural networks is computationally expensive.
method Formulated and solved the learning transfer problem by matching gradients.
result Transferred parameters achieve non-trivial accuracy before training.
Study on neural networks' performance in sequential task learning.
problem Understanding the performance of neural networks in sequential task learning.
method Theoretical analysis of generalization performance in continual learning using statistical mechanical analysis of kernel ridge-less regression.
result Characteristic transitions from positive to negative transfer observed in neural networks.
Proof of learning rate transfer in MLPs with μP parameterization.
problem Understanding and optimizing learning rates in neural networks with different parameterizations.
method Theoretical analysis and empirical validation of learning rate transfer in MLPs with μP, SP, and NTP parameterizations. result The optimal learning rate converges to a non-zero constant as width goes to infinity under μP, explaining learning rate transfer. Improved drug-protein interaction prediction using FTL method.
problem Predicting drug-protein interactions from noisy data with uncertain labels.
method Filtered Transfer Learning (FTL) method that fine-tunes a deep neural network across multiple tiers of data confidence.
result FTL method outperforms deep neural networks trained on single confidence ranges.
Anti-transfer learning prevents misleading representations for speech tasks.
problem Misleading representations learned from orthogonal tasks in speech processing.
method Penalizes similarity between activations of a network and another trained on an orthogonal task.
result Improves classification accuracy and invariance to the orthogonal task.
Transfer learning improves algorithm recommendation performance.
problem Improving algorithm recommendation with limited data.
method Train a neural network on meta-datasets, then transfer knowledge to similar datasets.
result Transfer learning enhances meta-learning for algorithm recommendation.
A powerful network teaches a weak one, improving its performance.
problem Improving the performance of a weak neural network using a more powerful one.
method During training, a weak network learns features from a strong network to minimize feature distance.
result A weak neural network can increase its performance by learning from a more powerful network.
Survey explores how transfer learning improves deep reinforcement learning.
problem Challenges in reinforcement learning efficiency and effectiveness.
method Categorizes and analyzes transfer learning approaches.
result Transfer learning enhances reinforcement learning performance.
We extend graph neural networks to transfer performance across different input sizes.
problem Transferability of graph neural networks across varying input dimensions.
method Introduce a general framework for transferability across dimensions, showing it corresponds to continuity in a limit space.
result Transferability of graph neural networks is driven by data and learning task, and can be ensured with design principles.
This paper reviews transfer learning for financial data predictions, highlighting its potential.
problem Accurate stock price prediction in financial time series is challenging due to noise and non-linear relationships.
method Transfer Learning applied to financial market predictions.
result Transfer Learning can improve financial prediction capability.
Paper classifies Parkinson's disease from speech in three languages using CNNs and transfer learning.
problem Classifying Parkinson's disease from speech in multiple languages.
method Convolutional Neural Networks (CNNs) with transfer learning among Spanish, German, and Czech.
result Transfer learning improves model accuracy by up to 8% and balances specificity-sensitivity.
Improved sales forecasting for new products using transfer learning.
problem Insufficient training data for new products leads to inaccurate sales forecasts.
method Network-based Transfer Learning approach for deep neural networks.
result Deep neural networks' prediction accuracy for food sales forecasting can be effectively increased.
Deep tabular models outperform GBDT in medical diagnosis tasks.
problem Transfer learning for tabular data in medical diagnosis.
method Proposes a pseudo-feature method for transfer learning between different feature sets.
result Tabular neural networks outperform GBDT in medical diagnosis tasks.
XMixup improves transfer learning accuracy by 1.9% with less training time.
problem Efficiently transfer knowledge from large source datasets to target tasks with small samples.
method Cross-domain Mixup technique that selects auxiliary samples from source datasets and augments training samples via mixup strategy.
result Improves accuracy by 1.9% on average over six real-world transfer learning datasets.
Survey on self-supervised pre-training for neural networks using unlabeled data.
problem Improving model performance using unlabeled data.
method Pre-training on unlabeled data followed by task-specific adaptation.
result Enhanced model performance through self-supervised pre-training.
Neural planners for RDDL MDPs produce deep reactive policies in an offline fashion. These scale well with large domains, but are sample inefficient and time-consuming to train from scratch for each new problem. To mitigate this, recent work has studied neural transfer learning, so that a generic planner trained on othe…
Transfer learning for deep neural networks is the process of first training a base network on a source dataset, and then transferring the learned features (the network's weights) to a second network to be trained on a target dataset. This idea has been shown to improve deep neural network's generalization capabilities …
Transfer learning adapted for hybrid classical-quantum neural networks.
problem Optimizing data preprocessing and feature embedding for quantum processors.
method Adapting transfer learning to hybrid networks, using a pre-trained classical network augmented by a quantum circuit.
result Demonstrated the effectiveness of quantum transfer learning for image recognition and quantum state classification.
Transfer learning improves image classifier performance in data-starved regimes.
problem Deploying image classifiers in domains with limited labeled data.
method Transfer learning with deep neural networks, focusing on feature reuse and overparameterization.
result Transfer learning enhances CNN performance in data-starved regimes.
SASSL improves self-supervised learning by preserving image structure.
problem Distorted augmented samples in self-supervised learning.
method Neural Style Transfer to decouple semantic and stylistic attributes.
result Boosts ImageNet top-1 accuracy by up to 2 percentage points.
The study explores how brain development can inspire efficient deep learning models.
problem Efficient and robust optimization procedures for deep learning.
method Inspiration from biological neural development to improve deep learning models.
result Biological neural development can inspire efficient and robust optimization procedures.
Randomized Geometric Algebra for Convex Neural Networks Optimizes Transfer Learning.
problem Training neural networks to global optimality via convex optimization.
method Randomized algorithms in Clifford's Geometric Algebra for hypercomplex vector spaces.
result Convex optimization and geometric algebra improve LLMs' robustness and reliability in transfer learning.
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
problem Efficient transfer of knowledge in reinforcement learning.
method Behavior Transfer (BT) that uses pre-trained policies for exploration.
result BT combined with pre-training leads to better solutions than without pre-training.
Paper explores using bi-fidelity data to train neural networks for uncertainty quantification.
problem Training neural networks requires large amounts of data, which may not be available for computationally expensive systems.
method Transfer learning techniques using high- and low-fidelity models, including standard transfer and bi-fidelity weighted learning.
result Bi-fidelity transfer learning improves accuracy over standard training approaches.
Transfer learning boosts chemically accurate neural network potentials for organic molecules.
problem Developing accurate interatomic potentials from ab-initio data.
method Discriminative fine-tuning of pre-trained neural networks.
result Fine-tuning with energy labels alone can achieve accurate atomic forces.
Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet, discrepancies between the underlying distributions of the source and target data are c…
MANGA transfers policies across environments with varying dynamics and noise.
problem Transferring policies across multiple environments with different dynamics and motor noise.
method Decouples policy learning from system identification, trains dynamics-conditioned policies, and learns dynamics parameters from rollouts.
result Demonstrates effective transfer of learned policies across four MuJoCo agents using agnostic RL and imitation learning methods.
Parameters in deep neural networks which are trained on large-scale databases can generalize across multiple domains, which is referred as "transferability". Unfortunately, the transferability is usually defined as discrete states and it differs with domains and network architectures. Existing works usually heuristical…
This work investigates how neural collapse improves transfer learning for large-scale models.
problem Improving transfer learning for large-scale models with limited labeled data.
method Investigates neural collapse and develops a fine-tuning method using skip-connections.
result Feature collapse on downstream data correlates with higher transfer accuracy.
HGKT transfers knowledge from seen to unseen classes in GZSL without prior unseen class info.
problem Learning to classify unseen classes in GZSL.
method Structured heterogeneous graph with graph neural network for knowledge transfer.
result Achieves state-of-the-art results on public benchmark datasets.
Tropical cyclone wind-intensity prediction is a challenging task considering drastic changes climate patterns over the last few decades. In order to develop robust prediction models, one needs to consider different characteristics of cyclones in terms of spatial and temporal characteristics. Transfer learning incorpora…
This paper proposes a method to train multiple neural networks with shared parameters using a reconstruction loss.
problem Training multiple neural networks for correlated tasks separately is inefficient.
method Introduces a novel approach with a reconstruction loss to encourage shared features across multiple tasks.
result The proposed method achieves efficient transfer learning with competitive performance.
Transfer Learning (TL) in Deep Neural Networks is gaining importance because in most of the applications, the labeling of data is costly and time-consuming. Additionally, TL also provides an effective weight initialization strategy for Deep Neural Networks . This paper introduces the idea of Adaptive Transfer Learning …
This work improves GNN training efficiency by maximizing ego-graph information.
problem Training dedicated GNNs is costly for large-scale graphs.
method Proposes EGI (Ego-Graph Information maximization) to capture essential graph information and establish a theoretical framework for transfer learning.
result Demonstrates the effectiveness of EGI in improving GNN training efficiency and transferability.
Transfer learning boosts deep learning in source code modeling.
problem Deep learning models are problem-specific and data-hungry.
method Transfer learning to improve performance of deep learning models.
result The proposed approach outperforms state-of-the-art models in source code suggestion task.
Transfer learning does not improve character recognition performance.
problem Improving character recognition performance using transfer learning.
method Performed experiments with varying levels of similarity between source and target tasks, transferring both parameters and features.
result No significant advantage gained by transfer learning over traditional machine learning.
Recently, several optimization methods have been successfully applied to the hyperparameter optimization of deep neural networks (DNNs). The methods work by modeling the joint distribution of hyperparameter values and corresponding error. Those methods become less practical when applied to modern DNNs whose training ma…
Transfer learning improves highway traffic forecasting using graph neural networks.
problem Lack of historical data for traffic forecasting on large highway networks.
method Developed a transfer learning approach for DCRNN, a graph neural network for highway forecasting.
result TL-DCRNN can forecast traffic on unseen regions of the highway network with high accuracy.