Study measures impact of data and neural net similarity on transferability in restaurant sales data.
problem Identify indicators for successful transferability of neural nets across different data sets.
method Empirical study on sales data from six restaurants, calculating indicators based on data and neural net similarities.
result Negative correlations between transferability and indicators, allowing better model performance and fewer transfers.
Physics-informed neural networks simulate radiative transfer efficiently.
problem Simulating radiative transfer accurately and efficiently.
method Physics-informed neural networks trained to minimize radiative transfer equations.
result PINNs provide an easy-to-implement, robust, and accurate method for radiative transfer simulation.
Size-independent neural transfer for RDDL planning.
problem Sample inefficiency and time-consuming training for neural planners of RDDL MDPs.
method Two key innovations: state encoder and parameter-tied action decoder.
result Powerful transfer across problem sizes with superior learning curves.
We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural A…
Paper introduces a new method for improving reinforcement learning performance using transfer learning.
problem Improving reinforcement learning performance with limited sample sizes in dynamic decision-making scenarios.
method Developed a novel ``re-weighted targeting procedure'' and ``transfer deep Q∗-learning'' approach. result Demonstrated improved reinforcement learning performance through strategic sample construction.
We extend graph neural networks to transfer performance across different input sizes.
problem Transferability of graph neural networks across varying input dimensions.
method Introduce a general framework for transferability across dimensions, showing it corresponds to continuity in a limit space.
result Transferability of graph neural networks is driven by data and learning task, and can be ensured with design principles.
Measures neural network information transfer for generalization.
problem Estimating the generalizable information in neural networks.
method Proposes Information Transfer (LIT) based on prequential coding. result Consistently correlates with generalizable information in neural networks.
New attention mechanism improves meta-transfer learning in dynamic tasks.
problem Underfitting in meta-transfer learning with dynamic tasks.
method Proposed Recurrent Memory Reconstruction (RMR) attention mechanism.
result ASNP-RMR significantly outperforms baselines in various tasks.
Paper shows pre-training and transfer learning reduce sample complexity for neural networks.
problem Training high-dimensional supervised learning with limited labeled data.
method Study of single-layer neural networks via online stochastic gradient descent, considering concept shift.
result Pre-training and transfer learning reduce sample complexity by polynomial factors under general assumptions.
Study shows benefits of transfer learning with neural networks.
problem Understanding generalization errors in transfer learning.
method Mean-field analysis applied to α-ERM and fine-tuning. result Established conditions for generalization error and convergence rates.
Simplifies transfer learning with deep neural networks using ridge regression.
problem High computational cost of finetuning deep models for transfer learning.
method Leverage the low-rank property of deep neural networks' feature vectors in kernel ridge regression.
result Successful on supervised and semi-supervised transfer learning tasks.
The paper establishes limits of transfer learning with neural networks.
problem Understanding the fundamental limits of transfer learning.
method Statistical minimax framework for regression with linear and neural network models.
result Lower bounds for target generalization error.
Study on neural networks' performance in sequential task learning.
problem Understanding the performance of neural networks in sequential task learning.
method Theoretical analysis of generalization performance in continual learning using statistical mechanical analysis of kernel ridge-less regression.
result Characteristic transitions from positive to negative transfer observed in neural networks.
Connections between nodes of fully connected neural networks are usually represented by weight matrices. In this article, functional transfer matrices are introduced as alternatives to the weight matrices: Instead of using real weights, a functional transfer matrix uses real functions with trainable parameters to repre…
Paper proposes transferring neural network learning trajectories.
problem Training deep neural networks is computationally expensive.
method Formulated and solved the learning transfer problem by matching gradients.
result Transferred parameters achieve non-trivial accuracy before training.
Improves transfer learning performance with Lautum regularization.
problem Improving transfer learning performance with limited labeled data.
method Proposes Lautum information regularization for semi-supervised transfer learning.
result Demonstrates improved transferability of deep neural networks.
New method improves adversarial transferability from Bayesian neural networks.
problem Improving the effectiveness of black-box evasion attacks.
method Sampling from the posterior distribution of neural network weights to build a surrogate.
result Significantly improved success rates of state-of-the-art attacks (up to 83.2 percentage points).
Proof of learning rate transfer in MLPs with μP parameterization.
problem Understanding and optimizing learning rates in neural networks with different parameterizations.
method Theoretical analysis and empirical validation of learning rate transfer in MLPs with μP, SP, and NTP parameterizations. result The optimal learning rate converges to a non-zero constant as width goes to infinity under μP, explaining learning rate transfer. Anti-transfer learning prevents misleading representations for speech tasks.
problem Misleading representations learned from orthogonal tasks in speech processing.
method Penalizes similarity between activations of a network and another trained on an orthogonal task.
result Improves classification accuracy and invariance to the orthogonal task.
Improved drug-protein interaction prediction using FTL method.
problem Predicting drug-protein interactions from noisy data with uncertain labels.
method Filtered Transfer Learning (FTL) method that fine-tunes a deep neural network across multiple tiers of data confidence.
result FTL method outperforms deep neural networks trained on single confidence ranges.
A powerful network teaches a weak one, improving its performance.
problem Improving the performance of a weak neural network using a more powerful one.
method During training, a weak network learns features from a strong network to minimize feature distance.
result A weak neural network can increase its performance by learning from a more powerful network.
Paper shows spectral filters can transfer between different graphs discretizing the same space.
problem Transferability of spectral filters between different graphs.
method Analysis of spectral filters on graphs discretizing the same space.
result Spectral filters have similar effects on graphs discretizing the same space.
This work extends GNNs to handle multiple graphs with non-commuting operators, proving transferability.
problem Handling multiple graphs with non-commuting operators in graph neural networks.
method Developed a mathematical theory for graph-tuple neural networks (GtNNs) with non-commuting non-expansive operators.
result Proved universal transferability of GtNNs, ensuring no non-transferable energy under convergence.
Paper proposes CNN-based time series anomaly detection with transfer learning.
problem Time series anomaly detection in automated monitoring systems.
method CNN for segmentation, transfer learning framework, fine-tuning on unseen classes.
result Successfully tested on multiple synthetic and real data sets.
Transfer learning for deep neural networks is the process of first training a base network on a source dataset, and then transferring the learned features (the network's weights) to a second network to be trained on a target dataset. This idea has been shown to improve deep neural network's generalization capabilities …
Improved sales forecasting for new products using transfer learning.
problem Insufficient training data for new products leads to inaccurate sales forecasts.
method Network-based Transfer Learning approach for deep neural networks.
result Deep neural networks' prediction accuracy for food sales forecasting can be effectively increased.
Paper classifies Parkinson's disease from speech in three languages using CNNs and transfer learning.
problem Classifying Parkinson's disease from speech in multiple languages.
method Convolutional Neural Networks (CNNs) with transfer learning among Spanish, German, and Czech.
result Transfer learning improves model accuracy by up to 8% and balances specificity-sensitivity.
This paper reviews transfer learning for financial data predictions, highlighting its potential.
problem Accurate stock price prediction in financial time series is challenging due to noise and non-linear relationships.
method Transfer Learning applied to financial market predictions.
result Transfer Learning can improve financial prediction capability.
New attack method makes poisoned images transferable without access to victim model.
problem Injecting poisoned images into training data to mislead deep nets.
method Designing poison images to surround targeted images in feature space, using Dropout.
result Achieves over 50% transferable success with only 1% poisoning of training set.
Deep tabular models outperform GBDT in medical diagnosis tasks.
problem Transfer learning for tabular data in medical diagnosis.
method Proposes a pseudo-feature method for transfer learning between different feature sets.
result Tabular neural networks outperform GBDT in medical diagnosis tasks.
Transfer learning from breast histopathology improves prostate cancer detection.
problem Insufficient prostate histopathology datasets for training.
method Transfer learning from breast histopathology images.
result Proposed approach outperforms transfer learning from ImageNet dataset.
Study improves infant cry-based asphyxia diagnosis using transfer learning.
problem Improving accuracy of diagnosing perinatal asphyxia in newborns.
method Neural transfer learning from adult speech to infant cries.
result Transfer learning models are resilient to noise and signal loss.
Combines NES and PPO to enhance exploration in various environments.
problem Improving exploration in reinforcement learning environments.
method Parameter transfer and parameter space noise methods for combining NES and PPO.
result PPO benefits from both NES methods in discrete and continuous control tasks.
Transfer learning improves algorithm recommendation performance.
problem Improving algorithm recommendation with limited data.
method Train a neural network on meta-datasets, then transfer knowledge to similar datasets.
result Transfer learning enhances meta-learning for algorithm recommendation.
XMixup improves transfer learning accuracy by 1.9% with less training time.
problem Efficiently transfer knowledge from large source datasets to target tasks with small samples.
method Cross-domain Mixup technique that selects auxiliary samples from source datasets and augments training samples via mixup strategy.
result Improves accuracy by 1.9% on average over six real-world transfer learning datasets.
Graph neural networks can be adapted to new graphs with a limit object called graphon NNs.
problem Transferability of graph neural networks across different graphs.
method Introduced graphon NNs as limit objects of GNNs and proved a bound on the difference between GNN and graphon-NN outputs.
result The bound on the difference between GNN and graphon-NN outputs vanishes with growing number of nodes if the graph convolutional filters are bandlimited.
Efficiently transfers style to content without distorting the content structure.
problem Arbitrary style transfer in computer vision.
method Rigid alignment of style features to content features.
result High-quality stylized images with intact content structure.
Reduces data transfer for neural network inference on limited bandwidth.
problem Limited bandwidth during neural network inference.
method Automatic selection of relevant input data parts.
result Significant reduction in data transfer without compromising model quality.
Parameters in deep neural networks which are trained on large-scale databases can generalize across multiple domains, which is referred as "transferability". Unfortunately, the transferability is usually defined as discrete states and it differs with domains and network architectures. Existing works usually heuristical…
This work improves GNN training efficiency by maximizing ego-graph information.
problem Training dedicated GNNs is costly for large-scale graphs.
method Proposes EGI (Ego-Graph Information maximization) to capture essential graph information and establish a theoretical framework for transfer learning.
result Demonstrates the effectiveness of EGI in improving GNN training efficiency and transferability.
Survey explores how transfer learning improves deep reinforcement learning.
problem Challenges in reinforcement learning efficiency and effectiveness.
method Categorizes and analyzes transfer learning approaches.
result Transfer learning enhances reinforcement learning performance.
Develops hyperparameter transfer methods for Dense Associative Memories.
problem Challenges in transferring hyperparameters for DenseAMs due to unique architecture and activation functions.
method Derives explicit prescriptions for hyperparameter transfer from small to large models.
result Excellent agreement between theoretical and empirical results.
Transfer learning adapted for hybrid classical-quantum neural networks.
problem Optimizing data preprocessing and feature embedding for quantum processors.
method Adapting transfer learning to hybrid networks, using a pre-trained classical network augmented by a quantum circuit.
result Demonstrated the effectiveness of quantum transfer learning for image recognition and quantum state classification.
Survey on self-supervised pre-training for neural networks using unlabeled data.
problem Improving model performance using unlabeled data.
method Pre-training on unlabeled data followed by task-specific adaptation.
result Enhanced model performance through self-supervised pre-training.
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
problem Efficient transfer of knowledge in reinforcement learning.
method Behavior Transfer (BT) that uses pre-trained policies for exploration.
result BT combined with pre-training leads to better solutions than without pre-training.
ANN predicts methane heat transfer in rocket engines efficiently.
problem Accurate heat transfer prediction for supercritical methane in rocket engine cooling channels.
method Artificial neural networks trained on CFD simulation data.
result ANN model predicts maximum wall temperature with convincing precision.
Transfer learning improves image classifier performance in data-starved regimes.
problem Deploying image classifiers in domains with limited labeled data.
method Transfer learning with deep neural networks, focusing on feature reuse and overparameterization.
result Transfer learning enhances CNN performance in data-starved regimes.
Randomized Geometric Algebra for Convex Neural Networks Optimizes Transfer Learning.
problem Training neural networks to global optimality via convex optimization.
method Randomized algorithms in Clifford's Geometric Algebra for hypercomplex vector spaces.
result Convex optimization and geometric algebra improve LLMs' robustness and reliability in transfer learning.