Paper tackles transferring knowledge for learning distance metrics across different domains.
problem Mitigating label information deficiency in target distance metric learning.
method Develops a flexible HTDML framework that uses knowledge fragments from source domain to help target metric learning.
result Reduces generalization error in the target domain using the proposed transfer strategy.
TML uses knowledge from related domains to improve metric learning in target domain.
problem Insufficient label information in real-world applications.
method Leveraging knowledge from related domains to improve metric learning in target domain.
result TML can improve metric learning performance in target domain.
Develops a method to learn metrics across multiple domains using heterogeneous transfer learning.
problem Limited labeled data in target domain and heterogeneous data across multiple domains.
method HMTML framework that learns metrics and transformations across multiple domains, maximizing high-order covariance in a common subspace.
result Effective feature transformations and metrics learned across multiple domains, validated by extensive experiments.
This paper explores the connection between adversarial and knowledge transferability.
problem Understanding the factors affecting knowledge transferability.
method Theoretical analysis and practical metrics for adversarial transferability.
result Adversarial transferability and knowledge transferability are closely related.
BeST metric selects best source models for transfer learning.
problem Efficiently selecting top source models for a new target task.
method Quantization-level optimization procedure for similarity measure.
result BeST consistently identifies best source models for target tasks.
Optimal transport metric transfers deep network representations efficiently.
problem Efficiently transfer deep network representations for new tasks.
method Use optimal transport to quantify representation similarity and regularize student network.
result Optimal transport distance promotes similarity between teacher and student representations.
Deep learning improves handwriting style transfer and extraction.
problem Improving handwriting style transfer and extraction using deep neural networks.
method Used a deep conditioned autoencoder on IRON-OFF handwriting data-set to explore style transfer and extraction.
result Improved metrics of state-of-the-art methods by a large margin in style transfer and extraction experiments.
The paper proposes a uniformity regularization scheme to improve deep neural network transferability.
problem Improving deep neural network transferability and adaptation to new tasks.
method Introduces a uniformity regularization scheme to encourage high uniformity in embedding space.
result Uniformity regularization consistently offers benefits over baseline methods and achieves state-of-the-art performance in Deep Metric Learning and Meta-Learning.
New metrics solve machine learning limitations.
problem Previous success metrics restrict application to specific forms of machine learning.
method Define decomposable metrics as linear operations on probability distributions.
result Demonstrated theorems bounding success in various ways, generalizing existing results.
ProtoTransfer learns from unlabeled data to classify unseen tasks.
problem Few-shot classification with limited labeled data.
method Self-supervised prototypical transfer learning.
result ProtoTransfer outperforms unsupervised meta-learning methods.
Enhances generative model accuracy through knowledge transfer.
problem Improving generative model precision across different tasks.
method Introduces a novel framework for transfer learning using shared structures.
result Demonstrates enhanced performance in diffusion and normalizing flows models.
Paper proposes a few-shot learning method for human activity recognition.
problem Efficiently recognize human activities with limited labeled data.
method Uses deep learning for feature extraction and knowledge transfer from existing models.
result Promising results show the proposed method's advantages over traditional approaches.
The focus in machine learning has branched beyond training classifiers on a single task to investigating how previously acquired knowledge in a source domain can be leveraged to facilitate learning in a related target domain, known as inductive transfer learning. Three active lines of research have independently explor…
Deconfounds neural network representation similarity metrics to improve consistency and accuracy.
problem Confounding by population structure in similarity metrics like RSA and CKA.
method Covariate adjustment regression to adjust for confounders.
result Improves detection of semantically similar neural networks and consistency in transfer learning.
Study measures impact of data and neural net similarity on transferability in restaurant sales data.
problem Identify indicators for successful transferability of neural nets across different data sets.
method Empirical study on sales data from six restaurants, calculating indicators based on data and neural net similarities.
result Negative correlations between transferability and indicators, allowing better model performance and fewer transfers.
New landmark states improve transfer learning in multi-task RL.
problem Improving sample complexity and regret in new RL tasks.
method Topological landmark covering, landmark value functions, action pruning.
result Theoretical bounds on Q values at state-action pairs.
Transfer learning improves model robustness against adversarial attacks.
problem Understanding how transfer learning affects model robustness against adversarial attacks.
method Extensive empirical evaluations of white-box and black-box attacks on fine-tuned transfer learning models.
result Adversarial examples are more transferable when fine-tuning is used than when networks are trained independently.
This study investigates how much knowledge from natural images can be transferred to pathology images.
problem Quantifying how much knowledge from natural images can be transferred to pathology images.
method Proposes a framework to quantify knowledge gain by a particular layer, conducts empirical investigation in pathology image centered transfer learning.
result Early layers of deep models can transfer knowledge to pathology image classification tasks.
New method for RL tasks transfer using Lipschitz continuity.
problem Knowledge transfer in RL tasks over time.
method Established Lipschitz continuity between MDPs and applied it to RL.
result Improved convergence rate and no negative transfer with high probability.
URL benchmark evaluates uncertainty quantification in pretrained models.
problem Need for reliable uncertainty estimates in transferable pretrained models.
method Proposes URL benchmark to measure transferability of representations and uncertainty estimates.
result Transferable uncertainty quantification remains challenging but not contradictory to traditional goals.
We define a class of metrics that extend the Sasaki metric of a tangent manifold of a Riemannian manifold. The new metrics are obtained by the transfer of the generalized (pseudo-)Riemannian metrics of the pullback of the big tangent bundle of a manifold to the tangent manifold. We obtain the expression of the transfer…
Improved CSKS with limited data using novel loss functions and transfer learning.
problem Spotting keywords in continuous speech with limited training data.
method Combination of Prototypical networks' loss and metric loss with transfer learning.
result Improves F1 score by over 10%
This paper investigates why adversarial attacks can be effective against different models.
problem Understanding why adversarial attacks transfer between models.
method Developed a unifying optimization framework and formal definition for attack transferability.
result Identified two main factors: target model's vulnerability and surrogate model complexity.
Anti-transfer learning prevents misleading representations for speech tasks.
problem Misleading representations learned from orthogonal tasks in speech processing.
method Penalizes similarity between activations of a network and another trained on an orthogonal task.
result Improves classification accuracy and invariance to the orthogonal task.
Bayesian method transfers knowledge between brain tumor datasets.
problem Applying deep learning to small medical datasets.
method Generative Bayesian Prior network for knowledge transfer.
result Best results in Dice Similarity Coefficient for BRATS2018.
New method for one-shot timbre transfer in music.
problem One-shot timbre transfer in music.
method Self-supervised VQ-VAE extension for disentangled representations.
result Outperforms selected baselines on objective metrics.
Novel method transfers orometric measures to metric data sets, identifying key items.
problem Identifying key items in metric data sets like knowledge graphs.
method Transfers orometric measures to bounded metric spaces, using 'isolation' and 'prominence' functions.
result Identifies structurally relevant items in geographic data sets of Germany and France.
Bayesian method transfers knowledge between brain tumor datasets for MRI segmentation.
problem Limited applicability of deep learning methods for small medical datasets.
method Generative Bayesian Prior network for knowledge transfer.
result Best results in Dice Similarity Coefficient metric for BRATS2018.
DM improves self-supervised transfer learning by matching target distributions.
problem Improving self-supervised transfer learning performance.
method Distribution Matching (DM) method that drives representation distribution towards a predefined reference distribution.
result DM outperforms existing methods on target classification tasks.
A modular GP framework for efficient transfer learning.
problem Efficiently transfer knowledge across different tasks or datasets.
method Modular variational Gaussian processes (GPs) with a dictionary of well-fitted GPs.
result Reduces computational costs and allows the transfer of uncertainty metrics.
CNNs improve medical image classification with few samples.
problem Classifying medical images with limited training data.
method Transfer learning using CNNs, representation extraction, and a novel metric for performance prediction.
result CNN-based transfer learning outperforms feature-based methods with high correlation to test set performance.
This study evaluates how adversarial examples transfer between different models.
problem Transferability of adversarial examples across models poses a threat to machine learning reliability.
method Evaluation of three adversarial attacks (FGSM, Basic Iterative Method, Carlini & Wagner) on two model classes (VGG and Inception). Use of specific parameters and metrics (L-Infinity clipping, SSIM) for assessment.
result Adversarial examples can be transferred between models, indicating a vulnerability in machine learning systems.
The paper introduces metrics to evaluate NILM algorithms' performance on unseen buildings.
problem Assessing NILM algorithms' performance on new, unseen buildings.
method Developed several metrics to evaluate NILM algorithms' generalization ability.
result Demonstrated the utility of the proposed metrics through two case studies.
Graph few-shot learning improves node classification with prior knowledge transfer.
problem Challenging semi-supervised node classification with few labeled nodes.
method Incorporates prior knowledge from auxiliary graphs into a transferable metric space.
result Improves classification accuracy on target graphs.
We propose a framework that learns a representation transferable across different domains and tasks in a label efficient manner. Our approach battles domain shift with a domain adversarial loss, and generalizes the embedding to novel task using a metric learning-based approach. Our model is simultaneously optimized on …
New theory explains when pre-trained models can improve downstream tasks.
problem Lack of theoretical understanding of task similarity for transfer learning.
method Feature-centric viewpoint, theoretical results on transferability phase diagram.
result Transfer learning outperforms training from scratch when target task is well represented in feature space.
Domain adaptation leverages the knowledge in one domain - the source domain - to improve learning efficiency in another domain - the target domain. Existing heterogeneous domain adaptation research is relatively well-progressed, but only in situations where the target domain contains at least a few labeled instances. I…
Few-shot NER classification with limited data.
problem Limited annotated data for Named Entity Recognition.
method Prototypical Network + Transfer Learning
result Well-performing classifiers trained on only 20 instances.
FiT combines transfer and meta-learning for efficient few-shot image classification.
problem Few-shot image classification in personalized and federated learning settings.
method Combines transfer learning and meta-learning with fixed pretrained backbones and fine-tuned FiLM adapter layers.
result Achieves state-of-the-art accuracy on VTAB-1k benchmark with fewer than 1% of updateable parameters.
Paper improves feature selection accuracy using transfer learning.
problem Improving feature selection accuracy in information criteria-based methods.
method Proposes TLCp, a transfer learning procedure based on Mallows' Cp.
result TLCp outperforms conventional Cp in accuracy and stability.
New insights into continual learning with task similarity.
problem Challenges in learning similar tasks without interference.
method Linear teacher-student model with latent structure.
result High input feature similarity with low readout similarity is catastrophic.
This work studies adversarial transferability and proposes ensemble methods to improve robustness.
problem Adversarial transferability in neural networks and its implications for robustness.
method Investigates the effect of various factors on adversarial transferability and proposes ensemble attack methods.
result Transferability is significantly hampered by input quantization and architectural mismatch, but not by initialization.
This paper quantifies hyperparameter transfer and finds embedding layer learning rate is key.
problem Quantifying optimal hyperparameters for large language models across scales.
method Developed three metrics to quantify hyperparameter transfer and investigated the importance of embedding layer learning rate.
result Maximal Update (μP) parameterization offers high-quality learning rate transfer compared to standard parameterization (SP).
The paper develops methods to create fair and transferable representations without subgroup discrimination.
problem Creating fair and transferable representations without discriminating subgroups in the population.
method The approach involves modifying data representations to meet fairness constraints, leveraging task similarities via low rank matrix factorization.
result The learned fair representation transfers well to novel tasks, improving prediction performance and fairness metrics.
Autoencoder selects relevant source samples for self-taught learning.
problem Negative transfer from irrelevant source samples.
method Autoencoder with ℓ2,1-norm sparsity constraint for relevance metric. result Promising results in expanded training set for classifier training.
Proposes Gaussian optimal transport for image style transfer.
problem Image style transfer and mixing different artistic styles.
method Encoder/Decoder framework with optimal transport for Gaussian measures.
result Simple methodology for generating stylized content interpolating between many styles.
Study uses deep learning to predict mycotoxin levels in Irish oats.
problem Predicting mycotoxin contamination in Irish oats to improve crop quality and safety.
method Investigated neural networks and transfer learning models for multi-response prediction.
result Transfer learning model TabPFN provided the best performance.
LIMP learns latent shapes with metric preservation, improving generative models.
problem Insufficient training data for high-fidelity latent representations.
method Metric preservation as a prior, geometric distortion criterion, geodesic loss.
result Synthetic samples of higher quality achieved through metric preservation.