Effective multilingual search with instance-based transfer learning.
problem Search in multilingual setting, especially next-sentence prediction and inverse cloze.
method Instance-based transfer learning, analyzing vocabulary overlaps and transitive overlaps.
result Positive transfer on all 35 target languages and two tasks, even with no vocabulary overlap.
In recent years, supervised machine learning models have demonstrated tremendous success in a variety of application domains. Despite the promising results, these successful models are data hungry and their performance relies heavily on the size of training data. However, in many healthcare applications it is difficult…
Transfer learning framework for fragility modeling under domain shift and class imbalance
problem Data gaps in structural fragility modeling
method Transfer learning
result Improves failure detection and predictive stability in low-data regimes
The paper investigates instance-based interpretation methods for VAEs.
problem Understanding how VAEs make predictions without supervision.
method Formally framed influence functions for VAEs and developed VAE-TracIn.
result Varying training samples significantly impacts VAE predictions.
In multi-task learning, a learner is given a collection of prediction tasks and needs to solve all of them. In contrast to previous work, which required that annotated training data is available for all tasks, we consider a new setting, in which for some tasks, potentially most of them, only unlabeled training data is …
Object ranking or "learning to rank" is an important problem in the realm of preference learning. On the basis of training data in the form of a set of rankings of objects represented as feature vectors, the goal is to learn a ranking function that predicts a linear order of any new set of objects. In this paper, we pr…
We describe a new instance-based learning algorithm called the Boundary Forest (BF) algorithm, that can be used for supervised and unsupervised learning. The algorithm builds a forest of trees whose nodes store previously seen examples. It can be shown data points one at a time and updates itself incrementally, hence i…
Imbalanced classification has been a major challenge for machine learning because many standard classifiers mainly focus on balanced datasets and tend to have biased results towards the majority class. We modify entropy fuzzy support vector machine (EFSVM) and introduce instance-based entropy fuzzy support vector machi…
New efficient algorithm for approximate PML distribution.
problem Computing the profile maximum likelihood (PML) distribution efficiently.
method Exploiting sparsity structure and new matrix rounding algorithm.
result First provable computationally efficient implementation of PseudoPML.
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learni…
Paper presents a Transformer model for automatic domain adaptation.
problem Challenges in selecting or designing domain adaptation algorithms.
method Transformer model approximates and selects domain adaptation algorithms.
result Transformers can approximate and automatically select domain adaptation algorithms.
Paper introduces Native Guide for generating time series counterfactual explanations.
problem Lack of explainability for time series data in AI systems.
method Model-agnostic, instance-based counterfactual generation for time series classification.
result Native Guide produces better counterfactual explanations than benchmarks.
Classification is a fundamental problem in machine learning and data mining. During the past decades, numerous classification methods have been presented based on different principles. However, most existing classifiers cast the classification problem as an optimization problem and do not address the issue of statistic…
Study how communication and feedback graphs affect learning outcomes.
problem Understanding the impact of feedback graphs on cooperative online learning.
method Analyzed network regret in terms of the independence number of the strong product of communication and feedback graphs.
result Proved bounds for network regret and demonstrated the non-improvable nature of positive results in pathological cases.
We address the problem of learning fine-grained cross-modal representations. We propose an instance-based deep metric learning approach in joint visual and textual space. The key novelty of this paper is that it shows that using per-image semantic supervision leads to substantial improvement in zero-shot performance ov…
Paper analyzes transfer risk in transfer learning for finance.
problem Evaluate transferability of transfer learning in finance.
method Proposes transfer risk concept and applies to stock return prediction and portfolio optimization.
result Transfer risk correlates with transfer learning performance and identifies appropriate source tasks.
Transfer learning improves portfolio optimization by identifying transfer risk.
problem Financial portfolio optimization problem.
method Introduces transfer risk concept within transfer learning framework.
result Transfer risk is a significant indicator of transferability and enhances portfolio management efficiency.
Mathematical framework for transfer learning feasibility and transfer risk.
problem Theoretical analysis of transfer learning.
method Reformulated transfer learning as an optimization problem, introduced transfer risk concept.
result Demonstrated the potential and benefits of incorporating transfer risk in transfer learning evaluation.
Transfer learning borrows knowledge from a source domain to facilitate learning in a target domain. Two primary issues to be addressed in transfer learning are what and how to transfer. For a pair of domains, adopting different transfer learning algorithms results in different knowledge transferred between them. To dis…
Survey connects and systematizes transfer learning research.
problem Reduce dependence on target domain data for target learners.
method Systematic review of 40+ transfer learning approaches.
result Importance of choosing appropriate transfer learning models.
Recently, researches related to unsupervised disentanglement learning with deep generative models have gained substantial popularity. However, without introducing supervision, there is no guarantee that the factors of interest can be successfully recovered. Motivated by a real-world problem, we propose a setting where …
New research on limits of transfer learning, proving key selection and dependence requirements.
problem Insufficient theoretical foundation for transfer learning.
method Proved novel results on transfer learning, emphasizing selection of information and dependence between domains.
result Upper bound on improvement possible with transfer learning, highlighting the need for careful selection.
The paper analyzes phase transitions in transfer learning for perceptrons.
problem Understanding when transfer learning from a source task to a target task is beneficial.
method Theoretical analysis of a pair of related perceptron learning tasks.
result Reveals a phase transition from negative to positive transfer as task similarity changes.
This paper explores the connection between adversarial and knowledge transferability.
problem Understanding the factors affecting knowledge transferability.
method Theoretical analysis and practical metrics for adversarial transferability.
result Adversarial transferability and knowledge transferability are closely related.
Investigates transfer learning in spatial statistics.
problem Applying transfer learning to spatial statistics.
method Simple MLP models for spatial data.
result Potential of transfer learning in spatial statistics.
Proposes a transfer learning method for high-dimensional quantile regression.
problem Inadequate handling of heterogeneity and heavy tails in transfer learning.
method High-dimensional quantile regression framework with double transfer learning estimator.
result Established error bounds and valid confidence intervals for high-dimensional quantile regression coefficients.
Paper introduces a new method for improving reinforcement learning performance using transfer learning.
problem Improving reinforcement learning performance with limited sample sizes in dynamic decision-making scenarios.
method Developed a novel ``re-weighted targeting procedure'' and ``transfer deep Q∗-learning'' approach. result Demonstrated improved reinforcement learning performance through strategic sample construction.
In the context of post-hoc interpretability, this paper addresses the task of explaining the prediction of a classifier, considering the case where no information is available, neither on the classifier itself, nor on the processed data (neither the training nor the test data). It proposes an instance-based approach wh…
Transfer learning aims at building robust prediction models by transferring knowledge gained from one problem to another. In the semantic Web, learning tasks are enhanced with semantic representations. We exploit their semantics to augment transfer learning by dealing with when to transfer with semantic measurements an…
Transfer learning does not improve character recognition performance.
problem Improving character recognition performance using transfer learning.
method Performed experiments with varying levels of similarity between source and target tasks, transferring both parameters and features.
result No significant advantage gained by transfer learning over traditional machine learning.
ART improves transfer learning performance with robust theory and methods.
problem Improving performance of primary tasks using auxiliary data.
method Adaptive Robust Transfer Learning (ART) pipeline with theoretical guarantees.
result ART provides a provable theoretical guarantee for adaptive transfer and robustness.
Defines related tasks for transfer learning using foliations.
problem Lack of a foundational description of related tasks in transfer learning.
method Introduces foliations as a mathematical framework for relatedness between tasks.
result Identifies foliations as a way to represent relatedness in transfer learning.
Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
We observe standard transfer learning can improve prediction accuracies of target tasks at the cost of lowering their prediction fairness -- a phenomenon we named discriminatory transfer. We examine prediction fairness of a standard hypothesis transfer algorithm and a standard multi-task learning algorithm, and show th…
LEEP measures transferability of learned representations efficiently.
problem Evaluating the transferability of learned representations in machine learning.
method LEEP: Log Expected Empirical Prediction, a simple measure requiring one pass through the target data set.
result LEEP predicts transfer and meta-transfer learning performance and convergence speed, outperforming existing measures.
Survey explores how transfer learning improves deep reinforcement learning.
problem Challenges in reinforcement learning efficiency and effectiveness.
method Categorizes and analyzes transfer learning approaches.
result Transfer learning enhances reinforcement learning performance.
Paper tackles continuous transfer learning with evolving target domains.
problem Challenges of negative transfer in evolving target domains.
method Proposes label-informed C-divergence for measuring distribution shift and negative transfer.
result Demonstrates effectiveness of TransLATE framework in minimizing classification error and C-divergence.
DERWENT learns paths for distant transfer learning via deep random walk.
problem Transfer learning between distant domains is challenging.
method DERWENT uses deep random walk to explicitly learn paths between source and target domains.
result DERWENT achieves state-of-the-art performance on benchmark datasets.
New analysis shows transfer learning can significantly reduce sample size for complex models.
problem Reducing sample size needed for complex models like large language models.
method Optimal transport viewpoint applied to analyze transfer learning efficiency.
result Transfer learning can achieve better sample efficiency for complex models.
Survey on negative transfer in machine learning.
problem Negative transfer in transfer learning reduces target domain performance.
method Systematic review of 50+ approaches to mitigate negative transfer.
result Lack of systematic survey on negative transfer.
Bayesian methods improve learning from related domains.
problem Finding optimal information to transfer between domains.
method Bayesian approaches to transfer learning.
result Bayesian methods outperform frequentist competitors in simulations.
Paper tackles robust transfer learning with unreliable source data.
problem Challenges in robust transfer learning stemming from ambiguity in Bayes classifiers and weak transferable signals.
method Introduces ambiguity level, proposes Transfer Around Boundary (TAB) model, establishes general theorem.
result Demonstrates efficiency and robustness of TAB model improving classification while avoiding negative transfer.
We consider a transfer-learning problem by using the parameter transfer approach, where a suitable parameter of feature mapping is learned through one task and applied to another objective task. Then, we introduce the notion of the local stability and parameter transfer learnability of parametric feature mapping,and th…
The paper investigates what enables successful transfer learning and separates feature reuse from data statistics.
problem Understanding what enables successful transfer learning and identifying the responsible parts of the network.
method Analyzes transfer learning on block-shuffled images to distinguish feature reuse from data statistics.
result Some benefit of transfer learning comes from learning low-level statistics of data, not just feature reuse.
Adversarial perturbations fool wearable sensor systems, showing transferability across different systems.
problem Adversarial examples fool wearable sensor systems, showing transferability across different systems.
method Study of adversarial transferability in wearable sensor systems from four perspectives: systems, subjects, sensor body locations, and datasets.
result Strong untargeted transferability in most cases, targeted attacks less successful.
Paper proposes a statistical test for transfer learning in linear regression.
problem Theoretical framework for parameter transfer in linear regression.
method Developed a statistical test to predict transfer quality.
result The test can predict if a fine-tuned model has lower prediction risk.
Double descent in transfer learning explained for linear regression problems.
problem Understanding generalization errors in transferring parameters between overparameterized linear regression tasks.
method Analytical characterization of generalization error in terms of transfer learning factors.
result Generalization error follows a two-dimensional double descent trend controlled by transfer learning factors.
Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities. However, most of existing work only concentrates on learning shared feature representation by minimizing the distribution discrepancy across different domains. Due to the fact that all the domai…