This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
Unsupervised domain adaptation is the problem setting where data generating distributions in the source and target domains are different, and labels in the target domain are unavailable. One important question in unsupervised domain adaptation is how to measure the difference between the source and target domains. A pr…
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
Appropriately evaluating the discrepancy between domains is essential for the success of unsupervised domain adaptation. In this paper, we first point out that existing discrepancy measures are less informative when complex models such as deep neural networks are used, in addition to the facts that they can be computat…
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
Proposes TFDF to learn transferable and discriminative features for unsupervised domain adaptation.
problem Difficult to induce supervised classifier without labeled data in unsupervised domain adaptation.
method TFDF optimizes transferability and discriminability by aligning distributions and minimizing class confusion.
result TFDF achieves better performance on real-world datasets compared to existing methods.
Private algorithms adapt from public to private domains with minimal labeled data.
problem Adapting from a public source domain to a private target domain with few labeled data.
method Differentially private discrepancy minimization algorithms based on Frank-Wolfe and Mirror-Descent methods.
result Effective adaptation with strong generalization and privacy guarantees.
Recent unsupervised approaches to domain adaptation primarily focus on minimizing the gap between the source and the target domains through refining the feature generator, in order to learn a better alignment between the two domains. This minimization can be achieved via a domain classifier to detect target-domain feat…
A new method for domain generalization using unlabeled data.
problem Learning from multiple domains with limited labeled data.
method Combines meta learning and semi-supervised learning with entropy-based pseudo-labeling and discrepancy loss.
result Significantly outperforms state-of-the-art methods on benchmark datasets.
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
Supervised learning results typically rely on assumptions of i.i.d. data. Unfortunately, those assumptions are commonly violated in practice. In this work, we tackle such problem by focusing on domain generalization: a formalization where the data generating process at test time may yield samples from never-before-seen…
Study shows cross-domain X-ray prediction performance discrepancies and label shifts.
problem Quantifying generalization limits across different X-ray datasets.
method Large-scale study on multiple X-ray datasets, focusing on performance and label shifts.
result Interesting discrepancies found between model performance and agreement, and concept similarity across tasks.
Transfer learning has been demonstrated to be successful and essential in diverse applications, which transfers knowledge from related but different source domains to the target domain. Online transfer learning(OTL) is a more challenging problem where the target data arrive in an online manner. Most OTL methods combine…
Deep learning has recently been shown to be instrumental in the problem of domain adaptation, where the goal is to learn a model on a target domain using a similar --but not identical-- source domain. The rationale for coupling both techniques is the possibility of extracting common concepts across domains. Considering…
Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
The learning of domain-invariant representations in the context of domain adaptation with neural networks is considered. We propose a new regularization method that minimizes the discrepancy between domain-specific latent feature representations directly in the hidden activation space. Although some standard distributi…
In this work, we connect two distinct concepts for unsupervised domain adaptation: feature distribution alignment between domains by utilizing the task-specific decision boundary and the Wasserstein metric. Our proposed sliced Wasserstein discrepancy (SWD) is designed to capture the natural notion of dissimilarity betw…
Stochastic Stein Discrepancies improve inference efficiency.
problem Intractable computation of Stein discrepancies.
method Subsampled approximations of Stein operators.
result Stochastic Stein Discrepancies inherit convergence properties of standard SDs.
Unified analysis of generalization and sample complexity for semi-supervised domain adaptation.
problem Theoretical foundations of domain adaptation remain underexplored, especially for modern approaches.
method Unified theoretical study of domain adaptation algorithms based on domain alignment, considering joint learning of feature transformations and shared classifiers in a semi-supervised setting.
result Unified theoretical analysis of domain adaptation algorithms, providing generalization bounds and sample complexity bounds for MMD and adversarial models.
Introduces Cauchy-Schwarz divergence for domain adaptation.
problem Evaluating discrepancy between source and target domains in unsupervised domain adaptation.
method Introduces Cauchy-Schwarz divergence as a measure for evaluating discrepancy between marginal and conditional distributions.
result CS divergence offers a tighter generalization error bound than Kullback-Leibler divergence.
DRDA robustly adapts models across domains with mismatched distributions.
problem Vulnerability of DA methods to noise and inability to generalize to unseen samples.
method DRDA uses distributionally robust optimization (DRO) with MMD metric to learn robust decision functions.
result DRDA outperforms existing robust learning approaches in experiments.
Improved UDA framework using f-divergence measures.
problem Addressing distribution shifts in machine learning.
method Refined f-divergence-based discrepancy and f-domain discrepancy. result Novel target error and sample complexity bounds.
Study improves domain adaptation with limited labeled data.
problem Adapting a model to a target domain with few labeled samples.
method Discrepancy-based sample reweighting methods and learning algorithms.
result Improved solutions for domain adaptation problems.
Paper tackles noise-robust domain adaptation in noisy environments.
problem Learning machines struggle with domain adaptation in noisy environments.
method The paper proposes offline curriculum learning and proxy distribution based margin discrepancy to mitigate label and feature noise.
result The proposed algorithm significantly outperforms state-of-the-art methods in noisy environments.
A new method for density estimation using mixture discrepancy and moments.
problem Generalizing histogram statistics to higher dimensions.
method Density estimation via mixture discrepancy and moments (DSP-mix and MSP).
result DSP-mix and MSP are computationally tractable and maintain accuracy with increased speed.
Unified framework for multi-domain learning and data imputation.
problem Improving performance across different domains with missing data.
method Adversarial autoencoder for domain-invariant embeddings and data imputation.
result Superior performance compared to state-of-the-art methods in various settings.
Domain adaptation is transfer learning which aims to generalize a learning model across training and testing data with different distributions. Most previous research tackle this problem in seeking a shared feature representation between source and target domains while reducing the mismatch of their data distributions.…
A new method trains deep neural networks for open set domain adaptation without negative open set difference.
problem Training deep neural networks for open set domain adaptation without negative open set difference.
method Proposes a new upper bound of target-domain risk, including source-domain risk, ε-open set difference (Δε), distributional discrepancy, and constant. Uses gradient descent for source-domain risk and Δε, and adversarial training for distributional discrepancy. Trains DNNs via minimizing the new upper bound. result Shows state-of-the-art performance on benchmark datasets.
Secure SMMD enables data privacy in federated learning.
problem Data privacy in federated learning.
method Homomorphic encryption-based Secure Maximum Mean Discrepancy (SMMD).
result Secure SMMD avoids data leakage and enables effective knowledge transfer.
The recent empirical success of unsupervised cross-domain mapping algorithms, between two domains that share common characteristics, is not well-supported by theoretical justifications. This lacuna is especially troubling, given the clear ambiguity in such mappings. We work with adversarial training methods based on IP…
New bounds for contrastive learning handle domain shifts and generalization.
problem Domain shifts and generalization challenges in downstream tasks.
method Novel generalization bounds accounting for both domain shift and generalization.
result Performance of contrastively learned representations depends on statistical discrepancy between pretraining and downstream distributions.
This paper refines MMD for domain adaptation by balancing intra-class and inter-class distances.
problem Balancing intra-class and inter-class distances for better feature discriminability in domain adaptation.
method The paper theoretically proves two facts about MMD and proposes a novel discriminative MMD method to balance intra-class and inter-class distances.
result The proposed method improves feature discriminability and outperforms state-of-the-art methods.
In unsupervised domain adaptation, it is widely known that the target domain error can be provably reduced by having a shared input representation that makes the source and target domains indistinguishable from each other. Very recently it has been studied that not just matching the marginal input distributions, but th…
This paper establishes a theoretical foundation for super-models via domain adaptation.
problem Reducing computational and data costs in AI for small and medium-sized enterprises.
method Two-stage diffusion process modeling, including pre-training and fine-tuning stages, using the Uhlenbeck-Ornstein process.
result The generalization error of the fine-tuning stage is dominant in domain adaptation.
Paper proposes DAPNA to improve few-shot learning with domain adaptation.
problem Improving recognition of unseen classes with limited samples.
method Domain adaptation prototypical network with attention (DAPNA).
result DAPNA outperforms state-of-the-art FSL alternatives in experiments.
Study introduces KorFinMTEB for Korean financial texts, revealing model limitations.
problem Limited evaluation benchmarks for low-resource domains, especially Korean.
method Developed KorFinMTEB, a tailored benchmark for Korean financial texts.
result Models perform better on translated benchmarks than on domain-specific ones.
Generative adversarial networks (GANs) generate data based on minimizing a divergence between two distributions. The choice of that divergence is therefore critical. We argue that the divergence must take into account the hypothesis set and the loss function used in a subsequent learning task, where the data generated …
In this paper we propose a novel dual adversarial co-learning approach for multi-domain text classification (MDTC). The approach learns shared-private networks for feature extraction and deploys dual adversarial regularizations to align features across different domains and between labeled and unlabeled data simultaneo…
The paper improves generalization bounds for domain adaptation.
problem Improving generalization bounds for domain adaptation under practical conditions.
method Derives generalization bounds for domain adaptation based on finitely many moments and smoothness conditions.
result Obtains generalization bounds for domain adaptation.
Improves survival analysis across multiple domains with machine learning.
problem Adapting survival analysis to new or rare illness types with limited labeled data.
method Introduces a new survival metric and discrepancy measure for censored data, enabling domain adaptation.
result Superb performance on target domains, better treatment recommendations, and interpretable weight matrix.
Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domai…
Sparse coding has shown its power as an effective data representation method. However, up to now, all the sparse coding approaches are limited within the single domain learning problem. In this paper, we extend the sparse coding to cross domain learning problem, which tries to learn from a source domain to a target dom…
Unsupervised domain adaptation studies the problem of utilizing a relevant source domain with abundant labels to build predictive modeling for an unannotated target domain. Recent work observe that the popular adversarial approach of learning domain-invariant features is insufficient to achieve desirable target domain …
New research sets the minimax lower bound for KSD estimation at sqrt(n).
problem Estimating goodness-of-fit using Kernel Stein Discrepancy (KSD) on high-dimensional spaces.
method Two complementary results proving the minimax lower bound of KSD estimation.
result The minimax lower bound of KSD estimation is n^(-1/2), indicating exponential difficulty with dimensionality.
Given iid observations from an unknown absolute continuous distribution defined on some domain Ω, we propose a nonparametric method to learn a piecewise constant function to approximate the underlying probability density function. Our density estimate is a piecewise constant function defined on a binary partition o…
L2M learns to match distributions for domain adaptation without relying on hand-crafted priors.
problem Difficult to directly match cross-domain distributions; existing methods fail to uncover underlying relationships.
method Learning to Match (L2M) framework that learns distribution matching loss using a meta-network in a data-driven way.
result L2M outperforms state-of-the-art methods on public datasets and transfers from pneumonia to COVID-19 images.
The study examines how hyperparameters affect prediction discrepancies in machine learning models.
problem Prediction inconsistencies across different machine learning models trained on the same dataset.
method Investigation of six models (Elastic Net, Decision Tree, k-NN, SVM, RF, XGBoost) on 21 benchmark datasets, focusing on key hyperparameters.
result Hyperparameter tuning improves model performance but increases prediction discrepancies, especially in Extreme Gradient Boosting.