A new method for training GNNs without a teacher model.
problem Training over-parameterized GNN models is difficult and inefficient.
method GNN Self-Distillation (GNN-SD) with NDR and ADR.
result Improves GNN performance with less training cost and better generalization.
This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
A new method improves Bayesian inference for multimodal posteriors.
problem Insensitivity to well-separated modes in multimodal posteriors.
method Weighted Kernel Stein Discrepancy method.
result Significantly improved mode sensitivity compared to standard KSD-Bayes.
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
problem Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
method Reformulating Stein discrepancy construction as an explicit SNR^2 maximisation problem
result Avoiding exponential SNR^2 collapse and achieving stable SNR^2
Unsupervised domain adaptation is the problem setting where data generating distributions in the source and target domains are different, and labels in the target domain are unavailable. One important question in unsupervised domain adaptation is how to measure the difference between the source and target domains. A pr…
Appropriately evaluating the discrepancy between domains is essential for the success of unsupervised domain adaptation. In this paper, we first point out that existing discrepancy measures are less informative when complex models such as deep neural networks are used, in addition to the facts that they can be computat…
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Private algorithms adapt from public to private domains with minimal labeled data.
problem Adapting from a public source domain to a private target domain with few labeled data.
method Differentially private discrepancy minimization algorithms based on Frank-Wolfe and Mirror-Descent methods.
result Effective adaptation with strong generalization and privacy guarantees.
Study on discrepancy principle for learning algorithms in nonparametric regression.
problem Determining optimal iteration number in nonparametric regression with unknown optimal iteration.
method Investigates discrepancy principle and modified principles for kernelized spectral filters, using deviation inequalities and change-of-norm arguments.
result Classical discrepancy principle is adaptive for slow rates, while modified principles are adaptive for faster rates.
Proposes TFDF to learn transferable and discriminative features for unsupervised domain adaptation.
problem Difficult to induce supervised classifier without labeled data in unsupervised domain adaptation.
method TFDF optimizes transferability and discriminability by aligning distributions and minimizing class confusion.
result TFDF achieves better performance on real-world datasets compared to existing methods.
A new model for imputing missing values in time series data across domains.
problem Imputing missing values in time series data across domains with domain shifts and high missing rates.
method A diffusion-based imputation model that integrates shared spectral components and domain-specific temporal structures, with cross-domain consistency alignment.
result Our model effectively handles missing values and domain shifts, outperforming existing methods.
New method for adaptive estimation and inference in econometric models without knowing smoothness.
problem Adaptive estimation and inference in ill-posed linear inverse problems with unknown smoothness.
method Discrepancy principle-based framework for adaptive hyperparameter selection.
result Achieves optimal rates in weak and strong metrics for linear functionals.
Modified dynamical systems retain Turing universality.
problem Embedding Turing machines into dynamical systems.
method Exploring flows with adapted 1-forms and homogeneity.
result Even slight modifications can lead to Turing universality.
We consider the problem of improving the efficiency of randomized Fourier feature maps to accelerate training and testing speed of kernel methods on large datasets. These approximate feature maps arise as Monte Carlo approximations to integral representations of shift-invariant kernel functions (e.g., Gaussian kernel).…
KSD Thinning uses KSD to thin MCMC samples efficiently.
problem Efficiently representing posterior distributions in Bayesian inference.
method KSD Thinning: retains only samples exceeding a KSD threshold.
result Established convergence and complexity tradeoffs for KSD Thinning.
Stein importance sampling is a widely applicable technique based on kernelized Stein discrepancy, which corrects the output of approximate sampling algorithms by reweighting the empirical distribution of the samples. A general analysis of this technique is conducted for the previously unconsidered setting where samples…
Recent unsupervised approaches to domain adaptation primarily focus on minimizing the gap between the source and the target domains through refining the feature generator, in order to learn a better alignment between the two domains. This minimization can be achieved via a domain classifier to detect target-domain feat…
EVI-MMD approximates target distributions via MMD minimization with adaptive kernel.
problem Approximating target distributions using kernel discrepancy methods.
method EVI-MMD uses Maximum Mean Discrepancy (MMD) to minimize kernel discrepancy, solving ODEs with implicit Euler scheme and L-BFGS optimization.
result EVI-MMD with adaptive bandwidth selection significantly improves performance in sampling problems.
Leveraging reference-only samples for two-sample testing under size asymmetry
problem Two-sample testing under size imbalance
method Adaptive aggregation of reference-dependent representations
result Strong performance with type I error control
ConvMMD improves inference in noisy data.
problem Inference degradation due to measurement error in noisy data.
method Convolutional Maximum Mean Discrepancy (convMMD) for inference with noisy, heteroscedastic observations.
result Established consistency and asymptotic normality of the convMMD-based estimator.
Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
Study improves domain adaptation with limited labeled data.
problem Adapting a model to a target domain with few labeled samples.
method Discrepancy-based sample reweighting methods and learning algorithms.
result Improved solutions for domain adaptation problems.
A new gradient flow for MMD with closed-form implementation.
problem Existing gradient flows either lack tractable numerical implementation or require strong assumptions.
method Introduces a (de)-regularized Maximum Mean Discrepancy (DrMMD) and its gradient flow.
result Guarantees near-global convergence for a broad class of targets in both continuous and discrete time.
In this work, we connect two distinct concepts for unsupervised domain adaptation: feature distribution alignment between domains by utilizing the task-specific decision boundary and the Wasserstein metric. Our proposed sliced Wasserstein discrepancy (SWD) is designed to capture the natural notion of dissimilarity betw…
Efficient event generation for collider phenomenology using parallel Langevin sampling and learned Stein diagnostics.
problem Event generation for precision collider phenomenology.
method Parallel Langevin sampling with learned Stein diagnostics.
result Relaxation time is estimated using a data-driven approach.
A new method for density estimation using mixture discrepancy and moments.
problem Generalizing histogram statistics to higher dimensions.
method Density estimation via mixture discrepancy and moments (DSP-mix and MSP).
result DSP-mix and MSP are computationally tractable and maintain accuracy with increased speed.
Paper tackles noise-robust domain adaptation in noisy environments.
problem Learning machines struggle with domain adaptation in noisy environments.
method The paper proposes offline curriculum learning and proxy distribution based margin discrepancy to mitigate label and feature noise.
result The proposed algorithm significantly outperforms state-of-the-art methods in noisy environments.
Introduces Cauchy-Schwarz divergence for domain adaptation.
problem Evaluating discrepancy between source and target domains in unsupervised domain adaptation.
method Introduces Cauchy-Schwarz divergence as a measure for evaluating discrepancy between marginal and conditional distributions.
result CS divergence offers a tighter generalization error bound than Kullback-Leibler divergence.
Improved UDA framework using f-divergence measures.
problem Addressing distribution shifts in machine learning.
method Refined f-divergence-based discrepancy and f-domain discrepancy. result Novel target error and sample complexity bounds.
The paper improves smoothed analysis for online problems with adaptive adversaries.
problem Online prediction, discrepancy minimization, and online optimization with adaptive adversaries.
method General technique to prove smoothed guarantees against adaptive adversaries, reducing to simpler oblivious adversaries.
result Strong smoothed guarantees for three online problems, matching or improving previous results.
Deep learning has recently been shown to be instrumental in the problem of domain adaptation, where the goal is to learn a model on a target domain using a similar --but not identical-- source domain. The rationale for coupling both techniques is the possibility of extracting common concepts across domains. Considering…
DRDA robustly adapts models across domains with mismatched distributions.
problem Vulnerability of DA methods to noise and inability to generalize to unseen samples.
method DRDA uses distributionally robust optimization (DRO) with MMD metric to learn robust decision functions.
result DRDA outperforms existing robust learning approaches in experiments.
Given i.i.d samples from some unknown continuous density on hyper-rectangle [0,1]d, we attempt to learn a piecewise constant function that approximates this underlying density non-parametrically. Our density estimate is defined on a binary split of [0,1]d and built up sequentially according to discrepancy crite…
A new sequential test for unnormalized densities.
problem Testing unnormalized densities with adaptive stopping.
method Sequential kernelized Stein discrepancy test, using non-uniform Stein kernels.
result Valid test with asymptotic lower bound for growth.
A new method improves EEG classification across subjects efficiently.
problem Challenges in adapting and retaining knowledge for EEG classifiers across different subjects.
method Meta UPdate Strategy (MUPS-EEG) for continuous EEG classification.
result Outperforms current state-of-the-art methods in adapting to new subjects and retaining knowledge of learned subjects.
Unified analysis of generalization and sample complexity for semi-supervised domain adaptation.
problem Theoretical foundations of domain adaptation remain underexplored, especially for modern approaches.
method Unified theoretical study of domain adaptation algorithms based on domain alignment, considering joint learning of feature transformations and shared classifiers in a semi-supervised setting.
result Unified theoretical analysis of domain adaptation algorithms, providing generalization bounds and sample complexity bounds for MMD and adversarial models.
Novel MBRL method for large-scale RL with reduced posterior complexity.
problem Theoretical guarantees for MBRL in large spaces with complex models.
method Kernelized Stein Discrepancy for compression of posterior estimate.
result Sublinear Bayesian regret and up to 50% reduction in training time.
A new method compares image classifiers using adaptive sampling of natural images.
problem Evaluation of image classifiers on small, fixed test sets may not generalize to real-world images.
method Adaptive sampling from a large corpus of unlabeled images to maximize classifier discrepancies measured by WordNet hierarchy.
result Human labeling of model-dependent image sets reveals relative classifier performance.
Generative adversarial networks (GANs) generate data based on minimizing a divergence between two distributions. The choice of that divergence is therefore critical. We argue that the divergence must take into account the hypothesis set and the loss function used in a subsequent learning task, where the data generated …
We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible. Our model is based on a variational autoencoding architecture with priors that encourage independence between sens…
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
Study on Wasserstein gradient flow for MMD between Coulomb measures.
problem Analyzing the long-time behavior of MMD between probability and target measures using Coulomb kernels.
method Existence of global weak solutions, ultracontractive estimate, regularity analysis, exponential decay proof, defective Polyak-Lojasiewicz inequality.
result Exponential decay of squared MMD toward a uniformly positive target measure on flat torus.
This paper refines MMD for domain adaptation by balancing intra-class and inter-class distances.
problem Balancing intra-class and inter-class distances for better feature discriminability in domain adaptation.
method The paper theoretically proves two facts about MMD and proposes a novel discriminative MMD method to balance intra-class and inter-class distances.
result The proposed method improves feature discriminability and outperforms state-of-the-art methods.
The paper improves generalization bounds for domain adaptation.
problem Improving generalization bounds for domain adaptation under practical conditions.
method Derives generalization bounds for domain adaptation based on finitely many moments and smoothness conditions.
result Obtains generalization bounds for domain adaptation.
TaylorPODA uses Taylor expansions to improve feature attributions for opaque models.
problem Lack of systematic framework for quantifying feature contributions in opaque models.
method Taylor expansion framework with postulates (precision, federation, zero-discrepancy, adaptation).
result TaylorPODA achieves competitive results and provides principled explanations.