Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.
problem Optimal rate of convergence and cost of adaptation in distributed nonparametric function estimation.
method Distributed minimax estimation and adaptive estimation under communication constraints for Gaussian sequence model and white noise model.
result Established minimax rate of convergence and exact communication cost for adaptation.
Generative models learn better with data-adaptive noise.
problem Learning heavy-tailed distributions in flow-based models.
method Data-adaptive latent noise using 1D quantile functions optimized via Wasserstein distance.
result Flexibility and effectiveness in learning heavy-tailed and compactly supported distributions.
The paper improves generalization bounds for domain adaptation.
problem Improving generalization bounds for domain adaptation under practical conditions.
method Derives generalization bounds for domain adaptation based on finitely many moments and smoothness conditions.
result Obtains generalization bounds for domain adaptation.
A quantum walk-based method for generating precise probability distributions efficiently.
problem Generating high-precision probability distributions for various applications.
method Integrates variational quantum circuits with split-step quantum walks to dynamically tune coin parameters and evolve quantum states.
result Achieves high simulation fidelity and reduces computational overhead compared to conventional methods.
The paper proposes a learning algorithm that improves adaptability and generalization.
problem Improving adaptability and generalization in learning models.
method Learning to meta-learn by meta-finetuning on related tasks before adapting to specific tasks.
result Learning to meta-learn improves adaptability and generalization across various tasks.
This thesis studies domain adaptation under minimal distribution similarity assumptions using moments.
problem Learning from samples with distributions different from training samples.
method Uses minimal similarity assumptions modeled by moments.
result Establishes learning bounds and algorithms for domain adaptation.
Paper tackles test-time adaptation for tabular data.
problem Performance degradation due to distribution shifts in testing.
method Proposes FTAT for robustly adapting tabular models during testing.
result FTAT outperforms state-of-the-art methods on benchmark datasets.
Algorithm adapts pretrained semantic segmentation models to new domains.
problem Adapting pretrained models to new, unlabeled domains without source data.
method Learn prototypical distribution in embedding space, align target domain with source domain.
result Method achieves competitive performance on benchmark tasks.
Modular neural causal models outperform other models in generalization and adaptation.
problem Robust out-of-distribution generalization and fast adaptation in machine learning.
method Factorizing data generating process into modules using only causal parents as predictors.
result Modular neural causal models offer robust generalization and fast adaptation, especially in low data regimes.
Adaptive method for prediction sets under changing data distributions.
problem Forming prediction sets in an online setting with varying data distributions.
method Adaptive conformal inference that re-estimates the distribution shift parameter over time.
result Adaptive method achieves desired coverage frequency over long-time intervals.
Proposes a new approach for domain adaptation using latent representations.
problem Handling distribution shifts between source and target domains in high-dimensional data.
method Learn compact latent representations based on the label's Markov blanket, partitioning into parents, children, and spouses.
result General domain adaptation can be achieved by learning representations of the label's parents, children, and spouses.
GRAM enhances deep RL for reliable real-world deployment.
problem Generalizing deep RL across in-distribution and out-of-distribution scenarios.
method Introduces a robust adaptation module and a joint training pipeline.
result GRAM achieves strong generalization performance in simulations and hardware.
DRDA robustly adapts models across domains with mismatched distributions.
problem Vulnerability of DA methods to noise and inability to generalize to unseen samples.
method DRDA uses distributionally robust optimization (DRO) with MMD metric to learn robust decision functions.
result DRDA outperforms existing robust learning approaches in experiments.
New method learns models to adapt to domain shifts at test time.
problem Learning models robust to distribution shifts in practical applications.
method Adaptive Risk Minimization (ARM) framework.
result Performance gains of 1-4% on image classification problems.
UAED discovers adaptive environments for robust learning.
problem Avoiding spurious correlations in data.
method Unified framework that learns a distribution over data transformations.
result Improves worst-case accuracy on standard benchmarks.
AdapTable adapts tabular models to shifts without source data, improving HELOC performance.
problem Distribution shifts in tabular data threaten model performance.
method Shift-aware uncertainty calibrator and label distribution handler.
result Up to 16% improvement on HELOC dataset.
Domain adaptation framework identifies latent variables for target distribution identifiability.
problem Unsupervised domain adaptation without identifiable joint distribution of features and labels.
method Formulated latent variable model with invariant and changing components, constrained domain shift to influence only changing components.
result Joint distribution of data and labels in target domain is identifiable under mild conditions.
The paper improves smoothed analysis for online problems with adaptive adversaries.
problem Online prediction, discrepancy minimization, and online optimization with adaptive adversaries.
method General technique to prove smoothed guarantees against adaptive adversaries, reducing to simpler oblivious adversaries.
result Strong smoothed guarantees for three online problems, matching or improving previous results.
Adaptive batch sizes improve local gradient methods in distributed training.
problem Communication bottlenecks in distributed deep learning.
method Adaptive batch size strategies for local gradient methods.
result Adaptive batch sizes reduce minibatch gradient variance and improve training efficiency.
FLAP adapts policies quickly to new tasks using shared linear representations.
problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.
Introduces Cauchy-Schwarz divergence for domain adaptation.
problem Evaluating discrepancy between source and target domains in unsupervised domain adaptation.
method Introduces Cauchy-Schwarz divergence as a measure for evaluating discrepancy between marginal and conditional distributions.
result CS divergence offers a tighter generalization error bound than Kullback-Leibler divergence.
AIS algorithm improves heavy-tailed distribution estimation.
problem Inconsistent estimators and slow convergence in AIS for heavy-tailed distributions.
method Adapts Student-t proposal distributions by matching escort moments and minimizing α-divergence.
result Improves estimation accuracy for heavy-tailed distributions.
Unsupervised domain adaptation is a promising way to generalize deep models to novel domains. However, the current literature assumes that the label distribution is domain-invariant and only aligns the feature distributions or vice versa. In this work, we explore the more realistic task of Class-imbalanced Domain Adapt…
New method transfers causal mechanisms for few-shot domain adaptation.
problem Few labeled target domain data for regression problems.
method Mechanism transfer using structural equations in causal modeling.
result Method can adapt from apparently different distributions.
Deep neural networks, trained with large amount of labeled data, can fail to generalize well when tested with examples from a \emph{target domain} whose distribution differs from the training data distribution, referred as the \emph{source domain}. It can be expensive or even infeasible to obtain required amount of lab…
AdaCat improves density estimation and planning in autoregressive models.
problem Efficiently modeling sharp density changes in continuous data.
method Adaptive Categorical Discretization (AdaCat) for autoregressive models.
result Improves density estimation and planning in various data types.
M-L2O adapts fast to new tasks by self-adapting during test-time.
problem L2O optimizers struggle with out-of-distribution tasks.
method Meta-training an L2O optimizer to adapt quickly to new tasks.
result M-L2O converges significantly faster than vanilla L2O with only 5 steps of adaptation.
DDG-DA predicts future data distribution to adapt models for predictable concept drift.
problem Adapting models to streaming data with predictable concept drift.
method Train a predictor to forecast future data distribution, generate training samples, and train models on them.
result Significant improvement on multiple models in real-world tasks.
This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
Paper tackles online adaptation to changing label distributions.
problem Adapting machine learning models to changing label distributions in real-world settings.
method Leverages novel analysis to show estimation of expected test loss is possible without true labels. Proposes adaptation algorithms inspired by classical online learning techniques.
result Empirically verified that OGD is particularly effective and robust to various label shift scenarios.
Adaptive optimal transport priors improve few-shot learning robustness.
problem Limited supervision and distribution shifts in few-shot learning.
method Prototype-Guided Distributionally Robust Optimization (PG-DRO) framework.
result PG-DRO achieves stronger robust generalization in few-shot scenarios.
L2M learns to match distributions for domain adaptation without relying on hand-crafted priors.
problem Difficult to directly match cross-domain distributions; existing methods fail to uncover underlying relationships.
method Learning to Match (L2M) framework that learns distribution matching loss using a meta-network in a data-driven way.
result L2M outperforms state-of-the-art methods on public datasets and transfers from pneumonia to COVID-19 images.
OSAMD adapts online to changing distributions with limited labels.
problem Models struggle with continual distribution shifts and expensive labeling in changing environments.
method Online Active Continual Adaptation with OSAMD, an online teacher-student structure and margin-based criterion.
result OSAMD achieves favorable dynamic regret bounds under changing environments with limited labels.
Improves multi-objective learning by adapting to local subintervals.
problem Learning a predictor satisfying multiple objectives in an online, changing data setting.
method Adapting an existing multi-objective learning method with an adaptive online algorithm.
result Improves predictions over subgroups and remains robust under distribution shift.
Gradient-based meta-RL fails with incorrect task distributions, leading to instability and poor performance.
problem Gradient-based meta-RL's sensitivity to task distributions causes instability and poor performance.
method Proposes meta Active Domain Randomization (meta-ADR) to learn task distributions for gradient-based meta-RL.
result Meta-ADR improves stability and generalization of MAML on simulated locomotion and navigation tasks.
New approach improves domain adaptation with label shift assumptions.
problem Improving domain adaptation when label distributions differ between source and target domains.
method Proposes generalized label shift (GLS) and modifies three DA algorithms (JAN, DANN, CDAN) to handle label distribution mismatches. result Modified DA algorithms outperform base versions, especially with large label distribution mismatches.
New framework improves model reliability under distribution shifts.
problem Lack of formal guarantees connecting shift magnitude to prediction reliability in TTA methods.
method Develops a PAC-Bayesian framework interpreting MMD-balls as credal sets.
result Establishes generalization bounds and provides epistemic uncertainty quantification.
We present a domain adaptation based generative framework for zero-shot learning. Our framework addresses the problem of domain shift between the seen and unseen class distributions in zero-shot learning and minimizes the shift by developing a generative model trained via adversarial domain adaptation. Our approach is …
Domain adaptation addresses the common problem when the target distribution generating our test data drifts from the source (training) distribution. While absent assumptions, domain adaptation is impossible, strict conditions, e.g. covariate or label shift, enable principled algorithms. Recently-proposed domain-adversa…
New method improves domain adaptation by aligning sub-domains.
problem Real-world domain adaptation with shifted class distributions.
method Rebalanced sub-domain alignment and novel generalization bound.
result Improves performance in shifted class distribution scenarios.
We provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different, but related, target distribution. Firstly, we propose an improvement of the previous approach we proposed in Germain et al. (2013), …
Optimal transport aligns source and target distributions for domain adaptation.
problem Unsupervised domain adaptation with joint class-conditional and label shifts.
method Minimizes importance weighted loss and Wasserstein distance for aligned marginals and class-conditional distributions.
result Our method outperforms competitors on various domain adaptation tasks.
Improves regression models' performance on covariate shift.
problem Out-of-distribution generalization for regression.
method Spectrally adapting the weights of a pre-trained neural regression model.
result Spectral adaptation improves out-of-distribution performance.
Distributed sensors compress and send features to a fusion center for linear regression.
problem Efficiently compress and transmit features from distributed sensors to a fusion center under varying communication constraints.
method Designs a distributed and adaptive feature compression scheme using optimal quantizers and simple adaptive strategies.
result Demonstrates improved inference performance through simulated experiments.
Adaptive importance sampling is a class of techniques for finding good proposal distributions for importance sampling. Often the proposal distributions are standard probability distributions whose parameters are adapted based on the mismatch between the current proposal and a target distribution. In this work, we prese…
Adapts exploration for better learning progress in distributed agents.
problem Optimizing exploration in distributed reinforcement learning environments.
method Uses a non-stationary multi-armed bandit to dynamically adapt data generation parameters.
result Unified approach produces comparable results to per-task tuning at lower cost.
An essential problem in domain adaptation is to understand and make use of distribution changes across domains. For this purpose, we first propose a flexible Generative Domain Adaptation Network (G-DAN) with specific latent variables to capture changes in the generating process of features across domains. By explicitly…
In this paper, we provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different target distribution. On the one hand, we propose an improvement of the previous approach proposed by Germain et al. (2…