A new method aligns source and target distributions by tuning their weights.
problem Domain adaptation on unlabeled target datasets using labeled source datasets.
method Weighted Joint Distribution Optimal Transport (WJDOT) method that finds alignment between source and target distributions and re-weighting of source distributions.
result Achieves state-of-the-art performance on simulated and real-life datasets.
MT-SGD samples from multiple target distributions using gradient descent.
problem Sampling from multiple unnormalized target distributions.
method Proposes MT-SGD, a flow of intermediate distributions to sample from multiple target distributions.
result Asymptotic analysis shows MT-SGD reduces to multiple-gradient descent for multi-objective optimization.
Adaptive sampling method improves efficiency in complex target distributions.
problem Efficiency of importance sampling in complex target distributions, especially multimodal distributions in high-dimensional spaces.
method Proposes an adaptive scheme combining global sampling with delayed weighting to promote efficient exploration of target distributions.
result The proposed algorithm is geometrically convergent under mild assumptions and demonstrates improved efficiency in various numerical experiments.
AIS algorithm improves heavy-tailed distribution estimation.
problem Inconsistent estimators and slow convergence in AIS for heavy-tailed distributions.
method Adapts Student-t proposal distributions by matching escort moments and minimizing α-divergence.
result Improves estimation accuracy for heavy-tailed distributions.
Paper shows robust generative learning with minimal assumptions on target distributions.
problem Learning generative models with minimal assumptions on target distributions.
method Lipschitz-regularized α-divergences with minimal assumptions. result Stable learning across various target distributions with minimal assumptions.
A novel framework synthesizes treatment data across sites using optimal transport.
problem Estimating treatment effects across different sites with varying conditions.
method Distributional causal inference, Optimal Transport for alignment of control group distributions.
result Synthetic treatment group data aligns with true target distribution under general conditions.
AIS uses a suboptimal extended target distribution, which this paper improves using SGM.
problem Improving the efficiency of Annealed Importance Sampling for marginal likelihood estimation.
method Leveraging score-based generative modeling to approximate the optimal extended target distribution.
result Demonstrated novel, differentiable AIS procedures on synthetic and real-world data.
New method accelerates diffusion models for broader target distributions.
problem Current diffusion models have limited acceleration for certain target distributions.
method Developed a novel accelerated stochastic DDPM sampler.
result Achieved accelerated performance for three broad distribution classes.
DAIS minimizes symmetrized KL divergence between initial and target distributions.
problem Optimizing over initial distributions in importance sampling.
method Differentiable annealed importance sampling (DAIS) minimizing symmetrized KL divergence.
result DAIS minimizes symmetrized KL divergence between initial and target distributions.
Hamiltonian Monte Carlo converges to target distributions under mild conditions.
problem Establishing convergence of Hamiltonian Monte Carlo algorithms.
method Analyzing Lq convergence for Hamiltonian Monte Carlo under mild conditions. result Outputs converge to target distributions under specified conditions.
Transductive Adversarial Networks (TAN) is a novel domain-adaptation machine learning framework that is designed for learning a conditional probability distribution on unlabelled input data in a target domain, while also only having access to: (1) easily obtained labelled data from a related source domain, which may ha…
Deep generative networks can simulate from a complex target distribution, by minimizing a loss with respect to samples from that distribution. However, often we do not have direct access to our target distribution - our data may be subject to sample selection bias, or may be from a different but related distribution. W…
Deep networks can approximate high-dimensional distributions from low-dimensional ones.
problem Approximating high-dimensional distributions from low-dimensional ones.
method Proved neural networks can transform low-dimensional distributions to high-dimensional ones with arbitrary closeness measured by Wasserstein distances and maximum mean discrepancy.
result Upper bounds of the approximation error are obtained in terms of the width and depth of neural network.
Proposes PGPS for efficient Bayesian inference.
problem Efficient sampling from complex posterior distributions.
method Path-guided particle-based sampling with Log-weighted Shrinkage.
result PGPS generates samples closer to target distribution.
The paper improves importance sampling and MCMC methods for complex distributions.
problem Improving sampling efficiency for distributions with atoms or heavy tails.
method Develops minimax optimal trial distributions and importance-tempered MCMC.
result Importance-tempered MCMC can be uniformly ergodic for certain distributions.
We study proper losses for discrete generative models without knowing the target distribution.
problem Evaluating generative models in the discrete setting without direct access to the target distribution.
method Define and construct black-box proper losses using statistical estimation theory.
result Black-box proper losses must be of polynomial form and involve more samples than the polynomial degree.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
The paper addresses score-mismatched diffusion models and zero-shot conditional samplers.
problem Theoretical guarantees for score-mismatched diffusion models in zero-shot conditional sampling.
method Theoretical analysis of score-mismatched diffusion models and zero-shot conditional samplers.
result Theoretical performance guarantees with explicit dimensional dependencies for score-mismatched diffusion samplers.
Estimates model performance under distribution shift using domain-invariant predictors.
problem Poor performance of models on test distributions different from training distributions.
method Uses domain-invariant predictors as a proxy for unknown target labels.
result Shows that the complexity of latent representations influences target risk.
The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.
problem Sampling from complex target distributions efficiently.
method A neural network architecture inspired by Langevin Monte Carlo is proposed to map samples from a simple reference distribution to samples from the target.
result The proposed neural network architecture achieves approximation rates in the Wasserstein-2 distance for smooth, log-concave target distributions.
SIXO improves inference by learning smoothing distributions from all observations.
problem Inference limitations due to ignoring future observations in filtering distributions.
method Density ratio estimation to warp filtering distributions into smoothing distributions, then use SMC with learned targets.
result Proves tighter log marginal lower bounds and more accurate inferences and estimates.
DRDA robustly adapts models across domains with mismatched distributions.
problem Vulnerability of DA methods to noise and inability to generalize to unseen samples.
method DRDA uses distributionally robust optimization (DRO) with MMD metric to learn robust decision functions.
result DRDA outperforms existing robust learning approaches in experiments.
Under covariate shift, training (source) data and testing (target) data differ in input space distribution, but share the same conditional label distribution. This poses a challenging machine learning task. Robust Bias-Aware (RBA) prediction provides the conditional label distribution that is robust to the worstcase lo…
ATC predicts target domain accuracy using only labeled and unlabeled data.
problem Predicting out-of-distribution performance with limited labeled data.
method Average Thresholded Confidence (ATC) method that learns a threshold on model confidence.
result ATC outperforms previous methods across various types of distribution shifts and datasets.
Piecewise normalizing flows improve multi-modal distribution modeling.
problem Improving accuracy in modeling multi-modal distributions.
method Divide target distribution into clusters, train flows to match standard normal base.
result Piecewise flows outperform standard approaches in accuracy.
DiffATD efficiently discovers targets in partially observable environments using diffusion dynamics.
problem Efficiently discovering targets in partially observable environments with limited sampling.
method DiffATD uses diffusion dynamics to maintain a belief distribution over unobserved states, balancing exploration and exploitation.
result DiffATD outperforms baselines and supervised methods in diverse domains.
EnMDAP aligns conditional distributions for multi-source domain adaptation using pseudolabels.
problem Training a target model with no labeled data in the absence of target data labels.
method EnMDAP uses label-wise moment matching and ensemble learning with multiple feature extractors.
result EnMDAP achieves state-of-the-art performance in multi-source domain adaptation tasks.
Optimal weights improve particle-based approximations of discrete distributions.
problem Improving particle-based approximations of discrete distributions.
method Proving optimality of weights and showing how to compute them efficiently.
result Optimal weights can be computed from existing particle-based methods without extra costs.
We observe that several existing policy gradient methods (such as vanilla policy gradient, PPO, A2C) may suffer from overly large gradients when the current policy is close to deterministic (even in some very simple environments), leading to an unstable training process. To address this issue, we propose a new method, …
ReTaSA tackles continuous target shift in regression problems.
problem Continuous target shift in regression settings.
method Nonparametric regularized approach to estimate importance weight function.
result The method provides theoretical justification for the estimated importance weight function.
In this paper, a new approach for classification of target task using limited labeled target data as well as enormous unlabeled source data is proposed which is called self-taught learning. The target and source data can be drawn from different distributions. In the previous approaches, covariate shift assumption is co…
Deep neural networks, trained with large amount of labeled data, can fail to generalize well when tested with examples from a \emph{target domain} whose distribution differs from the training data distribution, referred as the \emph{source domain}. It can be expensive or even infeasible to obtain required amount of lab…
New framework optimizes label shift adaptation using aligned distribution mixture.
problem Label shift where source and target label distributions differ.
method Aligned Distribution Mixture (ADM) framework, incorporating insights from generalization theory.
result The ADM framework improves four typical label shift methods and introduces a one-step approach.
We consider the problem of search through comparisons, where a user is presented with two candidate objects and reveals which is closer to her intended target. We study adaptive strategies for finding the target, that require knowledge of rank relationships but not actual distances between objects. We propose a new str…
Paper proposes a new loss function for conditional models using soft targets.
problem Improving generalization performance of deep neural networks on supervised classification tasks.
method Introduces a new loss function compatible with soft targets, based on noise contrastive estimation.
result Soft target InfoNCE loss performs on par with cross-entropy baselines and outperforms other losses.
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
Unified approach for sampling non-differentiable and heavy-tailed targets.
problem Sampling non-differentiable and heavy-tailed distributions using Langevin algorithms.
method Anchored Langevin dynamics, which modifies the Langevin diffusion with a smooth reference potential and multiplicative scaling.
result Non-asymptotic guarantees in the 2-Wasserstein distance to the target distribution.
Researchers establish bounds for SGMs' KL and Wasserstein divergences under various noise schedules.
problem Estimating the error between target and estimated distributions in SGMs.
method Established upper bounds for KL divergence and Wasserstein distance, incorporating target distribution properties and SGM hyperparameters.
result Optimal noise schedules identified for SGMs, improving generative quality.
This paper provides a simple procedure to fit generative networks to target distributions, with the goal of a small Wasserstein distance (or other optimal transport costs). The approach is based on two principles: (a) if the source randomness of the network is a continuous distribution (the "semi-discrete" setting), th…
This paper improves sampling from complex distributions using Langevin dynamics.
problem Pathological behaviors in normalizing flows for complex distributions.
method A Metropolis adjusted Langevin algorithm (MALA) to sample in the latent space.
result The method preserves tractability of the likelihood and works with any pre-trained NF network.
FAB combines flows with AIS to approximate complex distributions.
problem Challenges in flow-based methods, especially on complex targets.
method Combines flows with AIS, using α-divergence for training.
result FAB produces accurate approximations to complex distributions.
Deep neural networks can generate any 2D distribution with high accuracy.
problem Generating accurate high-dimensional distributions from random noise.
method A deep neural network with a space-filling property of sawtooth functions.
result The network can approximate any 2D Lipschitz-continuous distribution arbitrarily closely.
Domain randomization (DR) is a successful technique for learning robust policies for robot systems, when the dynamics of the target robot system are unknown. The success of policies trained with domain randomization however, is highly dependent on the correct selection of the randomization distribution. The majority of…
An important goal common to domain adaptation and causal inference is to make accurate predictions when the distributions for the source (or training) domain(s) and target (or test) domain(s) differ. In many cases, these different distributions can be modeled as different contexts of a single underlying system, in whic…
Corrects distribution shift in target shift scenarios using importance weighting.
problem Analyzes importance weighting for correcting distribution shift under target shift.
method Analyzed importance-weighted kernel ridge regression under target shift.
result Shows that importance weighting corrects the train-test mismatch without altering input-space complexity.
A new method called TemperFlow tackles multimodality in sampling from unnormalized distributions.
problem Sampling from unnormalized distributions with isolated modes.
method TemperFlow learns a sequence of tempered distributions to progressively approach the target distribution.
result TemperFlow overcomes the limitations of existing methods and achieves superior performance.
A Kernel Adaptive Metropolis-Hastings algorithm is introduced, for the purpose of sampling from a target distribution with strongly nonlinear support. The algorithm embeds the trajectory of the Markov chain into a reproducing kernel Hilbert space (RKHS), such that the feature space covariance of the samples informs the…
Generative multitask learning mitigates confounders causing targets.
problem Unobserved confounders causing targets but not inputs.
method Generative multitask learning (GMTL) modifies inference objective to remove joint target influence.
result Improved robustness to target shift across multitask learning methods.