Proposes a non-adversarial method for distribution matching.
problem Stability and optimization challenges in adversarial matching methods.
method Non-adversarial VAE-based matching method with alignment upper bounds.
result Demonstrates applicability of non-adversarial matching methods without modifying original architectures.
Paper tackles distribution matching by partially matching distributions, achieving robust results.
problem Robustly aligning two probability distributions.
method Developed a partial Wasserstein adversarial network (PWAN) to efficiently approximate the partial Wasserstein-1 (PW) discrepancy.
result The PWAN effectively produces highly robust matching results, outperforming state-of-the-art methods.
Study efficient iterative method for distribution matching using sliced optimal transport.
problem Efficiently match distributions using sliced optimal transport.
method Slice-matching scheme based on sliced optimal transport, with quantitative non-asymptotic rates derived.
result Derive quantitative non-asymptotic rates for convergence to target distribution.
We investigate the non-identifiability issues associated with bidirectional adversarial training for joint distribution matching. Within a framework of conditional entropy, we propose both adversarial and non-adversarial approaches to learn desirable matched joint distributions for unsupervised and supervised tasks. We…
Generative models learn distributions, new method finds inputs matching desired conditional distributions.
problem Designing inputs that produce specific conditional distributions, not just points.
method Conditional Distribution Matching (CDM) and MLGD-F algorithm.
result MLGD-F reliably recovers inputs matching diverse user-specified conditional distributions.
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.
LSDM uses unpaired data to match latent space distributions for generative modeling.
problem Generating high-quality images with limited paired data.
method Two-stage approach: latent space learning from paired and unpaired data, followed by joint distribution matching.
result LSDM enhances geometric fidelity in generated outputs and provides theoretical insights into LDMs.
Paper proposes MMI-ALI for scalable joint distribution matching across multiple domains.
problem Scalability issue in matching joint distributions across multiple domains.
method Adversarial training with Multivariate Mutual Information maximization.
result MMI-ALI achieves scalable joint distribution matching across multiple domains.
KD2M unifies feature matching in neural networks.
problem Matching neural network activations for knowledge transfer.
method Formalizes distribution matching strategy.
result New theoretical results for KD.
ABI bypasses likelihood intractability with nonparametric distribution matching.
problem Approximate Bayesian computation's inefficiency in high-dimensional settings and under diffuse priors.
method Adaptive Bayesian Inference (ABI) compares posterior distributions directly using nonparametric distribution matching and MSW distance.
result ABI significantly outperforms other methods in high-dimensional or dependent observation regimes.
SDM Policy accelerates inference for robotic tasks while maintaining high action quality.
problem Prolonged inference times in diffusion-based policies for high-frequency control tasks.
method Two-stage optimization: score matching and distribution matching; dual-teacher mechanism.
result 6x inference speedup with state-of-the-art action quality.
L2M learns to match distributions for domain adaptation without relying on hand-crafted priors.
problem Difficult to directly match cross-domain distributions; existing methods fail to uncover underlying relationships.
method Learning to Match (L2M) framework that learns distribution matching loss using a meta-network in a data-driven way.
result L2M outperforms state-of-the-art methods on public datasets and transfers from pneumonia to COVID-19 images.
A new method for fair classification using characteristic function distance.
problem Fairness in high-stakes decision-making with sensitive groups.
method Proposes a novel approach based on characteristic function distance to ensure minimal sensitive information in learned representations.
result Consistently matches or achieves better fairness and predictive accuracy than existing methods.
AUTOVC converts voices without parallel data, achieving state-of-the-art results.
problem Non-parallel many-to-many voice conversion and zero-shot voice conversion.
method Only an autoencoder with a carefully designed bottleneck is used, training on a self-reconstruction loss.
result AUTOVC achieves state-of-the-art results in many-to-many voice conversion with non-parallel data and performs zero-shot voice conversion.
Proposes PER loss to regularize neural network activations to normal distribution.
problem Improving neural network generalization and training speed.
method Regularizes activations to standard normal distribution via projected error function and Wasserstein distance.
result Minimizes Wasserstein distance between activation distribution and standard normal.
New algorithm learns from expert demonstrations more efficiently.
problem Efficient imitation learning from expert demonstrations.
method Transformed off-policy distribution matching objective.
result Achieves state-of-the-art sample efficiency and performance.
Enhances DIM to match learned representations to a specific distribution.
problem Learning representations conforming to a specific distribution.
method Injecting noise into normalized outputs of the encoder while keeping the InfoMax training objective.
result Learning uniformly and normally distributed representations, as well as representations of other absolutely continuous distributions.
FAST selects coresets more efficiently by matching distributions in the frequency domain.
problem Efficiently selecting representative subsets of large datasets for deep learning.
method FAST uses spectral graph theory and CFD to match distributions, addressing limitations of existing methods.
result FAST significantly outperforms state-of-the-art coreset selection methods in accuracy and energy efficiency.
Develops a framework for clustering and distribution matching with bandit feedback.
problem Clustering and distribution matching problems with limited feedback.
method General framework using K-armed bandit model, Track-and-Stop method, and Frank--Wolfe algorithm. result Average number of arm pulls matches lower bound, with asymptotic convergence to fundamental limit.
Bayesian method matches uncertainty to adapt across domains.
problem Label distribution shift across domains degrades model performance.
method Bayesian neural network quantifies uncertainty; joint feature and label distribution matching.
result Improves model performance on domain adaptation tasks.
New algorithm learns invariant representations for robust neural networks.
problem Learning robust neural network representations that are invariant to certain factors.
method Causal perspective and distribution matching approach.
result Empirically, the algorithm achieves state-of-the-art performance on domain generalization.
The paper provides a framework for weakly supervised disentanglement guarantees.
problem Learning disentangled representations in real-world data.
method Theoretical framework for analyzing disentanglement guarantees with weak supervision.
result Empirical verification of weak supervision methods' predictive power and usefulness.
New method improves language model fine-tuning without forgetting.
problem Fine-tuning language models to match specific distributions without forgetting.
method Combines Distribution Matching and Reinforcement Learning techniques.
result Adding a baseline improves constraint satisfaction, stability, and efficiency.
In this work, we formulate the fixed-length distribution matching as a Bayesian inference problem. Our proposed solution is inspired from the compressed sensing paradigm and the sparse superposition (SS) codes. First, we introduce sparsity in the binary source via position modulation (PM). We then present a simple and …
Unified calibration metrics improve forecast sharpness and accuracy.
problem Improving the sharpness of probabilistic forecasts while maintaining calibration.
method Kernel-based calibration metrics that unify and generalize existing methods for classification and regression.
result Enhanced calibration, sharpness, and decision-making across various tasks.
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's razor" interpretation. Unfortunately, calculating the potentials in the maximum-entro…
New method recovers predictions from unobservable source subpopulation in binary classification.
problem Challenging binary classification with unobservable subpopulation in source domain.
method Distribution matching method to estimate subpopulation proportions, rigorous derivation of prediction models.
result Our method outperforms naive benchmarks in synthetic and real-world datasets.
Paper tackles adapting multiple domains to a target domain using distillation and dictionary learning.
problem Adapting multiple heterogeneous labeled source domains to an unlabeled target domain.
method Combines Multi-Source Domain Adaptation and Dataset Distillation with Dataset Dictionary Learning.
result Achieves state-of-the-art adaptation performance even with minimal labeled data.
Estimates class prior for unlabeled data using kernel embedding.
problem Estimating class prior in PU learning scenario where only positive and full population samples are available.
method Direct estimator based on distribution matching and kernel embedding in Reproducing Kernel Hilbert Space.
result Asymptotic consistency and explicit deviation bound for the estimator.
MobILE learns from expert demonstrations without access to actions, achieving strong performance guarantees.
problem Imitation Learning from Observations alone (ILFO) where actions are not observed.
method MobILE integrates optimism in the face of uncertainty into distribution matching IL framework, trading off exploration and imitation.
result MobILE provides strong performance guarantees for certain MDP dynamics and demonstrates efficacy on OpenAI Gym tasks.
New algorithm guarantees domain generalization with few environments.
problem Performing well on unseen environments with limited training data.
method Iterative feature matching algorithm with theoretical guarantees.
result Guaranteed domain generalization with logarithmic environments.
We consider the problem of training speech recognition systems without using any labeled data, under the assumption that the learner can only access to the input utterances and a phoneme language model estimated from a non-overlapping corpus. We propose a fully unsupervised learning algorithm that alternates between so…
Proposes a method to preserve information in heterogeneous domain adaptation.
problem Preserving information in different feature spaces between domains.
method Joint information preservation method integrating paired and structural information.
result Superior performance compared to state-of-the-art HDA algorithms.
New method uncovers zero entropy in dependent observations after finite samples.
problem Understanding uncertainty reduction in dependent observations.
method Minimum list entropy coupling, greedy algorithm.
result Zero entropy achieved with O(log(1/P_min)) samples for dependent observations.
A new method for learning policies from demonstrations without reinforcement.
problem Learning policies from demonstrations without access to reinforcement signals.
method Energy-based distribution matching (EDM) to learn policy parameters and state marginals.
result EDM yields consistent performance gains over existing algorithms for strictly batch imitation learning.
New algorithm preserves transport maps for better diffusion model training.
problem Training diffusion models with task-specific optimality structures.
method Generalized Schrödinger Bridge Matching (GSBM), inspired by conditional stochastic optimal control.
result GSBM better preserves transport maps, enabling stable convergence and improved scalability.
In real-world classification problems, the class balance in the training dataset does not necessarily reflect that of the test dataset, which can cause significant estimation bias. If the class ratio of the test dataset is known, instance re-weighting or resampling allows systematical bias correction. However, learning…
We introduce a non-parametric method to recover physical probability distributions of asset returns based on their European option prices and some other sparse parametric information. Thus the main problem is similar to the one considered foir instance in the Recovery Theorem by Ross (2015), except that here we conside…
Adaptive moment methods have been remarkably successful in deep learning optimization, particularly in the presence of noisy and/or sparse gradients. We further the advantages of adaptive moment techniques by proposing a family of double adaptive stochastic gradient methods~\textsc{DASGrad}. They leverage the complemen…
Unsupervised domain adaptation methods aim to alleviate performance degradation caused by domain-shift by learning domain-invariant representations. Existing deep domain adaptation methods focus on holistic feature alignment by matching source and target holistic feature distributions, without considering local feature…
Sources of variability in experimentally derived data include measurement error in addition to the physical phenomena of interest. This measurement error is a combination of systematic components, originating from the measuring instrument, and random measurement errors. Several novel biological technologies, such as ma…
A new generative adversarial network is developed for joint distribution matching. Distinct from most existing approaches, that only learn conditional distributions, the proposed model aims to learn a joint distribution of multiple random variables (domains). This is achieved by learning to sample from conditional dist…
Paper introduces MinDiff framework for balancing classifier performance and fairness.
problem Balancing classifier performance and fairness in machine learning models.
method MinDiff framework with kernel-based statistical dependency tests.
result Demonstrates real-world improvements in classifier performance and fairness.
Multi-source transfer learning has been proven effective when within-target labeled data is scarce. Previous work focuses primarily on exploiting domain similarities and assumes that source domains are richly or at least comparably labeled. While this strong assumption is never true in practice, this paper relaxes it a…
Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. Algorithms for the two tasks, such as normalizing flows and generative adversarial networks (GANs), are often developed independently. In this pape…
With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose the limitations of Variational Autoencoders (VAEs), which consistently fail to learn marginal distributions in both latent and visible spaces…
Proposes a method to train deep models with one-element batches.
problem Training deep models with small batches (one element) is challenging.
method Splits the batch into historical and current elements for training.
result Allows training on higher resolution images with one-element batches.
MIRA scores assess conditional distribution accuracy using joint samples.
problem Assessing the accuracy of candidate conditional distributions.
method Analytic expression for Mira score based on equal probability mass regions.
result Mira enables Bayesian model comparison by quantifying alignment with true process.