In this report, we review the calculation of entropy-regularised Wasserstein loss introduced by Cuturi and document a practical implementation in PyTorch. Code is available at https://github.com/t-vi/pytorch-tvmisc/blob/master/wasserstein-distance/Pytorch_Wasserstein.ipynb
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Wasserstein loss”
Wasserstein GANs fail to approximate Wasserstein distance, leading to their success.
problem Approximating Wasserstein distance in deep generative models.
method Analysis of differences between theoretical setup and training reality.
result Wasserstein GANs' success is due to their failure to approximate Wasserstein distance.
Paper develops a generative model using Wasserstein-2 loss.
problem Creating realistic data samples from limited data.
method Uses a distribution-dependent ODE with a gradient flow for W2 loss.
result The method converges to the true data distribution exponentially.
Paper provides finite-sample guarantees for Wasserstein DRO without dimensionality curse.
problem Tackles empirical success of Wasserstein DRO in operations and ML with performance guarantees.
method Develops non-asymptotic framework for analyzing out-of-sample performance and generalization bound.
result First finite-sample guarantee for generic Wasserstein DRO problems without curse of dimensionality.
Learning to predict multi-label outputs is challenging, but in many problems there is a natural metric on the outputs that can be used to improve predictions. In this paper we develop a loss function for multi-label learning, based on the Wasserstein distance. The Wasserstein distance provides a natural notion of dissi…
Study on deep matrix factorization with Bures-Wasserstein loss, focusing on critical points and convergence.
problem Analyzing critical points and convergence of generative deep linear networks trained with Bures-Wasserstein loss.
method Characterization of critical points and minimizers of Bures-Wasserstein distance, analysis of Hessian at low-rank matrices, convergence results for gradient flow and descent.
result Established convergence results for gradient flow and finite step size gradient descent under certain assumptions.
Paper proposes SinkhornDRL for distributional RL using Sinkhorn divergence and regularized Wasserstein loss.
problem Improving distributional reinforcement learning by minimizing Bellman return distribution differences.
method Introduces SinkhornDRL, a distributional RL algorithm using Sinkhorn divergence and regularized Wasserstein loss.
result SinkhornDRL consistently outperforms or matches existing algorithms on Atari games, especially in multi-dimensional reward settings.
Unified framework for non-negative matrices and tensors using Wasserstein loss.
problem Finding low-dimensional representations of high-dimensional datasets with non-negative constraints.
method Unified mathematical framework with a smoothed Wasserstein loss, convex dual formulation for efficient computation.
result Efficient solution for non-negative matrix and tensor factorisations with Wasserstein loss.
Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.
problem Estimating multimodal aleatoric uncertainty in regression tasks from images.
method Regression-by-classification paradigm with hinge-Wasserstein loss.
result Hinge-Wasserstein loss improves uncertainty estimation on challenging tasks.
WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.
problem Inherent instability in actor-critic reinforcement learning algorithms.
method Wasserstein adaptive value estimation with Sinkhorn approximation.
result Achieves $\mathcal{O}\left(\frac{1}{k}
ight)$ convergence rate for critic's mean squared error.
A new supervised tree-Wasserstein distance improves document classification.
problem Measuring document similarity efficiently and accurately.
method Rewriting Wasserstein distance on tree metric, using contrastive loss for optimization.
result The Supervised Tree-Wasserstein (STW) distance improves document classification accuracy.
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…
Learning a generative model is a key component of model-based reinforcement learning. Though learning a good model in the tabular setting is a simple task, learning a useful model in the approximate setting is challenging. In this context, an important question is the loss function used for model learning as varying th…
A new method optimizes projection directions for sliced Wasserstein distances.
problem Finding informative projecting directions for sliced Wasserstein distances is computationally expensive.
method Amortized projection optimization to predict directions efficiently.
result Proposed amortized models improve generative modeling performance.
The paper proposes a method to create domain-invariant representations using Wasserstein distance.
problem Domain shifts in training data affect machine learning model performance across different domains.
method The method combines classification/regression losses with a GAN-type discriminator to minimize the Wasserstein distance between domains.
result The approach produces the highest minimum classification accuracy and most invariant representation across domains.
The paper introduces a new Wasserstein distance for approximating posteriors in inverse problems.
problem Approximating posterior measures in inverse problems using conditional Wasserstein distances.
method Introduces a conditional Wasserstein distance with restricted couplings and derives its dual.
result Shows that conditional Wasserstein GANs can yield favorable properties for posterior sampling.
Wasserstein distributionally robust optimization (DRO) has recently achieved empirical success for various applications in operations research and machine learning, owing partly to its regularization effect. Although connection between Wasserstein DRO and regularization has been established in several settings, existin…
The Wasserstein metric is an important measure of distance between probability distributions, with applications in machine learning, statistics, probability theory, and data analysis. This paper provides upper and lower bounds on statistical minimax rates for the problem of estimating a probability distribution under W…
Study improves adversarial classification using distributionally robust models.
problem Improving robustness against adversarial attacks in classification models.
method Distributionally robust chance constraints with Wasserstein ambiguity, reformulated as a regularized ramp loss minimization problem.
result Standard descent methods can converge to the global minimizer for the distributionally robust adversarial classification model.
Optimal transport is a foundational problem in optimization, that allows to compare probability distributions while taking into account geometric aspects. Its optimal objective value, the Wasserstein distance, provides an important loss between distributions that has been used in many applications throughout machine le…
New PAC-Bayes bounds use Wasserstein distances to improve generalization.
problem Lack of geometric properties in existing PAC-Bayes bounds.
method Developed new PAC-Bayes bounds with Wasserstein distances.
result Optimization guarantees translate to good generalization abilities.
Proposes a fair classification model using Wasserstein ambiguity sets.
problem Ensuring fairness in classification models.
method Distributionally robust optimization with Wasserstein ambiguity sets and equal opportunity fairness constraint.
result Improves fairness without significant loss in predictive accuracy.
Proposes new loss functions for better handling bimodal predictive uncertainty.
problem Bimodal predictive uncertainty in machine learning models.
method Family of distribution-aware loss functions integrating normalized RMSE with Wasserstein and Cramér distances.
result Proposed loss functions reduce predictive uncertainty estimation error by 45% on complex bimodal datasets.
A new method for conditional sampling using paired Wasserstein Autoencoders.
problem Conditional sampling from complex data distributions.
method Derive a novel loss function for Wasserstein Autoencoders to enable sampling from OT-type couplings.
result Learned cost-optimal transport maps and conditional sampling from an OT-type coupling.
New method uses Wasserstein loss for data unfolding, offering better accuracy than classical techniques.
problem Removing noise or artifacts from measurements in physics experiments.
method Alternative formulation using Wasserstein loss, developing a convergent algorithm.
result Optimal transport approach offers robust, accurate performance compared to classical techniques, especially in cases with significant binning artifacts.
Paper introduces a new robust loss function for RL.
problem Heuristic selection of threshold parameters in quantile Huber loss.
method Derived from Wasserstein distance, captures noise in quantile values.
result Enhances robustness against outliers and enables parameter adjustment.
A faster Wasserstein k-means algorithm for histogram data reduces computation and maintains clustering quality.
problem Efficiently clustering histogram data with reduced computation time.
method Sparse simplex projection to reduce data samples, centroids, and ground cost matrix, dynamically removing lower-valued samples.
result Significant reduction in computational complexity without compromising clustering quality.
In this paper we study generative modeling via autoencoders while using the elegant geometric properties of the optimal transport (OT) problem and the Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE), which are generative models that enable one to shape the distribution of the latent space int…
Wasserstein distributionally robust optimization estimators are obtained as solutions of min-max problems in which the statistician selects a parameter minimizing the worst-case loss among all probability models within a certain distance (in a Wasserstein sense) from the underlying empirical measure. While motivated by…
Study compares geometric approaches for shape and deformation statistics.
problem Characterizing statistical models of shapes and deformations.
method Information geometry and Wasserstein geometry.
result Wasserstein estimator is robust against waveform perturbation.
Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Functionstat.ML
We develop a new loss function for estimating quasiprobabilistic density ratios.
problem Discontinuous or non-surjective relationships between optimal classifiers and target densities.
method Introduce a convex loss function compatible with both probabilistic and quasiprobabilistic densities.
result Achieve state-of-the-art results in estimating di-Higgs production in particle physics.
Generative Adversarial Networks (GANs) have been used to model the underlying probability distribution of sample based datasets. GANs are notoriuos for training difficulties and their dependence on arbitrary hyperparameters. One recent improvement in GAN literature is to use the Wasserstein distance as loss function le…
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
The paper studies properties of Sliced Wasserstein energy for discrete measures.
problem Optimizing discrete probability measures using Sliced Wasserstein loss.
method Investigates the regularity and optimisation properties of the Sliced Wasserstein energy and its Monte-Carlo approximation.
result Convergence results on the critical points of Monte-Carlo approximations to the Sliced Wasserstein energy.
Generative models learn complex data from low-dimensional manifolds.
problem Theoretical justification for generative models on manifold structures.
method Prove statistical guarantees of generative networks under Wasserstein-1 loss, considering intrinsic dimensionality.
result Generative networks converge to zero at a fast rate depending on intrinsic dimensionality, not ambient data dimension.
MonoFlow rethinks GANs using Wasserstein gradient flows.
problem Inconsistencies between GAN theory and practice.
method Unified generative modeling framework based on Wasserstein gradient flows.
result Adversarial training can be seen as particle flow optimization.
Algorithm aligns 3D density maps using Wasserstein distance.
problem Aligning 3D density maps in cryogenic electron microscopy.
method Minimizing 1-Wasserstein distance after rigid transformation using Bayesian optimization.
result Improved accuracy and efficiency in protein molecule alignment.
New DG method minimizes barycentric alignment and reconstruction loss.
problem Improving domain generalization in machine learning.
method Introduces a new upper bound and WBAE algorithm.
result WBAE outperforms state-of-the-art DG algorithms.
A general duality proof for Wasserstein distributionally robust optimization.
problem Optimizing under uncertainty with Wasserstein distance.
method One-dimensional convex analysis and interchangeability principle.
result General duality result holds for various distributions and costs.
Hierarchical Federated Learning bounds generalize using Wasserstein distance.
problem Bounding generalization error in Federated Learning with hierarchical sampling.
method Introduced a hierarchical sampling framework and derived generalization bounds using Wasserstein distance.
result Recover and strictly imply existing CMI bounds for bounded losses.
New method for summarizing Bayesian mixture models using sliced Wasserstein distances.
problem Estimating the mixing measure in nonparametric Bayesian mixture models.
method Decision-theoretic approach using sliced Wasserstein distances for Gaussian mixtures.
result Effective estimation of the mixing measure and mixture density.
Exact generalization guarantees for robust models using Wasserstein distance are established.
problem Capturing data uncertainty and distribution shifts in machine learning models.
method Establishes exact generalization guarantees for robust models based on the Wasserstein distance, covering various cases and transport costs.
result Exact generalization guarantees are provided for a wide range of cases, including deep learning objectives with nonsmooth activations.
Wasserstein gradient boosting predicts probability distributions for supervised learning.
problem Distribution-valued supervised learning where outputs are probability distributions.
method Fits a new weak learner to Wasserstein gradients of loss functionals of probability distributions.
result Superior performance in probabilistic prediction compared to existing methods.
The paper introduces a new ODE approach to improve Wasserstein GANs.
problem Improving Wasserstein GANs for better training results.
method Derives an ODE representing the gradient flow of Wasserstein-1 loss and proposes a new model W1-FE.
result W1-FE outperforms WGAN in training experiments across various dimensions.
This work improves generalisation bounds using chaining and information theory.
problem Improving generalisation bounds for supervised learning algorithms.
method Developed a theoretical framework linking generalisation bounds to their chained counterparts, derived new bounds using Wasserstein distance.
result Chained generalisation bounds can be tighter than standard bounds, especially for concentrated hypothesis distributions.
Unified stability bounds for noisy SGD across convex and non-convex losses.
problem Deriving generalization bounds for noisy stochastic gradient descent.
method Unified approach using Lyapunov functions and applied probability.
result Time-uniform stability bounds for SGD on various loss functions.
Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood. It constructs an approximate posterior distribution by finding parameters for which the simulated data are close to the observations in terms of summary statistics. T…
DAG-WGAN learns causal structures using Wasserstein distance.
problem Learning causal structures from data with combinatorial challenges.
method Combines Wasserstein distance, auto-encoder, and acyclicity constraint.
result Demonstrates good performance compared to state-of-the-art models.