New findings on optimal transport gradient for generative models, addressing numerical instabilities.
problem Numerical instabilities in training Wasserstein Generative Adversarial Networks (WGAN).
method Valid differentiation theorem for entropic regularized transport, semi-discrete gradient formulation, and optimization algorithm.
result Existence of optimal transport gradient for generative models under specified conditions.
New metric for probability measures connects physics and geometry.
problem Developing a new metric for probability measures.
method Transport Hessian metric, formulated dynamical systems.
result Connections to physics equations and mathematical models.
Derives inequality for optimal transport on manifolds.
problem Optimal transport theory on manifolds.
method Five gradients inequality for cost functions on Lie groups and Riemannian manifolds.
result Derives inequality for optimal transport on specific manifolds.
Paper relaxes optimal transport using convex functions for data science.
problem Optimal transport problem on finite spaces.
method Relaxation via strictly convex functions (Kullback-Leibler divergence, Bregman divergences). Gradient descent iterative process.
result Mathematical foundations and iterative process for the relaxed optimal transport problem.
New gradient flows for non-negative and probability measures combining optimal transport and interaction forces.
problem Optimizing non-negative and probability measures using interaction forces and optimal transport.
method Interaction-Force Transport (IFT) gradient flows and their spherical variant, developed via infimal convolution of Wasserstein and spherical MMD tensors, with a particle-based optimization algorithm.
result The spherical IFT gradient flow provides global exponential convergence guarantees for both MMD and KL energy.
Wasserstein GANs with Gradient Penalty compute a different optimal transport problem called congested transport.
problem Training generative models to produce high-quality synthetic data.
method Wasserstein GANs with Gradient Penalty (WGAN-GP) approach to calculate the Wasserstein 1 distance.
result WGAN-GP computes the minimum of the congested transport problem, not the Wasserstein 1 distance.
Novel approach learns optimal transport using convex neural networks.
problem Learning optimal transport between distributions from samples.
method Solving a minimax optimization to learn two convex functions, representing the optimal transport map.
result The approach finds optimal transport mappings that are independent of initialization and can handle discontinuous distributions.
New method uses transport maps for efficient Bayesian inference.
problem Efficiently perform sequential Bayesian inference of static model parameters.
method Estimation of structured transport maps to extract conditional distributions.
result Gradient-based characterization of posterior density for online parameter estimation.
Stein transport improves Bayesian inference with faster convergence and reduced variance.
problem Efficiently approximating posterior distributions in Bayesian inference.
method A novel Bayesian inference method using Stein transport, which pushes particles along a curve of tempered distributions.
result Stein transport reaches posterior approximations faster and more accurately than Stein variational gradient descent (SVGD).
A new method corrects staleness in online optimization by transporting past gradients.
problem Reducing staleness and variance in online optimization methods.
method Implicit gradient transport (IGT) to correct past gradients at the current iterate.
result IGT reduces variance and bias in updates over time and achieves state-of-the-art results.
We observe that gradients computed via the reparameterization trick are in direct correspondence with solutions of the transport equation in the formalism of optimal transport. We use this perspective to compute (approximate) pathwise gradients for probability distributions not directly amenable to the reparameterizati…
Sharp estimates for heat flow on nonconvex domains.
problem Quantitative estimates for heat flow on nonconvex domains.
method Sharp gradient and transport estimates with novel dependence on time.
result Equivalent characterization of lower bound on second fundamental form.
Gradient flow solves optimal mass transport for covariance matrices.
problem Optimal mass transport for covariance matrices.
method Gradient flow on fiber bundle structure.
result Global convergence to polar decomposition.
Proposes m-POT to improve m-OT's misspecified mappings issue.
problem Misspecified mappings in mini-batch optimal transport.
method Partial optimal transport (POT) between mini-batch empirical measures.
result m-POT alleviates incorrect mappings compared to current methods.
Proves hardness of semi-discrete optimal transport and proposes regularization methods.
problem Computing Wasserstein distance between discrete and non-discrete probability measures.
method Proves hardness, introduces distributionally robust dual optimal transport, regularizes primal objective, uses stochastic gradient descent.
result Regularization schemes and improved convergence guarantees for semi-discrete optimal transport problems.
Paper introduces SGA for barycenter optimization in optimal transport.
problem Optimizing Wasserstein barycenter for discrete distributions.
method Sobolev gradient ascent algorithm tailored to Wasserstein geometry.
result SGA achieves convergence rate similar to subgradient descent.
A new algorithm for parallel transport on shape spaces is presented and compared to existing methods.
problem Statistical analysis of shape data, especially in time series and optimization.
method Pole ladder algorithm for parallel transport on Kendall shape spaces, compared to integration methods.
result The pole ladder algorithm is a more efficient method for parallel transport.
Spark implementation for distributed function minimization.
problem Optimization of functions in distributed computing environments.
method Gradient and quasi-Newton methods on Apache Spark.
result Scalable solution for classification and regression problems.
The paper explores rectified flows and their relation to optimal transport.
problem Understanding the connection between rectified flows and optimal transport.
method Investigates invariance properties, explicit constructions, and analysis of rectified flows in various settings.
result Rectified flows, when gradient constrained, do not generally solve optimal transport problems.
A new method improves Bayesian filtering in nonlinear systems.
problem Bayesian filtering in nonlinear dynamical systems with non-Gaussian posteriors.
method Transport maps with block-triangular structure and gradient flows for MMD minimization.
result Accurate approximation of non-Gaussian posteriors without particle collapse.
New algorithm reduces variance in Monte Carlo simulations using deep neural networks and policy gradients.
problem Reducing variance in Monte Carlo simulations for estimating function values.
method Optimal correlation search using deep neural networks and policy gradients.
result Optimal correlation function reduces variance by approximating and calibrating policy.
New scalable methods for unbalanced optimal transport improve efficiency and applicability.
problem Scalable algorithms for unbalanced optimal transport remain underexplored.
method Analysis of semi-dual formulation and adaptive gradient methods.
result SGD methods achieve a convergence rate of O(n/εT) for large-scale applications.
We present a short overview on the strongest variational formulation for gradient flows of geodesically λ-convex functionals in metric spaces, with applications to diffusion equations in Wasserstein spaces of probability measures. These notes are based on a series of lectures given by the second author for the Summer…
1-Lipschitz neural networks produce clearer, more focused Saliency Maps for explainable AI.
problem Noisy and limited Saliency Maps from traditional neural networks.
method Dual loss of optimal transport problem for 1-Lipschitz neural networks.
result Saliency Maps from 1-Lipschitz networks are highly concentrated and less noisy, aligning with human explanations.
A novel approach to computing barycenters on graph-supported probability measures.
problem Computing weighted averages of measures on graphs.
method Dynamic optimal transport formulation on the simplex, gradient descent on the probability simplex.
result Intrinsic gradient descent provides a coherent framework for synthesizing and analyzing measures on graphs.
New Langevin dynamics samples from entropy-regularized optimal transport.
problem Sampling from entropy-regularized optimal transport.
method Introduced analogous diffusion dynamics constrained to Π(μ,ν). result Long-time limit is the unique solution of an entropic optimal transport problem.
Paper introduces ICGNs to model convex gradients.
problem Modeling convex gradients efficiently.
method Integrates Jacobian-vector product in a neural network.
result Single layer ICGN outperforms single layer ICNN in fitting.
New approach to sparse optimal transport for matching tokens with experts.
problem Sparse matching of tokens with experts in neural networks.
method Sparsity-constrained optimal transport with cardinality constraints.
result Solves nonconvex cardinality constraints with gradient methods.
Smoothed top-k operator improves model training efficiency.
problem Discontinuous top-k operation makes models untrainable end-to-end.
method SOFT top-k operator approximates top-k as EOT solution.
result Improved performance in k-nearest neighbors and beam search.
Efficiently computes optimal transport maps and Wasserstein barycenters using conditional normalizing flows.
problem Computing optimal transport maps and Wasserstein barycenters in high-dimensional spaces.
method Uses conditional normalizing flows to approximate distributions and solve the primal problem.
result Shows computational feasibility for hundreds of input distributions and yields accurate results.
This paper analyzes minibatch optimal transport distances and their applications.
problem Optimal transport distances are complex and impractical for large datasets.
method Extended analysis of minibatch optimal transport distances, focusing on various kernels and debiased functions.
result Minibatch optimal transport distances are unbiased estimators and have statistical and optimisation properties.
New method trains normalizing flows using entropy-regularized transport.
problem Training continuous normalizing flows efficiently.
method Formulates flows as gradients of scalar potentials, training only these potentials.
result Trains normalizing flows without explicit flow computation during training.
New sampling method uses gradient-free IPS with RKHS velocity field.
problem Efficient sampling from unnormalized target densities.
method Gradient-free interacting particle systems (IPS) with RKHS velocity field.
result IPS produce high-quality samples from various target distributions.
Gradient flow method solves for optimal transport starting distributions.
problem Finding the optimal starting distribution for a martingale in optimal transport.
method Following the gradient flow of the Bass functional's L2-lift.
result Gradient flow converges to a minimizer of the Bass functional.
Optimal Transport (OT) naturally arises in many machine learning applications, yet the heavy computational burden limits its wide-spread uses. To address the scalability issue, we propose an implicit generative learning-based framework called SPOT (Scalable Push-forward of Optimal Transport). Specifically, we approxima…
Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the po…
A new method for fast optimal transport using sliced Wasserstein generalized geodesics.
problem Computing optimal transport distances efficiently and accurately.
method Proposes a new proxy of squared Wasserstein distance based on one-dimensional projections.
result min-SWGG is an upper bound of Wasserstein distance with similar computational complexity.
Paper analyzes minibatch Wasserstein for machine learning applications.
problem Optimal transport distances on large datasets.
method Analysis of minibatch optimal transport.
result Minibatch Wasserstein is equivalent to implicit regularization with desirable properties.
We exploit the link between the transport equation and derivatives of expectations to construct efficient pathwise gradient estimators for multivariate distributions. We focus on two main threads. First, we use null solutions of the transport equation to construct adaptive control variates that can be used to construct…
Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
New method estimates rate-distortion function using optimal transport.
problem Estimating the fundamental performance limit of data compression.
method Wasserstein gradient descent to learn optimal reproduction distribution.
result Local convergence and sample complexity analysis of R-D estimator.
InfoOT improves data alignment by maximizing mutual information.
problem Optimal transport's limitations in handling clusters, outliers, and new data.
method InfoOT extends optimal transport by maximizing mutual information while minimizing distances.
result InfoOT outperforms optimal transport in domain adaptation, cross-domain retrieval, and single-cell alignment.
A new algorithm for optimizing probability distributions converges linearly.
problem Optimizing functionals over families of probability distributions.
method Variational transport: particle-based algorithm approximating Wasserstein gradient descent.
result Variational transport converges linearly to the global minimum of the objective functional.
Develops regularity theory for Beckmann's optimal transport problem.
problem Minimizing total squared flux in continuous transport from source to target.
method Unconstrained Lagrangian formulation, variational first order optimality conditions, Schauder estimates.
result Exact Hölder regularity of potential, flux, and flow generating on bounded, regular domains.
Study optimal transport for robust optimization, showing how adversary's strategy relates to regularization.
problem Optimizing under uncertain parameters with a fictitious adversary reshaping a reference distribution.
method Introduces optimal transport and regularization to relate robustification to variation and Lipschitz norms.
result Conditions for existence and computability of Nash equilibrium between decision-maker and adversary.
Unified theory of optimal transport for random measures.
problem Statistical uncertainty in optimal transport.
method Constructing L2 over Wasserstein space for random probability measures. result Unified treatment of random optimal transport and principled inference.
New method uses continuous OT for fairness, outperforming discrete OT.
problem Fairness issues in machine learning models.
method Stochastic-gradient fairness method based on continuous optimal transport.
result Continuous OT method outperforms discrete OT when data is limited.
The paper analyzes stability and convergence rates of entropic and Sinkhorn potentials.
problem Stability and convergence rates of entropic and Sinkhorn potentials.
method Semiconcavity properties of entropic potentials and Schrödinger bridges.
result Exponential convergence rates for gradient and Hessian of Sinkhorn iterates.