OTSeg uses multi-prompt Sinkhorn attention to improve zero-shot semantic segmentation.
problem Leveraging pre-trained CLIP knowledge to align text embeddings with pixel embeddings.
method OTSeg employs Multi-Prompts Sinkhorn (MPS) and Multi-Prompts Sinkhorn Attention (MPSA) to enhance semantic feature matching.
result OTSeg achieves state-of-the-art performance in zero-shot semantic segmentation tasks.
Paper proposes Sinkformers for Transformers with doubly stochastic attention.
problem Improving Transformer models' accuracy in vision and natural language processing.
method Using Sinkhorn's algorithm to make attention matrices doubly stochastic instead of SoftMax normalization.
result Sinkformers enhance model accuracy in vision and natural language processing tasks.
Applications of optimal transport have recently gained remarkable attention thanks to the computational advantages of entropic regularization. However, in most situations the Sinkhorn approximation of the Wasserstein distance is replaced by a regularized version that is less accurate but easy to differentiate. In this …
New initialization methods speed up Sinkhorn algorithm for OT problems.
problem Improving runtime of the Sinkhorn algorithm for optimal transport problems.
method Data-dependent initializers for Sinkhorn algorithm, based on closed-form solutions for specific settings.
result Data-dependent initializers result in dramatic speed-ups without affecting differentiability.
Improved online Sinkhorn algorithm for large-scale data processing.
problem Efficiency of Sinkhorn algorithm for large-scale data streams.
method Revisit and improve convergence analysis of online Sinkhorn algorithm, introduce compressed online Sinkhorn algorithm.
result New faster convergence rate for online Sinkhorn algorithm under certain conditions.
Continuous-time Sinkhorn flow generalizes and unifies existing dynamics.
problem Entropy-regularized optimal transport problems.
method Continuous-time mirror descent framework.
result Unified perspective on various dynamics in ML and math.
The Sinkhorn-Knopp derivatives converge with linear rate.
problem Optimal transport problem with entropic regularization.
method Iterative proportional fitting procedure.
result Derivatives converge with linear rate.
Paper solves barycenter of probability distributions using Sinkhorn divergence.
problem Computing the barycenter of a set of probability distributions under the Sinkhorn divergence.
method Recast as unconstrained functional optimization and develop Sinkhorn Descent (SD) method.
result SD converges to a stationary point at a sublinear rate and asymptotically finds a global minimizer.
Improved Sinkhorn algorithm for UOT with near-linear complexity.
problem Solving the entropic regularized Unbalanced Optimal Transport problem efficiently.
method Geometric convergence analysis of Sinkhorn updates and primal solution properties.
result Near-linear time complexity for finding ε \varepsilon ε -approximate UOT solutions. We introduce in this paper a novel strategy for efficiently approximating the Sinkhorn distance between two discrete measures. After identifying neglectable components of the dual solution of the regularized Sinkhorn problem, we propose to screen those components by directly setting them at that value before entering t…
Survey of Sinkhorn algorithm for optimal transport, emphasizing its geometric origins.
problem Solving optimal transport problems efficiently and accurately.
method Discretization of a non-linear integral equation.
result Geometric interpretation and discretization of the Sinkhorn algorithm.
Generative model extracts road networks from images.
problem Extracting road networks from image data.
method Developed Generative Graph Transformer (GGT) for image-conditioned graph generation.
result Demonstrated effective road network generation from semantic segmentation data.
SNS accelerates Sinkhorn algorithm with sparse Newton iterations.
problem Slow runtime of Sinkhorn algorithm for optimal transport.
method Early stopping and Newton-type subroutine for matrix scaling steps.
result SNS converges orders of magnitude faster than Sinkhorn.
Transformers model contextual relations using probabilistic measures, revealing their expressive power.
problem Lack of clear understanding of Transformer's ability to model contextual relations.
method Introduced a measure-theoretic framework connecting softmax attention and entropy-regularized optimal transport.
result Transformer architectures can approximate arbitrary contextual relations, and the choice of normalization affects how these relations are represented.
The Sinkhorn flow converges to a Wasserstein mirror gradient flow from the Sinkhorn algorithm.
problem Optimizing joint distributions using the Sinkhorn algorithm.
method Wasserstein mirror gradient flow derived from the Sinkhorn algorithm.
result The Sinkhorn flow converges to a Wasserstein mirror gradient flow.
New stability theory for Sinkhorn semigroups with explicit decay rates.
problem Stability and convergence of Sinkhorn iterations for various divergences.
method Operator-theoretic framework based on Lyapunov techniques.
result Explicit exponential decay rates for Sinkhorn iterates.
The Sinkhorn "distance", a variant of the Wasserstein distance with entropic regularization, is an increasingly popular tool in machine learning and statistical inference. However, the time and memory requirements of standard algorithms for computing this distance grow quadratically with the size of the data, making th…
This work studies the statistical performance of Sinkhorn iterations in estimating Schrödinger bridges.
problem Estimating Schrödinger bridges with limited samples.
method Intermediate Sinkhorn iterations applied to the time-dependent drifts of SDEs.
result Established a statistical bound on the squared total variation error of Sinkhorn bridge iterations.
This paper tackles Sinkhorn DRO by reformulating it as a bilevel program and proposes sampling-based algorithms.
problem Distributionally robust optimization with ambiguity sets defined via the Sinkhorn discrepancy.
method Primal perspective reformulation as a bilevel program, double-loop and single-loop sampling-based algorithms.
result Simultaneously obtain the optimal robust decision and the worst-case distribution.
Paper uses Sinkhorn distances to improve imitation learning effectiveness.
problem Improving imitation learning algorithms by comparing occupancy measures.
method Formulates imitation learning as Sinkhorn distance minimization, combining optimal transport and cosine distances.
result Proposes a new critic network and transport plan that guide imitation learning.
Optimizes distributions robustly with Sinkhorn distance.
problem Distributionally robust optimization with Wasserstein distance.
method Convex programming dual reformulation, stochastic mirror descent algorithm.
result Demonstrates superior performance in synthetic and real data.
This paper refines the Gaussian Sinkhorn algorithm for general multivariate models.
problem Finite-dimensional solutions for general Gaussian multivariate models.
method Recursive formulation of the Sinkhorn algorithm for Gaussian models, including closed form expressions of entropic transport maps and Schrödinger bridges.
result Refined convergence analysis of Gaussian Sinkhorn algorithms.
A new associative memory uses Sinkhorn divergence for efficient pattern retrieval.
problem Efficiently retrieving patterns from large datasets of weighted point clouds.
method Derived retrieval dynamics as a SHK gradient flow, discretized for a deterministic algorithm.
result Proved basin invariance, geometric convergence, and robust recovery from perturbations.
This paper generalizes Sinkhorn algorithm for unbalanced optimal transport.
problem Handling distributions with different total mass and robustness to outliers.
method Alternates between standard Sinkhorn updates and pointwise application of a contractive function.
result Defines Sinkhorn divergences that are differentiable, positive, definite, convex, and robust.
Develops a new algorithm to calibrate signed datasets to specified marginals.
problem Calibrating signed datasets to specified marginals.
method Extends Schrödinger-Fortet-Sinkhorn paradigm to sign-indefinite multi-dimensional arrays.
result Proposes an optimization problem to update a sign-indefinite prior to match given marginals.
A new algorithm for training generative models using Sinkhorn divergence.
problem Training generative adversarial networks (GANs).
method Sinkhorn Natural Gradient (SiNG) algorithm for steepest descent on probability space.
result Explicit expression and efficient evaluation of the Sinkhorn information matrix (SIM).
Computing optimal transport distances such as the earth mover's distance is a fundamental problem in machine learning, statistics, and computer vision. Despite the recent introduction of several algorithms with good empirical performance, it is unknown whether general optimal transport distances can be approximated in …
We present a novel algorithm to estimate the barycenter of arbitrary probability distributions with respect to the Sinkhorn divergence. Based on a Frank-Wolfe optimization strategy, our approach proceeds by populating the support of the barycenter incrementally, without requiring any pre-allocation. We consider discret…
Study convergence and approximations of entropic regularized Wasserstein distances for Gaussian and RKHS measures.
problem Convergence and approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings.
method Analysis of convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings.
result Strictly weaker convergence in 2-Sinkhorn divergence for Gaussian measures compared to exact 2-Wasserstein distance.
Paper proposes a robust hypothesis testing method using Sinkhorn distance.
problem Hypothesis testing for small samples.
method Data-driven approach using Sinkhorn uncertainty sets.
result The method provides a more flexible detector compared to Wasserstein robust test.
Paper proposes SinkhornDRL for distributional RL using Sinkhorn divergence and regularized Wasserstein loss.
problem Improving distributional reinforcement learning by minimizing Bellman return distribution differences.
method Introduces SinkhornDRL, a distributional RL algorithm using Sinkhorn divergence and regularized Wasserstein loss.
result SinkhornDRL consistently outperforms or matches existing algorithms on Atari games, especially in multi-dimensional reward settings.
Optimal transport with f f f -divergence regularization using generalized Sinkhorn algorithm.
problem Optimal transport with f f f -divergence regularization. method Generalized Sinkhorn algorithm for solving optimal transport problems with various f f f -divergences. result Strong duality holds, optimums are attained, and convergence to an optimal solution is guaranteed under certain conditions.
A new method for faster estimation of Wasserstein distance using Sinkhorn divergence.
problem Estimating the squared Wasserstein distance between probability distributions.
method Proposes a new estimator based on the Sinkhorn divergence with debiasing terms, and analyzes its sample complexity and computational efficiency.
result The proposed estimator allows higher regularization levels, leading to improved computational complexity and speedup in practice.
New methods improve stability of Sinkhorn algorithm in machine learning.
problem Stability of Sinkhorn semigroups in high-dimensional settings.
method Semigroup analysis based on contraction coefficients and Lyapunov-type operator-theoretic techniques.
result Unified and simplified arguments in Sinkhorn algorithm stability.
The paper analyzes stability and convergence rates of entropic and Sinkhorn potentials.
problem Stability and convergence rates of entropic and Sinkhorn potentials.
method Semiconcavity properties of entropic potentials and Schrödinger bridges.
result Exponential convergence rates for gradient and Hessian of Sinkhorn iterates.
Paper proves generalized Talagrand inequality for Sinkhorn distance.
problem Proving a generalized Talagrand inequality for Sinkhorn distance.
method Using entropy power inequality and infinitesimal displacement convexity of optimal transport map.
result Extends previous results of Gaussian Talagrand inequality for Sinkhorn distance to strongly log-concave case.
End-to-end Sinkhorn Autoencoder reduces data simulation time with noise generation.
problem Efficiently simulating data collection processes with high fidelity and speed.
method End-to-end Sinkhorn Autoencoder with noise generator.
result Outperforms competing methods on various datasets.
Extends geostatistical simulation method to handle multiple variables and large grids.
problem Scalability and handling of multiple variables in geostatistical simulation.
method Uses Sinkhorn optimal transport with sparse matcher and FFT-MA Gaussian backbone.
result MST-Direct reproduces joint distribution with zero histogram error and accurately preserves spatial correlation.
Paper analyzes convergence of Sinkhorn algorithm for discrete probability measures on torus.
problem Achieving exponential convergence of Sinkhorn algorithm in general settings.
method Coupling by reflection techniques for controlled diffusions on the torus.
result Proves pointwise exponential convergence of Sinkhorn iterates and their gradient.
The Sinkhorn flow is a gradient flow in a nonlocal Wasserstein geometry.
problem Entropy-regularized optimal transport and its applications.
method Thermodynamic interpretation of the Sinkhorn algorithm.
result The Sinkhorn flow is the gradient flow of entropy in a nonlocal Wasserstein geometry.
A new method reduces the computational cost of Sinkhorn algorithm for OT and UOT problems.
problem High computational complexity of Sinkhorn algorithm for OT and UOT problems.
method Importance sparsification method called Spar-Sink to efficiently approximate entropy-regularized OT and UOT solutions.
result The method reduces computational cost from O ( n 2 ) O(n^2) O ( n 2 ) to O ~ ( n ) \widetilde{O}(n) O ( n ) , and is consistent under mild regularity conditions. We present several new complexity results for the entropic regularized algorithms that approximately solve the optimal transport (OT) problem between two discrete probability measures with at most n n n atoms. First, we improve the complexity bound of a greedy variant of Sinkhorn, known as \textit{Greenkhorn}, from $\wid…
New framework for robust hypothesis testing using Sinkhorn uncertainty sets.
problem Non-convex robust hypothesis testing problem.
method Exact mixed-integer exponential conic reformulation and convex approximation.
result Satisfactory testing performance and computational efficiency.
This paper surveys algorithmic advancements in Optimal Transport with applications in machine learning.
problem Quantifying differences between distributions in various fields.
method Examines classical and modern computational techniques, including Sinkhorn iterations and primal-dual strategies.
result Highlights the robustness and scalability of OT algorithms in high-dimensional problems.
Permutations and matchings are core building blocks in a variety of latent variable models, as they allow us to align, canonicalize, and sort data. Learning in such models is difficult, however, because exact marginalization over these combinatorial objects is intractable. In response, this paper introduces a collectio…
New discretization scheme for Wasserstein gradient flows using Schrödinger bridges.
problem Computing Wasserstein gradient flows efficiently and without score functions.
method Iterated Schrödinger bridge approximation with particle-based Sinkhorn algorithm.
result The scheme converges to Wasserstein gradient flows for certain flows, including heat flow.
New algorithm solves Schrödinger bridge problem with mismatched channels.
problem Solving Schrödinger bridge problem with input and noise channel mismatch.
method Design of a Sinkhorn recursion with memory for nonlinear PDEs.
result Demonstrates solving control-affine Schrödinger bridge problem.
Estimating mutual information is an important statistics and machine learning problem. To estimate the mutual information from data, a common practice is preparing a set of paired samples { ( x i , y i ) } i = 1 n ∼ i . i . d . p ( x , y ) \{(\mathbf{x}_i,\mathbf{y}_i)\}_{i=1}^n \stackrel{\mathrm{i.i.d.}}{\sim} p(\mathbf{x},\mathbf{y}) {( x i , y i ) } i = 1 n ∼ i.i.d. p ( x , y ) . However, in many situations, it…