Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
New transforms improve signal classification and data analysis.
problem Improving signal classification and data analysis.
method Algebraic generative models and transport transforms.
result Classes of signals are transformed into convex sets, simplifying classification.
Fusion of transformer networks using optimal transport for improved performance.
problem Improving performance of transformer-based models through fusion.
method Exploiting optimal transport for soft alignment of transformer components.
result Consistently outperforms vanilla fusion and individual parent models.
IGSD separates task-specific content channels in transformer components by comparing activation replacement with zero ablation.
problem Mechanistic interpretability of transformer components
method IGSD: paired-intervention framework for comparing activation replacement with zero ablation
result IGSD identifies an early-layer content channel in transformer components that standard importance methods underestimate.
The paper introduces a new method for risk measurement using weak optimal transport.
problem Risk measurement in insurance and financial contexts.
method Convex risk measures with weak optimal transport penalties, explicit representation via nonlinear transform, computational aspects, and approximations using neural networks.
result Explicit representation and computational methods for risk measures.
Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can…
A new method tests conditional independence by transforming it into an unconditional problem using transport maps.
problem Testing conditional independence between two random vectors given a third.
method Constructing transport maps to transform conditional independence into unconditional independence, estimating these maps from data using conditional continuous normalizing flow models.
result The proposed method is validated through simulations and real-data analysis, demonstrating practical effectiveness.
A new method matches measures across different spaces using cost-regularized optimal transport.
problem Matching measures in different spaces without aligned data.
method Cost-regularized optimal transport formulation to match measures across two Euclidean spaces.
result Demonstrated applicability to single-cell spatial transcriptomics/multiomics matching tasks.
Solves open problem on simple surfaces with novel twistor correspondence.
problem Existence of nontrivial holomorphic vector bundles on simple surfaces.
method Novel twistor correspondence, Nash-Moser inverse function theorem, and microlocal analysis.
result Simple surface twistor space supports no nontrivial holomorphic vector bundles.
New method reduces label and data shifts between domains using optimal transport.
problem Label shift between source and target domains in domain adaptation.
method Developed theory and LDROT method to mitigate label and data shifts.
result Theoretical and experimental validation of LDROT's effectiveness.
The motivation for this paper stems \cite{CR} from the need to construct explicit isomorphisms of (possibly nontrivial) principal G-bundles on the space of loops or, more generally, of paths in some manifold M, over which I consider a fixed principal bundle P; the aforementioned bundles are then pull-backs of P…
The paper extends optimal transport for linear separability of sheared distributions in supervised learning.
problem Learning on the space of probability measures using shifts and scalings.
method Embedding probability measures into L2 spaces using optimal transport, then applying regular machine learning techniques. result Sheared distributions can be linearly separated under certain conditions, with bounds on transformations.
Paper generalizes tensor-train approximation for complex random variables.
problem Characterizing intractable high-dimensional random variables.
method Extends inverse Rosenblatt transform to general reference measures and integrates into deep variable transformation framework.
result Deep inverse Rosenblatt transport significantly expands tensor approximations for complex random variables.
LOT embeds distributions for linear separability and classification.
problem Distribution discrimination in various scientific fields.
method Linear Optimal Transport (LOT) embedding into L2 space. result LOT embeds distributions into linearly separable spaces for certain transformations and perturbations.
Unified framework for ensemble transport-based smoothing of non-Gaussian time series.
problem Bayesian time series re-analysis with non-Gaussian distributions.
method Measure transport approach to derive consistent prior-to-posterior transformations.
result General ensemble framework for transport-based smoothing of state-space models.
Optimal transport aligns source and target distributions for linear regression in 2D.
problem Domain adaptation for linear regression in 2D with limited target data.
method Combining K-means and optimal transport for estimating geometric transformations.
result Optimal transport recovers geometric transformations like rotations, translations, and homotheties.
s-OTDD compares datasets efficiently without training, robust to class variations.
problem Efficiently compare datasets without training or class variations.
method Moment Transform Projection (MTP) and sliced optimal transport.
result s-OTDD correlates with optimal transport and transfer learning performance.
Optimal transport as a loss for machine learning optimization problems has recently gained a lot of attention. Building upon recent advances in computational optimal transport, we develop an optimal transport non-negative matrix factorization (NMF) algorithm for supervised speech blind source separation (BSS). Optimal …
The paper studies ray transforms on surfaces with negative curvature, proving injectivity and determining connections and Higgs fields.
problem Injectivity of ray transforms on surfaces with negative curvature and determination of connections and Higgs fields.
method Analysis of Gaussian thermostats on compact Riemannian surfaces with negative curvature, proving injectivity and determining connections and Higgs fields.
result Injectivity of the thermostat ray transform and determination of connections and Higgs fields.
This paper generalizes neural transport learning for free energy estimation in arbitrary state spaces.
problem Efficient estimation of free energy in various state spaces.
method Generalized neural transport learning approach for arbitrary state spaces.
result Validation of the proposed method's effectiveness and efficiency in diverse settings.
Transformers model contextual relations using probabilistic measures, revealing their expressive power.
problem Lack of clear understanding of Transformer's ability to model contextual relations.
method Introduced a measure-theoretic framework connecting softmax attention and entropy-regularized optimal transport.
result Transformer architectures can approximate arbitrary contextual relations, and the choice of normalization affects how these relations are represented.
New emulator bridges simulators using conditional optimal transport.
problem Bridging simulators with minimal distortion.
method Flow-based approach to learn likelihood transport, COT-FM for optimal matching.
result Emulator accurately captures full correction between simulators.
Introduces VSMD to improve generative diffusion processes without high costs.
problem High training costs and scalability issues in generative diffusion processes.
method Introduces variational Schrödinger momentum diffusion (VSMD) with adaptively transport-optimized variational scores and critical-damping transform.
result Efficiently generates anisotropic shapes while maintaining transport efficacy, outperforming alternatives.
Let M be a closed orientable surface of negative curvature. A connection is said to be transparent if its parallel transport along closed geodesics is the identity. We describe all transparent SU(2)-connections and we show that they can be built up from suitable Bäcklund transformations.
Introduces triangular transport for uncertain data.
problem Uncertainty in complex systems without known probabilistic representations.
method Characterizes and manipulates unknown probability distributions using triangular transport maps.
result Triangular transport guarantees desirable mathematical and computational properties.
We extend martingale transport results to weak martingale transport.
problem Applying martingale transport results to weak martingale transport.
method Change of numeraire for weak martingale transport.
result Established the correspondence between stretched Brownian motion and its geometric counterpart.
We use mass-transportation as a tool to compare surfaces (2-manifolds). In particular, we determine the "similarity" of two given surfaces by solving a mass-transportation problem between their conformal densities. This mass transportation problem differs from the standard case in that we require the solution to be inv…
In this technical paper, we present a new formulation of higher parallel transport in strict higher gauge theory required for the rigorous construction of Wilson lines and surfaces. Our approach is based on an original notion of Lie crossed module cocycle and cocycle 1- and 2-gauge transformation with a non standard do…
New algorithm for estimating multivariate quantiles using stochastic optimal transport.
problem Estimating multivariate quantiles from data.
method Stochastic algorithm for entropic optimal transport in Banach spaces, using Fourier coefficients.
result Almost sure convergence of the stochastic algorithm in infinite-dimensional Banach spaces.
The paper tackles efficient computation of optimal transport by approximating conjugates with amortized optimization.
problem Efficient computation of convex conjugates in optimal transport is challenging and limits the quality of transport maps.
method The approach combines amortized approximations of conjugates with a fine-tuning solver to improve transport map quality.
result The method significantly improves the quality of transport maps for the Wasserstein-2 benchmark and models many 2D couplings and flows.
Paper proposes a new approach to optimal transport for vector and matrix densities.
problem Optimal transport for vector and matrix densities with positivity and action transitivity constraints.
method Gauge-theoretic approach using semi-direct product groups of diffeomorphisms and gauge transformations.
result Bures-type metrics on semi-direct product groups relate to Wasserstein-type metrics on vector and matrix densities via Riemannian submersions.
A study on optimizing self-attention in tabular data using Optimal Transport.
problem Improving efficiency and accuracy of self-attention in tabular classification tasks.
method Developed an OT-based algorithm to generate class-specific dummy Gaussian distributions and train an MLP.
result Achieved comparable accuracy to Transformers with reduced computational cost and efficiency.
Improves QMC for complex distributions using transport maps.
problem Challenges in applying QMC to general target distributions.
method Train a transport map to approximate target distributions, ensuring RQMC achieves superior error rates.
result Transport QMC achieves faster convergence rates than standard Monte Carlo under mild conditions.
The paper explores rectified flows and their relation to optimal transport.
problem Understanding the connection between rectified flows and optimal transport.
method Investigates invariance properties, explicit constructions, and analysis of rectified flows in various settings.
result Rectified flows, when gradient constrained, do not generally solve optimal transport problems.
We propose a new approach to graph compression by appeal to optimal transport. The transport problem is seeded with prior information about node importance, attributes, and edges in the graph. The transport formulation can be setup for either directed or undirected graphs, and its dual characterization is cast in terms…
The need to reason about uncertainty in large, complex, and multi-modal datasets has become increasingly common across modern scientific environments. The ability to transform samples from one distribution P to another distribution Q enables the solution to many problems in machine learning (e.g. Bayesian inference…
A novel framework synthesizes treatment data across sites using optimal transport.
problem Estimating treatment effects across different sites with varying conditions.
method Distributional causal inference, Optimal Transport for alignment of control group distributions.
result Synthetic treatment group data aligns with true target distribution under general conditions.
Random features are improved by variance-reducing couplings, enhancing machine learning models.
problem Improving the efficiency and accuracy of random features in machine learning.
method Using optimal transport theory to find couplings that reduce variance in random features.
result Theoretical and practical gains in efficiency and accuracy for various machine learning models.
ATD measures language distance using neural models, recovering linguistic groupings.
problem Lack of a unified quantitative measure for cross-linguistic distance.
method Pretrained multilingual language models, attention mechanisms, optimal transport.
result ATD quantifies representational distance between languages, recovering linguistic groupings.
We address the following problem: given two smooth densities on a manifold, find an optimal diffeomorphism that transforms one density into the other. Our framework builds on connections between the Fisher-Rao information metric on the space of probability densities and right-invariant metrics on the infinite-dimension…
Bispectral OT improves dataset comparison by preserving intrinsic coherence.
problem Ignoring intrinsic coherence in dataset comparisons using pairwise geometric distances.
method Introduces Bispectral Optimal Transport, a symmetry-aware extension of discrete OT.
result Transport plans computed with Bispectral OT achieve greater class preservation accuracy.
This paper proposes a new method for conditional sampling using optimal transport.
problem Sampling conditional distributions in Bayesian inference and density estimation.
method Iterative block-triangular transport maps solving an optimal transport problem with a weighted L2 cost function.
result The proposed method extends the data-driven approach for conditional sampling.
Sliced Optimal Transport simplifies OT for fast computation.
problem Efficient computation of distances and barycenters for probability measures.
method Combines OT, integral geometry, and statistics for fast computation.
result Retains rich geometric structure while speeding up computations.
Study improves Calderón problem for systems, uniquely determining connections and potentials.
problem Determining connections and potentials in systems from boundary measurements.
method Introduces complex ray transform and parallel transport, proving uniqueness.
result Uniquely determines (A,Q) up to natural gauge invariances. FairPOT balances fairness and AUC performance by selectively transforming risk scores.
problem Balancing fairness and AUC performance in high-stakes domains.
method FairPOT uses proportional optimal transport to selectively transform risk scores.
result FairPOT consistently improves fairness with minimal AUC degradation or even positive gains.
We present a constructive approach to surface comparison realizable by a polynomial-time algorithm. We determine the "similarity" of two given surfaces by solving a mass-transportation problem between their conformal densities. This mass transportation problem differs from the standard case in that we require the solut…
CR-UOT improves matching of heterogeneous single-cell omics profiles.
problem Matching nonnegative finite Radon measures across heterogeneous spaces.
method Cost-regularized unbalanced optimal transport (CR-UOT) framework.
result CR-UOT improves alignment of heterogeneous single-cell omics profiles.
Efficiently computes optimal transport maps and Wasserstein barycenters using conditional normalizing flows.
problem Computing optimal transport maps and Wasserstein barycenters in high-dimensional spaces.
method Uses conditional normalizing flows to approximate distributions and solve the primal problem.
result Shows computational feasibility for hundreds of input distributions and yields accurate results.