Survey of linking information geometry and optimal transport.
problem Connecting two geometric frameworks for probability measures.
method Exploration of interactions and links between information geometry and optimal transport.
result Outstanding questions for both disciplines.
Optimization geometry affects deep learning performance.
problem The impact of optimization geometry on deep learning performance.
method Analysis of pseudogradient methods for learning generalized linear models.
result Non-asymptotic bounds on generalization error characterize model performance.
NLGS optimizes latent geometry for better model performance.
problem Improving machine learning model performance by aligning latent space geometry with data structure.
method NLGS uses product manifolds with Gromov-Hausdorff distance for latent geometry search.
result NLGS finds optimal latent geometry with query-efficient Bayesian optimization.
Cosine schedule is optimal for discrete diffusion models.
problem Choosing the best discretization schedule for diffusion models.
method Optimized using Fisher-Rao geometry.
result Cosine schedule is Fisher-Rao optimal.
Quantum field theory connects Riemannian geometry to quantum fluctuations.
problem Generating Riemannian structures from quantum fluctuations.
method QFT approach to Riemannian Geometry, focusing on Ricci curvature.
result Ricci curvature is crucial in generating Riemannian structures.
New Riemannian geometry for Compound Gaussian distributions applied to efficient change detection.
problem Change detection in multivariate image times series.
method Developed a recursive approach based on Riemannian optimization.
result Optimal performance achieved with computational efficiency.
New geometry for optimal transport cost based on Bregman divergences.
problem Optimal transport cost calculation with Bregman divergences.
method Established properties, defined interpolations, constructed dualistic geometry.
result Derived generalized Pythagorean inequality and Bregman-Wasserstein barycenters.
The Fisher-Rao geometry is applied to elliptical distributions for optimization and classification.
problem Optimizing and classifying covariance matrices using geometric tools.
method Riemannian optimization and intrinsic Cramér-Rao bounds.
result Geometric tools enhance covariance matrix estimation and classification.
Optimization rates improved for manifolds with bounded geometry.
problem Optimizing functions on manifolds with bounded geometry.
method Riemannian gradient descent and dynamic trivialization algorithm.
result Curvature-dependent convergence rates computed explicitly for common manifolds.
New approach uses isotropic geometry to solve Euclidean problems.
problem Solving systems of constraints in Euclidean geometry.
method Start with analogous problems in isotropic geometry to initialize optimization algorithms.
result Solutions in isotropic geometry provide insight and initialize Euclidean problem solutions.
This paper proposes a new geometric model optimization method.
problem Building adaptive manifold models with dynamic geometry.
method Optimizing metric tensor field on a manifold with variational framework.
result Metric optimization yields models with greater expressive power than fixed geometry models.
Develops Riemannian geometry for optimization on manifolds with detailed derivations.
problem Abstract high-level optimization on nonlinear spaces like matrix manifolds.
method Systematic derivation of geometric structures and constructions in coordinates and matrix form.
result Unified treatment of Riemannian geometry for optimization on manifolds.
Optimal prototypes found for challenging pathological geometries.
problem Finding optimal prototypes for pathological geometries is challenging.
method Analytical and heuristic algorithms for finding nearly-optimal prototypes.
result Optimal prototypes can be found analytically for challenging geometries.
CoNES optimizes blackbox functions using convex optimization and information geometry.
problem Optimizing high-dimensional blackbox functions efficiently.
method Formulated as a convex program that adapts evolutionary strategies gradient estimates.
result Vastly outperforms conventional blackbox optimization methods on benchmarks and MuJoCo tasks.
We study computational and statistical consequences of problem geometry in stochastic and online optimization. By focusing on constraint set and gradient geometry, we characterize the problem families for which stochastic- and adaptive-gradient methods are (minimax) optimal and, conversely, when nonlinear updates -- su…
Diffeologies unify infinite-dimensional geometry and PDEs, enhancing classical function spaces.
problem Combining infinite-dimensional geometry and PDEs for optimization problems.
method Review and extension of classical function spaces and mapping spaces.
result Diffeologies provide a unified framework for evolution equations and optimization problems.
Non-Euclidean BPM extends optimization theory to non-Euclidean norms.
problem Extending BPM's convergence theory to non-Euclidean norms.
method Iteratively minimizing over norm balls in non-Euclidean geometry.
result Most BPM guarantees carry over to non-Euclidean norms.
Optimal transport and information geometry both study geometric structures on spaces of probability distributions. Optimal transport characterizes the cost-minimizing movement from one distribution to another, while information geometry originates from coordinate-invariant properties of statistical inference. Their con…
We discuss contact geometry naturally related with optimal control problems (and Pontryagin Maximum Principle). We explore and expand the observations of [Ohsawa, 2015], providing simple and elegant characterizations of normal and abnormal sub-Riemannian extremals.
Geometric analysis on diffeomorphism groups for fluid dynamics and information geometry.
problem Geometric analysis of fluid flows and optimal mass transport.
method Review of metrics and topology on diffeomorphism groups.
result Introduction of new metrics and topology for diffeomorphism groups.
This work investigates implicit bias in multiclass separable data using a novel geometry-aware optimizer.
problem Understanding implicit bias in overparameterized models on multiclass separable data.
method Introduces NucGD, a geometry-aware optimizer enforcing low-rank structures through nuclear norm constraints.
result NucGD enables scalable training and characterizes the impact of stochastic optimization dynamics.
Deep learning models viewed through tame geometry for convergence guarantees.
problem Understanding convergence guarantees in deep learning models.
method Introducing tame geometry concepts and tools for nonsmooth nonconvex settings.
result Illustrates tame geometry as a natural framework for AI systems, especially deep learning.
Geometric structure reveals optimal investment and hedging products.
problem Optimal design of investment and hedging products.
method Investigation of geometric structure in risks and returns using a simple formula.
result Duality between hedging and investment with geometric interpretation of rationality.
Survey explores geometric aspects of policy optimization in control systems.
problem Understanding the geometric relationships between control design and optimization.
method Geometric perspective on policy optimization, focusing on parameterization and topology.
result Implications of policy geometry on stability and performance of local search algorithms.
A function is exponentially concave if its exponential is concave. We consider exponentially concave functions on the unit simplex. In a previous paper we showed that gradient maps of exponentially concave functions provide solutions to a Monge-Kantorovich optimal transport problem and give a better gradient approximat…
We define the notion of sub-Finsler geometry as a natural generalization of sub-Riemannian geometry with applications to optimal control theory. We compute a complete set of local invariants, geodesic equations, and the Jacobi operator for the three-dimensional case and investigate homogeneous examples.
We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or separable linear classification problems. We explore the question of whether the specifi…
A geometric approach to differential game theory is illustrated. The parallel pursuit is considered as a two-player zero-sum differential game. The optimal strategies of each player is designed based on Riemann-Finsler geometry. Our approach incorporates a closed loop optimal control and the presentation is familiar wi…
Paper explores Monge-Ampère in deep learning and quantum geometry.
problem Understanding the Monge-Ampère equation in deep learning.
method Review of Boltzmann learning, connection to optimal transport, insights from quantum geometry, renormalization group flow.
result Space of covariance matrices in learning dynamics coincides with the CAH cone.
A new geometry for comparing signals, overcoming traditional limitations.
problem Comparing and interpolating discontinuous and signed signals.
method Investigation of Riemannian geometry on signal space, introducing a metric that measures both horizontal and vertical deformations.
result Characterization of metric properties and establishment of geodesic regularity and stability.
New algorithms achieve uniform stability for empirical risk minimization.
problem Designing uniformly stable optimization algorithms for empirical risk minimization.
method Black-box conversion of smooth optimization algorithms and development of Mirror Descent for smooth optimization.
result Optimal algorithms with uniform stability and convergence rates for smooth optimization.
Stochastic optimization is key to efficient inversion in PDE-constrained optimization. Using 'simultaneous shots', or random superposition of source terms, works very well in simple acquisition geometries where all sources see all receivers, but this rarely occurs in practice. We develop an approach that interpolates d…
Adaptive optimization methods bias neural network trajectories towards regions of lower local geometry.
problem The success of adaptive optimization methods in neural networks is not fully explained by traditional second-order methods.
method Local trajectory analysis and introduction of a new statistic RextmedextOPT. result Adaptive methods like Adam bias trajectories towards regions of lower local geometry, leading to faster convergence.
The paper studies the geometry of probability measures on the unit circle.
problem Understanding the Riemannian geometry of probability measures on the unit circle.
method Developed an intrinsic framework using the Peter-Weyl Theorem to study the differential geometry of Wasserstein spaces of compact Lie groups.
result Explicitly demonstrated that the Wasserstein space of the unit circle is flat with vanishing curvature.
The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.
problem Optimal transport problem in Linear Quadratic optimal control systems.
method Well-posedness of the Monge problem, regularity of optimal transport map, displacement interpolation of measures.
result Derivation of general interpolation inequalities for entropy functionals.
It is well-known that normal extremals in sub-Riemannian geometry are curves which locally minimize the energy functional. Most proofs of this fact do not make, however, an explicit use of relations between local optimality and the geometry of the problem. In this paper, we provide a new proof of that classical result,…
The geometry and analysis on Finsler manifolds is a very important part of Finsler geometry. In this article, we introduce some important and fundamental topics in global Finsler geometry and discuss the related properties and the relationships in them. In particular, we optimize and improve the various definitions of …
Optimization with inequality constraints using embedded gradient vector field method
problem Optimization with inequality constraints
method Geometric framework using quadratic slack variables
result Derives Lagrange multiplier functions and second-order optimality conditions
This note exposes the differential topology and geometry underlying some of the basic phenomena of optimal transportation. It surveys basic questions concerning Monge maps and Kantorovich measures: existence and regularity of the former, uniqueness of the latter, and estimates for the dimension of its support, as well …
Given a family of probability measures in P(X), the space of probability measures on a Hilbert space X, our goal in this paper is to highlight one ore more curves in P(X) that summarize efficiently that family. We propose to study this problem under the optimal transport (Wasserstein) geometry, using curves that are re…
Personalization of cardiac models involves the optimization of organ tissue properties that vary spatially over the non-Euclidean geometry model of the heart. To represent the high-dimensional (HD) unknown of tissue properties, most existing works rely on a low-dimensional (LD) partitioning of the geometrical model. Wh…
New geometric approach gives apriori estimate for optimal transport maps.
problem Proving regularity of optimal transport maps under Ma--Trudinger--Wang condition.
method Geometric derivation using pseudo-Riemannian geometry.
result New derivation of C1 interior estimate for optimal maps. The paper presents the geometry of Lie algebroids and its applications to optimal control. The first part deals with the theory of Lie algebroids, connections on Lie algebroids and dynamical systems defined on Lie algebroids (mainly Lagrangian and Hamiltonian systems). In the second part we use the framework of Lie alg…
High-dimensional random geometry shows phase transitions in various problems.
problem Phase transitions in high-dimensional random geometry.
method Analysis of various financial, optimization, and ecological problems.
result Links between seemingly distant fields and further ramifications.
Develops optimal low-dimensional approximations to high-dimensional SDEs.
problem Approximating solutions to high-dimensional SDEs in a low-dimensional space.
method Introduces Ito-vector and Ito-jet projections for optimal approximation.
result Optimal projection filters yield better approximations than Stratonovich projection.
New saddle network architectures preserve convex-concave geometry in optimization problems.
problem Optimization models with convex x and concave y components.
method Structured separable decomposition and saddle network architectures.
result Proven one-dimensional approximation theorem and high accuracy on various test functions.
In this series of lectures we introduce the Monge-Kantorovich problem of optimally transporting one distribution of mass onto another, where optimality is measured against a cost function c(x,y). Connections to geometry, inequalities, and partial differential equations will be discussed, focusing in particular on recen…
Optimization on manifolds is a class of methods for optimization of an objective function, subject to constraints which are smooth, in the sense that the set of points which satisfy the constraints admits the structure of a differentiable manifold. While many optimization problems are of the described form, technicalit…