arXiv research

In this paper, we study Clifford-Wolf translations of Finsler spaces. We first give a characterization of Clifford-Wolf translations of Finsler spaces in terms of Killing vector fields. In particular, we show that there is a natural correspondence between Clifford-Wolf translations and the Killing vector fields of cons…

Improved Frank-Wolfe algorithm for polytopes converges linearly with dimension dependence on optimal face.

New algorithm improves CRF inference and learning.

The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications. However, its convergence rate is known to be slow (sublinear) when the solution lies at the boundary. A simple less-know…

This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.

In this paper, we introduce a new type of Finsler metrics, called $(α_1,α_2)$-metrics. We define the notion of the good datum of a homogeneous $(α_1,α_2)$-metric and use that to study the geometric properties. In particular, we give a formula of the S-curvature and deduce a condition for the S-curvature to be vanishing…

New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.

Paper improves Frank-Wolfe algorithm's efficiency bounds.

We consider a distributed parameter estimation problem, in which multiple terminals send messages related to their local observations using limited rates to a fusion center who will obtain an estimate of a parameter related to observations of all terminals. It is well known that if the transmission rates are in the Sle…

Improved Frank-Wolfe algorithm for constrained convex optimization with nearest extreme point oracle.

New method solves stochastic optimization problems with affine constraints.

A Clifford-Wolf translation of a connected Finsler space is an isometry which moves each point the same distance. A Finsler space $(M, F)$ is called Clifford-Wolf homogeneous if for any two points $x_1, x_2\in M$ there is a Clifford-Wolf translation $ρ$ such that $ρ(x_1)=x_2$. In this paper, we give a complete classifi…

Unified framework for efficient Frank-Wolfe optimization of Dominant Set Clustering.

A Clifford-Wolf translation of a connected Finsler space is an isometry which moves each point the sam distance. A Finsler space $(M, F)$ is called Clifford-Wolf homogeneous if for any two point $x_1, x_2\in M$ there is a Clifford-Wolf translation $ρ$ such that $ρ(x_1)=x_2$. In this paper, we study Clifford-Wolf transl…

The move from hand-designed to learned optimizers in machine learning has been quite successful for gradient-based and -free optimizers. When facing a constrained problem, however, maintaining feasibility typically requires a projection step, which might be computationally expensive and not differentiable. We show how …

Paper studies Frank-Wolfe algorithm for solving sparse reconstruction problems.

Boosted Frank-Wolfe accelerates optimization for nonconvex problems.

New Frank-Wolfe method for sparse neural networks.

Unified view of Lion and Muon as Stochastic Frank-Wolfe methods.

We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…

Recovering matrices from compressive and grossly corrupted observations is a fundamental problem in robust statistics, with rich applications in computer vision and machine learning. In theory, under certain conditions, this problem can be solved in polynomial time via a natural convex relaxation, known as Compressive …

Improved Frank-Wolfe for sparse/low-rank problems.

Momentum accelerates Frank Wolfe algorithms on certain problems.

We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constan…

Improved Frank-Wolfe algorithm for generalized self-concordant functions converges quickly.

Three online algorithms for submodular maximization with varying feedback types.

A dissertation on scalable projection-free optimization methods.

We study the formality of the total space of principal SU(2) and SO(3)-bundles over a Wolf space, that is a symmetric positive quaternionic Kähker manifold. We apply this to conclude that all the 3-Sasakian homogeneous spaces are formal. We also determine the principal SU(2) and SO(3)-bundles over the Wolf spaces whose…

We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…

New method solves constrained self-concordant minimization problems efficiently.

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate $O\left(\frac{1}{\varepsilon^2}\right)$ improving over the projection-free, Online Frank-Wolfe based stochastic gradient descent of Hazan an…

We propose a rank-$k$ variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation ($1$-SVD) in Frank-Wolfe with a top-$k$ singular-vector computation ($k$-SVD), which can be done by repeatedly applying $1$-SVD $k$ times. …

New Frank-Wolfe algorithm speeds up SVM-type multi-category learning.

Improved Frank-Wolfe method reduces dependence on data size for empirical risk minimization.

New algorithm estimates barycenters of distributions using Frank-Wolfe.

The main purpose of the following article is to introduce a \emph{Lie theoretical} approach to the problem of classifying pseudo quaternionic-Kähler (QK) reductions of the pseudo QK symmetric spaces, otherwise called \emph{generalized Wolf spaces}.

In this paper, we study Clifford-Wolf translations of homogeneous Randers metrics on spheres. It turns out that we can present a complete description of all the Clifford-Wolf translations of all the homogeneous Randers metrics on spheres. The most important point of this paper is that a new phenomena surfaces. Namely, …

We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope. The algorithm is based on the conditional gradient method (Frank-Wolfe) and moves pseudomarginals within the marginal polytope through repeated maximum a posteriori (MAP) calls. This m…

The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm constrained problems, since it does not require projections. However, FW often yields high rank intermediate iterates, which can be very expensive in time and space costs for large problems. To address this issue, we propose a rank-drop method …

An isometry of a Finsler space is called Clifford-Wolf translation (CW-translation) if it moves all points the same distance. A Finsler space $(M, F)$ is called Clifford-Wolf homogeneous (CW-homogeneous) if for any $x, y\in M$ there is a CW-translation $σ$ such that $σ(x)=y$. We prove that if $F$ is a homogeneous Finsl…

Researchers found a counterexample disproving a 1962 conjecture.

Optimized algorithms for online learning with linear constraints improve performance and provide worst-case analysis.

Frank-Wolfe optimization applied to a small deep network shows slower convergence compared to gradient descent.

In this paper, using connections between Clifford-Wolf isometries and Killing vector fields of constant length on a given Riemannian manifold, we classify simply connected Clifford-Wolf homogeneous Riemannian manifolds. We also get the classification of complete simply connected Riemannian manifolds with the Killing pr…

A new method improves convergence in large-scale stochastic optimisation.

Quantized Frank-Wolfe reduces communication costs in distributed optimization.

Motivated principally by the low-rank matrix completion problem, we present an extension of the Frank-Wolfe method that is designed to induce near-optimal solutions on low-dimensional faces of the feasible region. This is accomplished by a new approach to generating ``in-face" directions at each iteration, as well as t…

Improved Frank-Wolfe algorithm speeds up training of differentially private LASSO models.