We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Frank-Wolfe Reduction”
Improved Frank-Wolfe algorithm speeds up training of differentially private LASSO models.
problem Training differentially private LASSO models on sparse data.
method Adapted Frank-Wolfe algorithm for sparse inputs, reducing runtime.
result Training time reduced from to .
Unified framework for efficient Frank-Wolfe optimization of Dominant Set Clustering.
problem Optimizing Dominant Set Clustering with various Frank-Wolfe algorithms.
method Unified framework for pairwise, standard, and away-steps Frank-Wolfe algorithms, with explicit convergence rates.
result Explicit convergence rates for Frank-Wolfe methods in Dominant Set Clustering.
Boosted Frank-Wolfe accelerates optimization for nonconvex problems.
problem Optimizing nonconvex and quasar-convex objectives efficiently.
method Developed a novel step size strategy for stochastic Frank-Wolfe, extending it to various gradient estimators.
result Boosted Frank-Wolfe achieves faster convergence rates than non-boosted Frank-Wolfe.
Unified view of Lion and Muon as Stochastic Frank-Wolfe methods.
problem Optimization of constrained problems in deep learning.
method Interpreting Lion and Muon as Stochastic Frank-Wolfe methods and extending the approach to heavy-tailed noise.
result Convergence guarantees and KKT point convergence for Lion and Muon.
New algorithm for fair ranking in contextual bandits with concave rewards.
problem Fair ranking in recommendation systems.
method Geometric interpretation of CBCR as optimization, Frank-Wolfe analyses.
result First algorithm with provably vanishing regret for CBCR.
Momentum accelerates Frank Wolfe algorithms on certain problems.
problem Improving convergence rate of Frank Wolfe algorithms.
method Introducing momentum into Frank Wolfe algorithms and proving faster convergence rate.
result Accelerated Frank Wolfe (AFW) converges with a faster rate of .
The Frank-Wolfe method and its extensions are well-suited for delivering solutions with desirable structural properties, such as sparsity or low-rank structure. We introduce a new variant of the Frank-Wolfe method that combines Frank-Wolfe steps and steepest descent steps, as well as a novel modification of the "Frank-…
We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constan…
Improved Frank-Wolfe algorithm for generalized self-concordant functions converges quickly.
problem Efficiently solving learning problems with generalized self-concordant objectives.
method Simple Frank-Wolfe variant with open-loop step size strategy .
result Achieves convergence rate for primal and Frank-Wolfe gaps.
In this paper, we study the properties of the Frank-Wolfe algorithm to solve the \ExactSparse reconstruction problem. We prove that when the dictionary is quasi-incoherent, at each iteration, the Frank-Wolfe algorithm picks up an atom indexed by the support. We also prove that when the dictionary is quasi-incoherent, t…
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…
New method solves constrained self-concordant minimization problems efficiently.
problem Constrained self-concordant minimization problems.
method Newton Frank-Wolfe method using linear minimization oracles.
result The method uses nearly the same number of linear minimization calls as the Frank-Wolfe method.
In this paper, we propose three online algorithms for submodular maximisation. The first one, Mono-Frank-Wolfe, reduces the number of per-function gradient evaluations from [Chen2018Online] and [chen2018projection] to 1, and achieves a -regret bound of . The second one, Bandit-F…
We propose a rank- variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation (-SVD) in Frank-Wolfe with a top- singular-vector computation (-SVD), which can be done by repeatedly applying -SVD times. …
New algorithm improves CRF inference and learning.
problem Efficient inference and learning for dense CRFs.
method Regularized Frank-Wolfe algorithm for nonconvex CRF optimization.
result Regularized Frank-Wolfe outperforms mean field and CNN baselines.
New Frank-Wolfe algorithm speeds up SVM-type multi-category learning.
problem Improving pattern recognition performance in multi-category SVM learning.
method Developed a new optimization algorithm based on Frank-Wolfe framework for MC-SVM variants.
result Closed-form solutions for direction finding and line search in the Frank-Wolfe framework for MC-SVM.
Improved Frank-Wolfe method reduces dependence on data size for empirical risk minimization.
problem Reducing dependence on number of data observations in Frank-Wolfe methods.
method Taylor-series approximated gradients applied to Frank-Wolfe method.
result Significant speed-ups over existing methods on real-world datasets.
Paper improves Frank-Wolfe algorithm's efficiency bounds.
problem Establishing efficient iteration complexity for Frank-Wolfe algorithm.
method Using metric entropy to provide lower bounds.
result Frank-Wolfe requires many iterations for certain problems.
The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm constrained problems, since it does not require projections. However, FW often yields high rank intermediate iterates, which can be very expensive in time and space costs for large problems. To address this issue, we propose a rank-drop method …
A method for optimizing under unknown Markovian data distributions.
problem Optimizing under unknown and indirectly observed probability distributions via Markovian data.
method Data-driven distributionally robust optimization model with Frank-Wolfe algorithm.
result The proposed method finds a stationary point efficiently and outperforms state-of-the-art methods.
Paper addresses hybrid learning with constrained adversaries, achieving optimal performance.
problem Hybrid learning problem with i.i.d. features and adversarial labels.
method Structured adversarial setting, efficient algorithm with ERM oracle.
result Oracle-efficient algorithm with regret scaling with Rademacher complexity.
We propose a variant of the Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems. Our formulation includes Elastic Net, regularized SVMs and phase retrieval as special cases. The proposed Primal-Dual Block Frank-Wolfe algorithm reduces the per-iteration cost while maintaining linear conver…
Improved Frank-Wolfe algorithm for polytopes converges linearly with dimension dependence on optimal face.
problem Efficiently solving convex minimization problems over polytopes with linear rate.
method Revisiting Frank-Wolfe algorithm with strict complementarity assumption and away-steps.
result Linear convergence rate independent of polytope dimension for optimal face.
Optimized projection-free algorithms for online learning: construction and worst-case analysismath.OC
Optimized algorithms for online learning with linear constraints improve performance and provide worst-case analysis.
problem Improving online learning algorithms for constrained optimization problems.
method Developed an optimized variant of an online Frank-Wolfe algorithm and used semidefinite programming for numerical analysis.
result No pure online Frank-Wolfe algorithm can have a better regret guarantee than O(T^3/4) without additional assumptions.
Improved Frank-Wolfe algorithm for constrained convex optimization with nearest extreme point oracle.
problem Constrained smooth convex minimization with limited linear optimization oracle access.
method Frank-Wolfe algorithm with nearest extreme point oracle.
result Improved complexity bounds for specific feasible sets, including linear convergence for polytopes.
Frank-Wolfe optimization applied to a small deep network shows slower convergence compared to gradient descent.
problem Training deep neural networks is challenging due to many parameters and optimization difficulties.
method Frank-Wolfe optimization method applied to a deep network.
result Frank-Wolfe optimization converges slowly and is unstable in a stochastic setting.
Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Featuresmath.OC
Two new Frank-Wolfe algorithms improve convergence for constrained optimization.
problem Solving optimization problems with structured constraints in machine learning.
method Two new variants of the Frank-Wolfe (FW) method for stochastic finite-sum minimization.
result Best convergence guarantees for convex and non-convex objective functions.
New algorithm reduces online learning iterations by a factor of T^2/3.
problem Efficiency in online learning with smooth cost functions.
method Follow-the-Perturbed-Leader method using online primal-dual framework.
result Guaranteed T^2/3 regret for general online convex optimization.
FraPPE efficiently identifies Pareto optimal arms in multi-objective bandits.
problem Efficiently identifying Pareto optimal arms in multi-objective bandits with confidence.
method Deriving structural properties and using Frank-Wolfe optimisation to solve the maxmin optimisation problem.
result FraPPE achieves optimal sample complexity and identifies the exact Pareto set.
Minimizing a function over an intersection of convex sets is an important task in optimization that is often much more challenging than minimizing it over each individual constraint set. While traditional methods such as Frank-Wolfe (FW) or proximal gradient descent assume access to a linear or quadratic oracle on the …
We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework. Whenever possible, we perform computations asynchronously, which helps attain speedups on multicore machines as well as in distributed environments. Moreover…
We give a simple proof that the Frank-Wolfe algorithm obtains a stationary point at a rate of on non-convex objectives with a Lipschitz continuous gradient. Our analysis is affine invariant and is the first, to the best of our knowledge, giving a similar rate to what was already proven for projected gra…
There is renewed interest in formulating integration as an inference problem, motivated by obtaining a full distribution over numerical error that can be propagated through subsequent computation. Current methods, such as Bayesian Quadrature, demonstrate impressive empirical performance but lack theoretical analysis. A…
Paper addresses private online convex optimization with optimal algorithms in various geometries and high-dimensional bandits.
problem Private online convex optimization with streaming and continual release data.
method Proposes a private variant of online Frank-Wolfe algorithm with recursive gradients for variance reduction.
result Achieves optimal excess risk in linear time for and state-of-the-art excess risk for .
Motivated principally by the low-rank matrix completion problem, we present an extension of the Frank-Wolfe method that is designed to induce near-optimal solutions on low-dimensional faces of the feasible region. This is accomplished by a new approach to generating ``in-face" directions at each iteration, as well as t…
Recently, the Frank-Wolfe optimization algorithm was suggested as a procedure to obtain adaptive quadrature rules for integrals of functions in a reproducing kernel Hilbert space (RKHS) with a potentially faster rate of convergence than Monte Carlo integration (and "kernel herding" was shown to be a special case of thi…
We present a new proximal bundle method for Maximum-A-Posteriori (MAP) inference in structured energy minimization problems. The method optimizes a Lagrangean relaxation of the original energy minimization problem using a multi plane block-coordinate Frank-Wolfe method that takes advantage of the specific structure of …
Recovering matrices from compressive and grossly corrupted observations is a fundamental problem in robust statistics, with rich applications in computer vision and machine learning. In theory, under certain conditions, this problem can be solved in polynomial time via a natural convex relaxation, known as Compressive …
In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our i…
Boosting is a generic learning method for classification and regression. Yet, as the number of base hypotheses becomes larger, boosting can lead to a deterioration of test performance. Overfitting is an important and ubiquitous phenomenon, especially in regression settings. To avoid overfitting, we consider using …
New algorithms improve robust covariance estimation efficiency.
problem Efficiently compute robust covariance estimation for large datasets.
method Frank-Wolfe-based algorithms for Tyler's M-estimator.
result AFW and GAFW converge linearly under mild assumptions.
A dissertation on scalable projection-free optimization methods.
problem Efficient optimization algorithms for large-scale machine learning problems.
method Study of Frank-Wolfe variants and their extensions to distributed and derivative-free settings.
result Development of 1-SFW and QFW, achieving state-of-the-art complexity and efficiency.
Two of the most fundamental prototypes of greedy optimization are the matching pursuit and Frank-Wolfe algorithms. In this paper, we take a unified view on both classes of methods, leading to the first explicit convergence rates of matching pursuit methods in an optimization sense, for general sets of atoms. We derive …
Unified approach to graph matching using Hilbert spaces and entropy-regularized Frank-Wolfe algorithm.
problem Graph matching problems in computer vision, pattern recognition, and bioinformatics.
method Unified view of Koopmans-Beckmann's QAP and Lawler's QAP, introducing new rules for array operations in Hilbert spaces. Entropy-regularized Frank-Wolfe (EnFW) algorithm for QAP optimization.
result Our approach significantly outperforms state-of-the-art in matching accuracy and scalability.
Large-scale machine learning training suffers from two prior challenges, specifically for nuclear-norm constrained problems with distributed systems: the synchronization slowdown due to the straggling workers, and high communication costs. In this work, we propose an asynchronous Stochastic Frank Wolfe (SFW-asyn) metho…
Complexity Issues and Randomization Strategies in Frank-Wolfe Algorithms for Machine Learningstat.ML
Frank-Wolfe algorithms for convex minimization have recently gained considerable attention from the Optimization and Machine Learning communities, as their properties make them a suitable choice in a variety of applications. However, as each iteration requires to optimize a linear model, a clever implementation is cruc…
We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW optimization, we provide the first proof of convergence of a FW-type saddle point solv…