arXiv research

Hybrid model outperforms benchmarks in financial forecasting.

BOIS optimizes complex systems by combining known and unknown functions, improving efficiency.

A tutorial on optimizing complex functions with partial knowledge.

This paper presents a brief introduction to the key points of the Grey Machine Learning (GML) based on the kernels. The general formulation of the grey system models have been firstly summarized, and then the nonlinear extension of the grey models have been developed also with general formulations. The kernel implicit …

A framework for analyzing regularizers to ensure trustworthy theory-driven model estimation.

Unified framework for efficient Frank-Wolfe optimization of Dominant Set Clustering.

Unified view of Lion and Muon as Stochastic Frank-Wolfe methods.

Boosted Frank-Wolfe accelerates optimization for nonconvex problems.

We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…

Clustering analysis, a classical issue in data mining, is widely used in various research areas. This article aims at proposing a self-adaption grey DBSCAN clustering (SAG-DBSCAN) algorithm. First, the grey relational matrix is used to obtain the grey local density indicator, and then this indicator is applied to make …

We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constan…

Hybrid model combines physics and data to handle incomplete systems.

Improved Frank-Wolfe algorithm for polytopes converges linearly with dimension dependence on optimal face.

We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…

Frank-Wolfe optimization applied to a small deep network shows slower convergence compared to gradient descent.

Proposes a new model for equity options calibration.

New Frank-Wolfe algorithm speeds up SVM-type multi-category learning.

Dynamic risk assessment method for WUI fires improves upon static frameworks.

MOBONS optimizes complex systems with multi-objective Bayesian optimization.

Optimized algorithms for online learning with linear constraints improve performance and provide worst-case analysis.

Improved Frank-Wolfe algorithm for constrained convex optimization with nearest extreme point oracle.

We propose a rank-$k$ variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation ($1$-SVD) in Frank-Wolfe with a top-$k$ singular-vector computation ($k$-SVD), which can be done by repeatedly applying $1$-SVD $k$ times. …

Paper improves Frank-Wolfe algorithm's efficiency bounds.

Two new Frank-Wolfe algorithms improve convergence for constrained optimization.

Improved Frank-Wolfe method reduces dependence on data size for empirical risk minimization.

Motivated principally by the low-rank matrix completion problem, we present an extension of the Frank-Wolfe method that is designed to induce near-optimal solutions on low-dimensional faces of the feasible region. This is accomplished by a new approach to generating ``in-face" directions at each iteration, as well as t…

Residual generation helps diagnose engine faults using neural networks.

The Frank-Wolfe method and its extensions are well-suited for delivering solutions with desirable structural properties, such as sparsity or low-rank structure. We introduce a new variant of the Frank-Wolfe method that combines Frank-Wolfe steps and steepest descent steps, as well as a novel modification of the "Frank-…

Machine learning (ML) classifiers are vulnerable to adversarial examples. An adversarial example is an input sample which is slightly modified to induce misclassification in an ML classifier. In this work, we investigate white-box and grey-box evasion attacks to an ML-based malware detector and conduct performance eval…

The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications. However, its convergence rate is known to be slow (sublinear) when the solution lies at the boundary. A simple less-know…

Recently, the Frank-Wolfe optimization algorithm was suggested as a procedure to obtain adaptive quadrature rules for integrals of functions in a reproducing kernel Hilbert space (RKHS) with a potentially faster rate of convergence than Monte Carlo integration (and "kernel herding" was shown to be a special case of thi…

Algorithm optimizes cascaded functions with known structure.

We propose a variant of the Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems. Our formulation includes Elastic Net, regularized SVMs and phase retrieval as special cases. The proposed Primal-Dual Block Frank-Wolfe algorithm reduces the per-iteration cost while maintaining linear conver…

We demonstrate how to scalably solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO) over the constraint set. We prove that the number of LMO calls of our method is nearly the same as that of the Frank-Wolfe method in the L-smooth case. Specifically, our Newton Frank…

In many online learning problems the computational bottleneck for gradient-based methods is the projection operation. For this reason, in many problems the most efficient algorithms are based on the Frank-Wolfe method, which replaces projections by linear optimization. In the general case, however, online projection-fr…

The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm constrained problems, since it does not require projections. However, FW often yields high rank intermediate iterates, which can be very expensive in time and space costs for large problems. To address this issue, we propose a rank-drop method …

New algorithm improves CRF inference and learning.

A dissertation on scalable projection-free optimization methods.

We consider the problem of bandit optimization, inspired by stochastic optimization and online learning problems with bandit feedback. In this problem, the objective is to minimize a global loss function of all the actions, not necessarily a cumulative loss. This framework allows us to study a very general class of pro…

A Clifford-Wolf translation of a connected Finsler space is an isometry which moves each point the same distance. A Finsler space $(M, F)$ is called Clifford-Wolf homogeneous if for any two points $x_1, x_2\in M$ there is a Clifford-Wolf translation $ρ$ such that $ρ(x_1)=x_2$. In this paper, we give a complete classifi…

New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.

The move from hand-designed to learned optimizers in machine learning has been quite successful for gradient-based and -free optimizers. When facing a constrained problem, however, maintaining feasibility typically requires a projection step, which might be computationally expensive and not differentiable. We show how …

We consider a distributed parameter estimation problem, in which multiple terminals send messages related to their local observations using limited rates to a fusion center who will obtain an estimate of a parameter related to observations of all terminals. It is well known that if the transmission rates are in the Sle…

Two of the most fundamental prototypes of greedy optimization are the matching pursuit and Frank-Wolfe algorithms. In this paper, we take a unified view on both classes of methods, leading to the first explicit convergence rates of matching pursuit methods in an optimization sense, for general sets of atoms. We derive …

A Clifford-Wolf translation of a connected Finsler space is an isometry which moves each point the sam distance. A Finsler space $(M, F)$ is called Clifford-Wolf homogeneous if for any two point $x_1, x_2\in M$ there is a Clifford-Wolf translation $ρ$ such that $ρ(x_1)=x_2$. In this paper, we study Clifford-Wolf transl…

In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our i…

Kernel-based machine learning approaches are gaining increasing interest for exploring and modeling large dataset in recent years. Gaussian process (GP) is one example of such kernel-based approaches, which can provide very good performance for nonlinear modeling problems. In this work, we first propose a grey-box mode…

In this paper, we study Clifford-Wolf translations of Finsler spaces. We first give a characterization of Clifford-Wolf translations of Finsler spaces in terms of Killing vector fields. In particular, we show that there is a natural correspondence between Clifford-Wolf translations and the Killing vector fields of cons…