A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper explores properties of projections and gradient methods in hyperbolic space forms.
Optimizes reinsurance and investment strategies to minimize ruin probability.
New PG methods tackle nonconvex optimization with auto-conditioned stepsizes.
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
A new method for Bayesian inference in high dimensions using projected Stein variational gradient descent.
New projection techniques reduce the frequency of projections in solving LCPs.
This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems. Existing projection-free methods either achieve suboptimal regret bounds or have high per-iteration computational costs. To fill this gap, two efficient projection-free online methods called ORGFW and MORGFW are …
A new method improves stochastic gradient descent for faster and more efficient estimation.
Conditional gradients constitute a class of projection-free first-order algorithms for smooth convex optimization. As such, they are frequently used in solving smooth convex optimization problems over polytopes, for which the computational cost of orthogonal projections would be prohibitive. However, they do not enjoy …
A new method for Bayesian inference tackles high-dimensional problems.
Soft-Radial Projection solves gradient saturation in constrained deep learning.
Paper develops methods for non-quadratic loss low-rank matrix recovery.
EAGC boosts GCD by regulating gradient entanglement, improving known and novel category separability.
This paper deals with unsupervised clustering with feature selection. The problem is to estimate both labels and a sparse projection matrix of weights. To address this combinatorial non-convex problem maintaining a strict control on the sparsity of the matrix of weights, we propose an alternating minimization of the Fr…
Three new efficient algorithms project vectors onto weighted l1 ball.
We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …
We consider the problem of minimizing a Lipschitz differentiable function over a class of sparse symmetric sets that has wide applications in engineering and science. For this problem, it is known that any accumulation point of the classical projected gradient (PG) method with a constant stepsize satisfies the $L…
Accelerated optimization methods improve robustness and privacy in estimation.
We consider stochastic strongly convex optimization with a complex inequality constraint. This complex inequality constraint may lead to computationally expensive projections in algorithmic iterations of the stochastic gradient descent~(SGD) methods. To reduce the computation costs pertaining to the projections, we pro…
We propose a projected semi-stochastic gradient descent method with mini-batch for improving both the theoretical complexity and practical performance of the general stochastic gradient descent method (SGD). We are able to prove linear convergence under weak strong convexity assumption. This requires no strong convexit…
New analysis shows SNG's effectiveness in small samples.
FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.
This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists in minimizing a convex empirical risk function subject to an constraint, a pairwise constraint, or a pairwise constraint. …
New method speeds up PGD for CV robustness evaluation.
K-means is a classical clustering algorithm with wide applications. However, soft K-means, or fuzzy c-means at m=1, remains unsolved since 1981. To address this challenging open problem, we propose a novel clustering model, i.e. Probabilistic K-Means (PKM), which is also a nonlinear programming model constrained on lin…
The paper reinterprets policy gradient methods using operator theory.
SPGD improves adversarial training efficiency and accuracy.
Online optimization has been a successful framework for solving large-scale problems under computational constraints and partial information. Current methods for online convex optimization require either a projection or exact gradient computation at each step, both of which can be prohibitively expensive for large-scal…
Projective DP-SGD reduces privacy error by identifying low-dimensional gradient subspaces.
Gradient flow preserves speed for integral Menger curvature curves.
We study the projected gradient descent method on low-rank matrix problems with a strongly convex objective. We use the Burer-Monteiro factorization approach to implicitly enforce low-rankness; such factorization introduces non-convexity in the objective. We focus on constraint sets that include both positive semi-defi…
New method speeds up training of large kernel models.
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on…
Improved convergence for nonconvex optimization with dependent data.
In many online learning problems the computational bottleneck for gradient-based methods is the projection operation. For this reason, in many problems the most efficient algorithms are based on the Frank-Wolfe method, which replaces projections by linear optimization. In the general case, however, online projection-fr…
In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…
The Straight-Through (ST) estimator is a widely used technique for back-propagating gradients through discrete random variables. However, this effective method lacks theoretical justification. In this paper, we show that ST can be interpreted as the simulation of the projected Wasserstein gradient flow (pWGF). Based on…
Improves deep learning models by blending gradients from training loss and auxiliary objective.
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex sets. We propose a class of algorithms that perform both stochastic gradient desce…
Proof of convergence for multi-objective optimization using inverse reinforcement learning.
In many applications of supervised learning, multiple classification or regression outputs have to be predicted jointly. We consider several extensions of gradient boosting to address such problems. We first propose a straightforward adaptation of gradient boosting exploiting multiple output regression trees as base le…
Faster reconstruction of compressed signals using conditional GAN and NPGD.
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent…
A new algorithm estimates sparse gradients on graphs with improved risk bounds.
Zeroth-order optimization methods lack inherent privacy guarantees.
AOPU stabilizes NN training by approximating natural gradient, improving stability and convergence.
Stochastic approximation algorithms show exponential progress bounds.