Study on Santaló point for convex bodies in normed spaces.
problem Exploring Santaló point for convex bodies in normed spaces.
method Existence and uniqueness proof for C 1 C^1 C 1 norms, dual Santaló point for smooth curved unit balls. result Existence and uniqueness of Santaló point for convex bodies in normed spaces.
New method improves signal estimation by convexifying ℓ 0 \ell_0 ℓ 0 -norm constraints.
problem Signal estimation with sparsity and smoothness priors.
method Iterative convex conic quadratic relaxations exploiting ℓ 0 \ell_0 ℓ 0 -norm and smoothness terms. result Significantly better estimators than ℓ 1 \ell_1 ℓ 1 -norm approaches and interpretable parameters. Generalizes smoothness conditions for optimization methods.
problem Optimization under non-uniform smoothness conditions.
method Develops a new analysis technique for bounding gradients.
result Obtains convergence rates for gradient descent and Nesterov's method.
Unified convex surrogates improve matrix approximation efficiency.
problem Improving rank approximation for matrices with non-convex Schatten- p p p norms. method Developed convex surrogates for Schatten- p p p norms, extending to multiple factor matrices. result Convex and smooth factor matrix norms for any p > 0 p>0 p > 0 . Paper introduces a new G ⋆ G^\star G ⋆ regret measure for online convex optimization with smooth losses.
problem Online convex optimization with smooth losses.
method Introduces a new G ⋆ G^\star G ⋆ regret measure that depends on the cumulative squared gradient norm. result The G ⋆ G^\star G ⋆ regret can be arbitrarily sharper than existing measures when losses have vanishing curvature. New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
problem Non-Euclidean norms in binary classification problems.
method Black-box reduction method using uniformly convex regularizers.
result Achieves optimal statistical risk bounds on excess risk for non-Euclidean norms.
New algorithms reduce regret for convex bandits with small comparator norms.
problem Optimizing in bandit convex optimization with varying comparator norms.
method Developed algorithms using techniques from full-information setting and new gradient estimators.
result Regret bounds are small when comparator norm is small.
New algorithms optimize convex functions with high-order derivatives.
problem Optimizing convex functions with high-order derivatives under various norms.
method Developed a non-Euclidean inexact accelerated proximal point method using an inexact uniformly convex regularizer.
result Showed nearly optimal algorithms for high dimensions in the black-box oracle model for ℓ p \ell_p ℓ p -settings and all q ≥ 1 q \geq 1 q ≥ 1 . We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cas…
Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone K K K , a norm ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ and a smooth convex function f f f , we want either 1) to minimize the norm over the intersection of the cone and a level set of f f f , or 2) to minimize over the…
Unified scalable equivalent formulations for Schatten quasi-norms improve efficiency.
problem Efficiently solving Schatten quasi-norm minimization problems for large-scale matrices.
method Proved equivalence between Schatten-p quasi-norm and product/sum of Schatten-p1 and p2 norms of factor matrices.
result Transformed SQNM problems into simpler, more efficient algorithms for p>1/2.
New curvature measures characterize non-convex Wulff shapes in normed spaces.
problem Characterizing non-convex sets with curvature measures.
method Extending curvature measures to non-convex and non-smooth sets in normed spaces.
result Finite unions of disjoint Wulff shapes are the only sets with proportional curvature measures.
Improved Frank-Wolfe algorithm solves convex trace-norm ball problems.
problem Optimizing convex functions over trace-norm balls.
method Rank-k variant of Frank-Wolfe algorithm using top-k singular-vector computation.
result Linear convergence rate for smooth and strongly convex objectives with rank-limited solutions.
New guarantees for Group LASSO in sparse convex optimization.
problem Sparse convex optimization with vector-valued features.
method Group LASSO regularization and analysis of gradient norms.
result Group LASSO selects the same features as Orthogonal Matching Pursuit.
SGD achieves a O ( ε − 4 ) O(ε^{-4}) O ( ε − 4 ) bound for minimizing gradient norm of smooth functions.
problem Finding stationary points with SGD for gradient norm minimization.
method Stochastic Gradient Descent (SGD) for smooth, possibly nonconvex functions.
result The O ( ε − 4 ) O(ε^{-4}) O ( ε − 4 ) bound for gradient norm minimization cannot be improved upon. New algorithm achieves optimal privacy and efficiency in non-Euclidean convex optimization.
problem Optimizing convex functions while maintaining privacy in non-Euclidean settings.
method Developed a linear-time algorithm for ℓ p \ell_p ℓ p -setups, leveraging geometric properties. result Optimal excess risk achieved in linear time for 1 < p ≤ 2 1 < p \leq 2 1 < p ≤ 2 . New method accelerates steepest descent for convex optimization.
problem Achieving acceleration for general ℓ p \ell_p ℓ p smooth functions. method Primal-dual iterate sequences with differing norms, implicitly determined interpolation parameter.
result Improves iteration complexity to O ( d 1 − 2 p ) O(d^{1-\frac{2}{p}}) O ( d 1 − p 2 ) for ℓ p \ell_p ℓ p norm smooth problems. The paper extends the convolution operator to non-smooth valuations using geometric inequalities.
problem Extending the convolution operator to non-smooth valuations.
method Using geometric inequalities derived from optimal transport methods.
result Constructing a continuous extension of the convolution operator on smooth valuations to non-smooth valuations.
The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the l2,1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assum…
Proposes a filtering method for cluster analysis using ℓ 0 \ell_0 ℓ 0 -norm regularization.
problem Improving cluster analysis by filtering data.
method Minimizes a least squares function with a weighted ℓ 0 \ell_0 ℓ 0 -norm penalty, approximated by smooth non-convex functions. result The proposed method can enhance existing clustering techniques.
Study examines convexity properties of harmonic functions on evolving hypersurfaces.
problem Investigate convexity of harmonic functions on evolving hypersurfaces.
method Consider compact level sets of smooth regular functions, derive a differential inequality for L 2 L^{2} L 2 -norms of harmonic functions. result Obtain a new differential inequality for L 2 L^{2} L 2 -norms of harmonic functions over evolving hypersurfaces. Study smooths Finsler structures on Lie groups, proving extremal convergence.
problem Smooth left-invariant strongly convex C 0 C^0 C 0 -Finsler structures on Lie groups. method Introduce mollifier smoothing, study extremals using Pontryagin maximum principle.
result Pontryagin extremals on smoothed Finsler structures converge uniformly to those on original structure.
The paper examines curvature-dimension bounds on sub-Finsler Heisenberg groups.
problem Investigating synthetic curvature-dimension bounds in sub-Finsler Heisenberg groups.
method Study of measure contraction property (MCP) and curvature-dimension condition (CD).
result Sub-Finsler Heisenberg groups do not satisfy MCP or CD for any parameters.
New algorithm finds critical points in non-convex optimization with heavy-tailed gradients.
problem Non-convex stochastic optimization with heavy-tailed gradient estimates.
method Gradient clipping, momentum, and normalized gradient descent.
result High-probability convergence to critical points with best-known rates.
AdaGrad-Norm achieves optimal convergence rates for non-convex objectives without tuning.
problem Optimal convergence rates for non-convex, smooth objectives with adaptive step sizes.
method Adaptive SGD (AdaGrad-Norm) with self-tuning step sizes, analyzing under unbounded gradients and affine variance scaling.
result AdaGrad-Norm achieves order optimal convergence rate of $\mathcal{O}\left(\frac{\mathrm{poly}\log(T)}{\sqrt{T}}
ight)$ under optimal assumptions.
Universal algorithm for variational inequalities adapts to smoothness and noise.
problem Variational inequalities from monotone operators, including convex minimization and saddle-point problems.
method Mirror-Prox algorithm with adaptive step-size.
result Achieves optimal rates for smooth/non-smooth, noisy/noiseless settings without prior knowledge.
Kernel regression predicts graph signals in noisy environments.
problem Predicting smooth graph signals in the presence of sparse noise.
method Kernel regression with ℓ 1 \ell_1 ℓ 1 -norm and ℓ 2 \ell_2 ℓ 2 -norm optimization using IRLS. result Efficacy demonstrated on real-world temperature data.
Improved algorithms solve ℓ p \ell_p ℓ p -norm regression problems efficiently.
problem Efficiently solving ℓ p \ell_p ℓ p -norm regression problems for p ∈ ( 1 , 2 ) ∪ ( 2 , ∞ ) p \in (1,2) \cup (2,\infty) p ∈ ( 1 , 2 ) ∪ ( 2 , ∞ ) . method Iterative refinement scheme using smoothed ℓ p \ell_p ℓ p -norms to improve solutions. result Solves ℓ p \ell_p ℓ p -norm regression to 1 / e x t p o l y ( n ) 1 / ext{poly}(n) 1/ e x t p o l y ( n ) accuracy in i l d e O p ( m 1 3 ) ilde{O}_p(m^{\frac{1}{3}}) i l d e O p ( m 3 1 ) iterations. The curvature-dimension condition fails in sub-Finsler geometry, extending previous results in sub-Riemannian geometry.
problem The failure of the curvature-dimension condition in sub-Finsler geometry.
method Non-trivial adaptation of Juillet's work, introduction of new tools and ideas.
result The C D ( K , N ) \mathsf{CD}(K,N) CD ( K , N ) condition does not hold in sub-Finsler geometry for various norms and measures. Lower bound shows no acceleration for specific convex optimization class.
problem Proving lower bounds for convergence rates of convex optimization methods.
method Proving Ω ( L / T ) Ω(L/T) Ω ( L / T ) lower bound for minimization of convex and L L L -smooth functions relative to negative entropy. result Mirror descent is optimal up to a logarithmic factor in the class of functions considered.
New tensor recovery method improves efficiency under strict complementarity.
problem Efficiently recovering low-rank tensors using tensor nuclear norm.
method Developed strict complementarity condition for tensor nuclear norm ball and applied to gradient methods.
result Standard gradient methods achieve linear convergence and nearly linear runtime under strict complementarity.
Geodesics in Kähler metrics connect metrics with constant scalar curvature.
problem Deriving geodesics for relatively Kähler metrics on fibrations.
method Deriving geodesic equation, proving uniqueness, convexity of log-norm functional.
result Fibrations with optimal symplectic connections are polystable.
This paper analyzes convergence of RMSProp and Adam in non-convex optimization with tight complexity bounds.
problem Analyzing convergence of RMSProp and Adam in non-convex optimization with relaxed assumptions.
method Developed new convergence analyses for RMSProp and Adam, considering adaptive learning rates and affine noise variance.
result RMSProp and Adam converge to ε-stationary points with iteration complexities of O(ε^(-4)) under proper hyperparameters.
The paper introduces regularized OT to create sparse transportation plans.
problem Lack of sparsity in entropic regularization of OT.
method Regularizing primal and dual OT formulations with strongly convex terms, leading to sparse transportation plans.
result Regularized OT can lead to sparser transportation plans than entropic regularization.
The study proves a localized ellipsoid characterization for convex bodies and applies it to Finsler surfaces.
problem Characterizing convex bodies and their sections by planes.
method Localized ellipsoid characterization applied to Finsler surfaces in normed spaces.
result In certain cases, the intrinsic metric of a Finsler surface imposes restrictions on its extrinsic geometry.
The paper provides approximation guarantees for neural networks trained with gradient flow.
problem Approximating neural networks trained with gradient flow in continuous L 2 ( S d − 1 ) L_2(\mathbb{S}^{d-1}) L 2 ( S d − 1 ) -norm. method NTK argument for non-convex second but last layer, under-parametrized regime.
result Gradient flow convergence guarantees for neural networks under Sobolev smoothness assumptions.
SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.
problem Minimizing finite sums of convex and Lipschitz functions.
method Incorporates variance reduction techniques into SignSGD.
result Achieves convergence rates of O ( 1 / T ) \mathcal{O}(1 / \sqrt{T}) O ( 1/ T ) for expected norm of the gradient and O ( 1 / T ) \mathcal{O}(1/T) O ( 1/ T ) for smooth convex functions. The paper relaxes assumptions for analyzing stochastic optimization algorithms.
problem Analyzing the convergence of stochastic gradient algorithms under weaker variance assumptions.
method Building on and extending a connection to the Halpern iteration, the paper analyzes algorithms for convex nonsmooth optimization and min-max problems.
result Rates for optimality measures are obtained without requiring boundedness of the feasible set for problems beyond simple constrained optimization.
Step decay schedules improve convergence in non-convex optimization.
problem Improving convergence in non-convex optimization problems.
method Analyzing convergence rates of step decay schedules in non-convex, convex, and strongly convex problems.
result Step decay schedules achieve O ( ln T / T ) \mathcal{O}(\ln T/\sqrt{T}) O ( ln T / T ) convergence rates in various optimization scenarios. New algorithms achieve optimal performance in online convex optimization with strong convexity and squared ℓ 2 \ell_2 ℓ 2 norms.
problem Optimal performance in online convex optimization with specific cost structures.
method Proposed and analyzed new algorithms (G-OBD, R-OBD) with theoretical guarantees.
result G-OBD and R-OBD achieve optimal competitive ratios of O ( m − 1 / 2 ) O(m^{-1/2}) O ( m − 1/2 ) under specific conditions. Convex norms improve coupled matrix and tensor completion.
problem Efficiently complete coupled matrices and tensors with shared information.
method Proposed convex norms and completion algorithm.
result Excess risk bounds show improved performance compared to uncoupled norms.
New SPS variant improves non-smooth optimization without small gradients.
problem Improving non-smooth optimization without small gradients.
method Safeguarded Stochastic Polyak Step Size (SPS s a f e _{safe} s a f e ) for non-smooth optimization. result Rigorous convergence guarantees for non-smooth convex optimization without strong assumptions.
The paper proves conjectures about Minkowski norms with specific symmetry groups.
problem Proving conjectures about Minkowski norms with certain symmetries.
method Analyzing isometries of the Hessian metric for Minkowski norms invariant under S O ( k ) i m e s S O ( n − k ) SO(k) imes SO(n-k) S O ( k ) im es S O ( n − k ) . result Proves Laugwitz and Landsberg Unicorn conjectures for Minkowski norms with the specified symmetry.
Lie groups with bi-invariant distance are products of abelian and compact groups.
problem Characterizing Lie groups with bi-invariant distances.
method Analyzing the structure of Lie groups and introducing a Finsler norm.
result The sectional curvature of bi-invariant distances is non-negative and vanishes only for abelian subalgebras.
Lower bounds on queries needed for finding stationary points in non-convex optimization.
problem Finding ε ε ε -stationary points in non-convex stochastic optimization. method Proving lower bounds on the number of queries required by stochastic first-order methods.
result Lower bounds on the number of queries required to find ε ε ε -stationary points are tight and optimal. New bounds on adaptivity cost in stochastic optimization.
problem Understanding the cost of changing strategies in stochastic optimization.
method Proving impossibility results for adaptivity in non-smooth stochastic convex optimization.
result Lower bounds on the price of adaptivity for different levels of uncertainty.
In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible to adopt recent developments in saddle-point optimization, to achieve the optim…
Consider a convex polygon P in the plane, and denote by U a homothetical copy of the vector sum of P and (-P). Then the polygon U, as unit ball, induces a norm such that, with respect to this norm, P has constant Minkowskian width. We define notions like Minkowskian curvature, evolutes and involutes for polygons of con…