arXiv research

Recently, based on the idea of randomizing space theory, random convex analysis has been being developed in order to deal with the corresponding problems in random environments such as analysis of conditional convex risk measures and the related variational problems and optimization problems. Random convex analysis is …

Geodesic convexity generalizes the notion of (vector space) convexity to nonlinear metric spaces. But unlike convex optimization, geodesically convex (g-convex) optimization is much less developed. In this paper we contribute to the understanding of g-convex optimization by developing iteration complexity analysis for …

Generalizes smoothness conditions for optimization methods.

Unified analysis of stochastic gradient methods for convex and smooth optimization.

Prototypal analysis is introduced to overcome two shortcomings of archetypal analysis: its sensitivity to outliers and its non-locality, which reduces its applicability as a learning tool. Same as archetypal analysis, prototypal analysis finds prototypes through convex combination of the data points and approximates th…

Paper improves stability analysis of SGD for various loss functions and data distributions.

Classifies geodetically convex sets and functions on Heisenberg group.

New insights into using momentum for non-convex optimization.

We revisit the problem of robust principal component analysis with features acting as prior side information. To this aim, a novel, elegant, non-convex optimization approach is proposed to decompose a given observation matrix into a low-rank core and the corresponding sparse residual. Rigorous theoretical analysis of t…

We develop a convex relaxation method for analyzing neural network generalization.

Unified analysis improves SAM for non-convex optimization.

Novel analysis of neural networks using geometric algebra and convex optimization.

We consider the closely related problems of bandit convex optimization with two-point feedback, and zero-order stochastic convex optimization with two function evaluations per round. We provide a simple algorithm and analysis which is optimal for convex Lipschitz functions. This improves on \cite{dujww13}, which only p…

Study shows how to control jump-diffusion processes with stable feedback controls in reinforcement learning.

Bounds on chemical reaction network relaxation rates using convex analysis.

Set-functions appear in many areas of computer science and applied mathematics, such as machine learning, computer vision, operations research or electrical networks. Among these set-functions, submodular functions play an important role, similar to convex functions on vector spaces. In this tutorial, the theory of sub…

Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.

Paper introduces a continuous convexity measure for compact sets.

The paper finds new inequalities for convex polygons.

Study iterative regularization for linear models with convex bias, improving robust sparse recovery.

This paper delivers improved theoretical guarantees for the convex programming approach in low-rank matrix estimation, in the presence of (1) random noise, (2) gross sparse outliers, and (3) missing data. This problem, often dubbed as robust principal component analysis (robust PCA), finds applications in various domai…

Paper constructs $L^2$ estimates for flat vector bundles and generalizes Prékopa's theorem.

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

New analysis shows D-SGD can generalize well regardless of graph connectivity.

Develops a Riemannian archetypal analysis for interpretable non-linear data.

The subdifferential of convex functions of the singular spectrum of real matrices has been widely studied in matrix analysis, optimization and automatic control theory. Convex analysis and optimization over spaces of tensors is now gaining much interest due to its potential applications to signal processing, statistics…

Nonnegative matrix factorization (NMF) is a widely used linear dimensionality reduction technique for nonnegative data. NMF requires that each data point is approximated by a convex combination of basis elements. Archetypal analysis (AA), also referred to as convex NMF, is a well-known NMF variant imposing that the bas…

We consider the convex-concave saddle point problem $\min_{x}\max_{y} f(x)+y^\top A x-g(y)$ where $f$ is smooth and convex and $g$ is smooth and strongly convex. We prove that if the coupling matrix $A$ has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if $f$ is not stron…

Unified analysis of multi-attribute graph learning with non-convex penalties.

New binary AA methods improve on existing techniques.

The study proves curvature rigidity for convex polytopes.

ProxSkip achieves linear speedup in distributed non-convex optimization.

Study AFPP of unions of convex digital disks in 2D.

Optimal algorithms for online convex optimization with random order.

The paper proves consistency of archetypal analysis for multivariate data.

A new algorithm speeds up convex clustering.

This work shows neural networks can solve non-convex constraints problems.

In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledg…

Paper proposes distributed sparse multicategory discriminant analysis for classification.

The paper develops a new approach to conditional risk measures using modular convex analysis.

Improved dynamic regret analysis for strongly convex and smooth functions.

Unified analysis of Federated Averaging and Nesterov FedAvg for linear speedup.

This paper surveys recent theoretical advances in convex optimization approaches for community detection. We introduce some important theoretical techniques and results for establishing the consistency of convex community detection under various statistical models. In particular, we discuss the basic techniques based o…

New bounds derived for machine learning algorithms using convex functions.

In this work we develop a new algorithm for regularized empirical risk minimization. Our method extends recent techniques of Shalev-Shwartz [02/2015], which enable a dual-free analysis of SDCA, to arbitrary mini-batching schemes. Moreover, our method is able to better utilize the information in the data defining the ER…

AA extracts archetypes from data for clear feature extraction.

Unified framework for analyzing neural networks trained by gradient descent.

Tensor completion and robust principal component analysis have been widely used in machine learning while the key problem relies on the minimization of a tensor rank that is very challenging. A common way to tackle this difficulty is to approximate the tensor rank with the $\ell_1-$norm of singular values based on its …