arXiv research

New RGraSP framework for efficient non-convex optimization.

Non-convex sparsity-inducing penalties have recently received considerable attentions in sparse learning. Recent theoretical investigations have demonstrated their superiority over the convex counterparts in several sparse learning settings. However, solving the non-convex optimization problems associated with non-conv…

Multi-task feature learning aims to identity the shared features among tasks to improve generalization. It has been shown that by minimizing non-convex learning models, a better solution than the convex alternatives can be obtained. Therefore, a non-convex model based on the capped-$\ell_{1},\ell_{1}$ regularization wa…

Rank minimization (RM) is a wildly investigated task of finding solutions by exploiting low-rank structure of parameter matrices. Recently, solving RM problem by leveraging non-convex relaxations has received significant attention. It has been demonstrated by some theoretical and experimental work that non-convex relax…

Online SGD achieves consistent estimation in high-dimensional non-convex inference tasks.

Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.

Proposes a new method for logistic PCA to avoid overfitting.

Study accelerates optimization methods in non-convex problems, but doesn't improve the algorithm's performance.

CDP reduces point cloud dimensions by preserving detour-induced local non-convexity.

Dash selects dynamic pseudo labels from unlabeled data for semi-supervised learning.

Iterative Hard Thresholding (IHT) is a class of projected gradient descent methods for optimizing sparsity-constrained minimization models, with the best known efficiency and scalability in practice. As far as we know, the existing IHT-style methods are designed for sparse minimization in primal form. It remains open t…

We study --both in theory and practice-- the use of momentum motions in classic iterative hard thresholding (IHT) methods. By simply modifying plain IHT, we investigate its convergence behavior on convex optimization criteria with non-convex constraints, under standard assumptions. In diverse scenaria, we observe that …

Efficient distributed learning with Byzantine-resilient thresholding and error feedback.

New bounds found for optimizing non-convex functions with noisy data.

Given a nonconvex function that is an average of $n$ smooth functions, we design stochastic first-order methods to find its approximate stationary points. The convergence of our new methods depends on the smallest (negative) eigenvalue $-σ$ of the Hessian, a parameter that describes how nonconvex the function is. Our m…

Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.

New method for robust regression with near-optimal performance even with high corruption rates.

Paper proposes GPM for simultaneous community detection and group synchronization.

A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.

This paper is concerned with the hard thresholding operator which sets all but the $k$ largest absolute elements of a vector to zero. We establish a {\em tight} bound to quantitatively characterize the deviation of the thresholded solution from a given signal. Our theoretical result is universal in the sense that it ho…

Significant attention has been given to minimizing a penalized least squares criterion for estimating sparse solutions to large linear systems of equations. The penalty is responsible for inducing sparsity and the natural choice is the so-called $l_0$ norm. In this paper we develop a Momentumized Iterative Shrinkage Th…

Study on policy testing in MDPs with lower bounds and new algorithm.

Sparsity inducing regularization is an important part for learning over-complete visual representations. Despite the popularity of $\ell_1$ regularization, in this paper, we investigate the usage of non-convex regularizations in this problem. Our contribution consists of three parts. First, we propose the leaky capped …

New method solves graph-structured sparsity problems efficiently.

New algorithm robustly optimizes data streams with heavy-tailed or infinite variance samples.

Paper studies early-stopped mirror descent for noisy sparse phase retrieval.

Gradient-flow helps find good minima in complex models.

This work analyzes how overparameterization aids GANs in reaching global saddle points.

In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…

Continuous-time mirror descent solves sparse phase retrieval efficiently.

A new ODE model explains gradient descent dynamics near edge of stability.

NOODL solves dictionary and coefficient recovery for online learning.

Averaged Gradient Descent improves performance in rough landscapes.

The truncated singular value decomposition (SVD) of the measurement matrix is the optimal solution to the_representation_ problem of how to best approximate a noisy measurement matrix using a low-rank matrix. Here, we consider the (unobservable)_denoising_ problem of how to best approximate a low-rank signal matrix bur…

New method predicts positive samples with missing labels.

New method improves convergence of SGD for heavy-tailed noise.

Paper proposes a new algorithm combining gradient descent and Langevin dynamics.

This paper improves 3D pose recovery from 2D images using non-convex regularization.

The paper analyzes how good initial guesses affect the amount of data needed for low-rank matrix recovery.

New methods solve graph sparsity optimization problems faster.

A number of statistical estimation problems can be addressed by semidefinite programs (SDP). While SDPs are solvable in polynomial time using interior point methods, in practice generic SDP solvers do not scale well to high-dimensional problems. In order to cope with this problem, Burer and Monteiro proposed a non-conv…

Compact, non-convex curve flows are created.

The paper analyzes trace regression with low-rank matrices under various regularization methods.

New insights on robust learning under strong noise models.

Paper uses integer programming for non-convex boosting in classification.

Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…

First order methods can take extremely long to find global minima of non-convex functions.

New algorithm improves convergence for non-convex problems with boundaries.