arXiv research

Proximal splitting methods solve rank-constrained convex problems locally.

The classical multi-set split feasibility problem seeks a point in the intersection of finitely many closed convex domain constraints, whose image under a linear mapping also lies in the intersection of finitely many closed convex range constraints. Split feasibility generalizes important inverse problems including con…

Within the unmanageably large class of nonconvex optimization, we consider the rich subclass of nonsmooth problems that have composite objectives---this already includes the extensively studied convex, composite objective problems as a special case. For this subclass, we introduce a powerful, new framework that permits…

Innovative method solves nonconvex optimization on manifolds.

This work simplifies proximal mapping for low-rank norms.

The OSCAR (octagonal selection and clustering algorithm for regression) regularizer consists of a L_1 norm plus a pair-wise L_inf norm (responsible for its grouping behavior) and was proposed to encourage group sparsity in scenarios where the groups are a priori unknown. The OSCAR regularizer has a non-trivial proximit…

New algorithm samples from log concave distributions efficiently.

A new method solves convex optimization problems on manifolds efficiently.

Unified view of accelerated and stochastic optimization methods.

Convex optimization is an essential tool for machine learning, as many of its problems can be formulated as minimization problems of specific objective functions. While there is a large variety of algorithms available to solve convex problems, we can argue that it becomes more and more important to focus on efficient, …

The three operator splitting scheme was recently proposed by [Davis and Yin, 2015] as a method to optimize composite objective functions with one convex smooth term and two convex (possibly non-smooth) terms for which we have access to their proximity operator. In this short note we provide an alternative proof for the…

In this paper we develop a statistical theory and an implementation of deep learning models. We show that an elegant variable splitting scheme for the alternating direction method of multipliers optimises a deep learning objective. We allow for non-smooth non-convex regularisation penalties to induce sparsity in parame…

A new method solves optimization problems over Stiefel manifold.

We apply the OSCAR (octagonal selection and clustering algorithms for regression) in recovering group-sparse matrices (two-dimensional---2D---arrays) from compressive measurements. We propose a 2D version of OSCAR (2OSCAR) consisting of the $\ell_1$ norm and the pair-wise $\ell_{\infty}$ norm, which is convex but non-d…

We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjective linear map $\cal M$, with the proximal mappings of $τP$, $τ> 0$, simple to compute. This problem i…

Method solves optimisation problems on non-Riemannian surfaces with bilateral curvature bounds.

New stochastic algorithm improves on existing PRSM methods.

The total variation (TV) penalty, as many other analysis-sparsity problems, does not lead to separable factors or a proximal operatorwith a closed-form expression, such as soft thresholding for the $\ell\_1$ penalty. As a result, in a variational formulation of an inverse problem or statisticallearning estimation, it l…

In this paper, we develop an approach to recursively estimate the quadratic risk for matrix recovery problems regularized with spectral functions. Toward this end, in the spirit of the SURE theory, a key step is to compute the (weak) derivative and divergence of a solution with respect to the observations. As such a so…

We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into $N$ subproblems, and utilizes an augmented Lagrangian b…

New algorithm for fast nonsmooth optimization with applications in image processing and machine learning.

New algorithm speeds up matrix learning with nonconvex regularizers.

We consider a class of learning problems regularized by a structured sparsity-inducing norm defined as the sum of l_2- or l_infinity-norms over groups of variables. Whereas much effort has been put in developing fast optimization techniques when the groups are disjoint or embedded in a hierarchy, we address here the ca…

Unified view connects CoCoA and ADMM for distributed ERM.

We propose a novel SPARsity and Clustering (SPARC) regularizer, which is a modified version of the previous octagonal shrinkage and clustering algorithm for regression (OSCAR), where, the proposed regularizer consists of a $K$-sparse constraint and a pair-wise $\ell_{\infty}$ norm restricted on the $K$ largest componen…

Develops Frank-Wolfe Augmented Lagrangian for convex optimization.

The paper proposes an efficient algorithm for solving Schatten-$p$ quasi-norm problems.

Spatially positioned neurons in neural networks mimic biological systems.

Structural equation models (SEMs) have been widely adopted for inference of causal interactions in complex networks. Recent examples include unveiling topologies of hidden causal networks over which processes such as spreading diseases, or rumors propagate. The appeal of SEMs in these settings stems from their simplici…

Method estimates sparse inverse covariance and partial correlation matrices efficiently.

In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…

Improves time series classification with forest proximities.

Improved sampling guarantees for weakly log-concave distributions.

Suppose N is a compressible boundary component of a compact orientable irreducible 3-manifold M and Q is an orientable properly embedded essential surface in M in which each component is incident to N and no component is a disk. Let VN and QN denote respectively the sets of vertices in the curve complex for N represent…

CFR-Pro enhances treatment effect estimation by incorporating local proximity.

Improved random forest proximities capture data geometry.

Introduces PPMM algorithm for nonconvex robust regression problems.

Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.

Extends RF proximities to all supervised distance-based machine learning contexts.

In this paper we introduce a micro-clustering strategy for Functional Boxplots. The aim is to summarize a set of streaming time series splitted in non overlapping windows. It is a two step strategy which performs at first, an on-line summarization by means of functional data structures, named Functional Boxplot micro-c…

Improved bounds for proximal gradient algorithms with computational errors.

Ensembled neural networks improve MRI image quality.

Paper extends theorem on covering spaces and Jordan curves.

Inertial proximal gradient algorithm shows monotonically decreasing values.

Proximal algorithms applied to current deformation into cycles.

New PnP algorithm converges with relaxed proximal gradient descent.

The paper connects a proximal method to stochastic filters and Bayes updates.

EPINE enhances network embedding by improving adjacency matrix-based high-order proximity.