arXiv research

Two new methods improve block-sparse signal recovery from noisy data.

Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…

Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconve…

Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a linear loss to improve the effectiveness from existing methods that utilize hard constraints/hinge …

Two new algorithms solve nonconvex-strongly concave problems efficiently.

We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…

One-bit measurements widely exist in the real world, and they can be used to recover sparse signals. This task is known as the problem of learning halfspaces in learning theory and one-bit compressive sensing (1bit-CS) in signal processing. In this paper, we propose novel algorithms based on both convex and nonconvex s…

A new method solves a complex optimization problem efficiently.

Paper proposes an algorithm to solve complex minimax problems efficiently.

New algorithm AG-OG optimizes separable convex-concave problems efficiently.

In this paper we will provide a representation of the penalty term of general dynamic concave utilities (hence of dynamic convex risk measures) by applying the theory of g-expectations.

New algorithms solve nonconvex-concave minimax problems without parameter knowledge.

New algorithms solve complex minimax problems without needing derivatives.

Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.

Two important goals of high-dimensional modeling are prediction and variable selection. In this article, we consider regularization with combined $L_1$ and concave penalties, and study the sampling properties of the global optimum of the suggested method in ultra-high dimensional settings. The $L_1$-penalty provides th…

Paper solves minimax optimization gap with near-optimal algorithms.

New algorithm solves minimax games with linear constraints.

Paper improves risk bounds for nonconvex-strongly-concave minimax problems.

Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.

Study minimax risk of score estimation for log-concave distributions.

The paper analyzes how optimization algorithms affect the generalization of minimax models.

Paper improves algorithms for convex-concave minimax optimization problems.

A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.

The paper analyzes generalization bounds for NC-SC/NC-C stochastic minimax optimization.

A novel decentralized algorithm improves minimax optimization in federated learning.

This paper analyzes OGDA and EG methods for nonconvex minimax problems.

We consider nonconvex-concave minimax problems, $\min_{\mathbf{x}} \max_{\mathbf{y} \in \mathcal{Y}} f(\mathbf{x}, \mathbf{y})$, where $f$ is nonconvex in $\mathbf{x}$ but concave in $\mathbf{y}$ and $\mathcal{Y}$ is a convex and bounded set. One of the most popular algorithms for solving this problem is the celebrated…

SCOPE fuses categorical variable levels to estimate high-dimensional linear models.

Paper analyzes complexity of solving nonconvex-strongly-concave problems.

New algorithms solve complex minimax problems efficiently.

New method approximates sampling from smooth potential distributions using a vanishing penalty.

Sparse regression models are increasingly prevalent due to their ease of interpretability and superior out-of-sample performance. However, the exact model of sparse regression with an $\ell_0$ constraint restricting the support of the estimators is a challenging (\NP-hard) non-convex optimization problem. In this paper…

Fast accumulation of large amounts of complex data has created a need for more sophisticated statistical methodologies to discover interesting patterns and better extract information from these data. The large scale of the data often results in challenging high-dimensional estimation problems where only a minority of t…

In this paper we study nonconvex penalization using Bernstein functions. Since the Bernstein function is concave and nonsmooth at the origin, it can induce a class of nonconvex functions for high-dimensional sparse estimation problems. We derive a threshold function based on the Bernstein penalty and give its mathemati…

A new algorithm solves minimax problems without needing parameters.

New methods solve complex optimization problems without strong convexity assumptions.

Paper proposes SMO for solving bilevel optimization problems efficiently.

SREDA optimizes complex machine learning problems with fewer evaluations.

Current online learning methods suffer issues such as lower convergence rates and limited capability to select important features compared to their offline counterparts. In this paper, a novel framework for online learning based on running averages is proposed. Many popular offline regularized methods such as Lasso, El…

As surrogate functions of $L_0$-norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…

Paper tackles bilevel optimization problems using penalty methods.

The aim of this paper is to train an RBF neural network and select centers under concurrent faults. It is well known that fault tolerance is a very attractive property for neural networks. And center selection is an important procedure during the training process of an RBF neural network. In this paper, we devise two n…

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a uniq…

Optimizes algorithms for non-concave bandit problems.

Optimistic method adapted for faster convex-concave min-max problems.

Paper explores generalization of minimax learners, proposing a new metric.

In this paper, we consider the problem of recovering a sparse signal based on penalized least squares formulations. We develop a novel algorithm of primal-dual active set type for a class of nonconvex sparsity-promoting penalties, including $\ell^0$, bridge, smoothly clipped absolute deviation, capped $\ell^1$ and mini…

New algorithm solves nonconvex-convex minimax problems efficiently.