arXiv research

A new method solves large-scale sparse group square-root Lasso problems efficiently.

Support vector machines (SVMs) are successful modeling and prediction tools with a variety of applications. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the numerical difficulties of the SVMs will become severe with the increase of t…

In this work, we present a globalized stochastic semismooth Newton method for solving stochastic optimization problems involving smooth nonconvex and nonsmooth convex terms in the objective function. We assume that only noisy gradient and Hessian information of the smooth part of the objective function is available via…

A new method solves distributed optimization problems over networks.

Develops a new SPP algorithm with variance reduction for weakly convex optimization.

We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneousl…

In this paper, we consider high-dimensional nonconvex square-root-loss regression problems and introduce a proximal majorization-minimization (PMM) algorithm for these problems. Our key idea for making the proposed PMM to be efficient is to develop a sparse semismooth Newton method to solve the corresponding subproblem…

A new algorithm solves the metric nearness problem efficiently.

We propose a semismooth Newton algorithm for pathwise optimization (SNAP) for the LASSO and Enet in sparse, high-dimensional linear regression. SNAP is derived from a suitable formulation of the KKT conditions based on Newton derivatives. It solves the semismooth KKT equations efficiently by actively and continuously s…

We focus on solving the clustered lasso problem, which is a least squares problem with the $\ell_1$-type penalties imposed on both the coefficients and their pairwise differences to learn the group structure of the regression parameters. Here we first reformulate the clustered lasso regularizer as a weighted ordered-la…

Introduces PPMM algorithm for nonconvex robust regression problems.

Paper tackles multivariate shape-constrained convex regression problems.

Proposes a robust and sparse portfolio selection model to reduce estimation errors and transaction costs.

Many of the algorithms used to solve minimization problems with sparsity-inducing regularizers are generic in the sense that they do not take into account the sparsity of the solution in any particular way. However, algorithms known as semismooth Newton are able to take advantage of this sparsity to accelerate their co…

A new method for faster optimization on statistical manifolds.

Proposes a new robust expectile regression method for high-dimensional data.

Paper develops algorithms for sparse linear regression with generalized elastic net penalty.

This paper certifies cluster assignments from sum-of-norms clustering algorithms.

Efficiently estimates hub graphical models with structured sparsity.

Newton's method solves variational problems on manifolds.

Develops a new screening method called Newton screening for faster and more accurate sparse learning.

We develop a non-relativistic twistor theory, in which Newton--Cartan structures of Newtonian gravity correspond to complex three-manifolds with a four-parameter family of rational curves with normal bundle ${\mathcal O}\oplus{\mathcal O}(2)$. We show that the Newton--Cartan space-times are unstable under the general K…

Improves robustness of high-dimensional regression with rank objective and group lasso regularization.

The study explores various localized bases and their duals for scattered data approximation.

We propose an algorithmic framework for convex minimization problems of a composite function with two terms: a self-concordant function and a possibly nonsmooth regularization term. Our method is a new proximal Newton algorithm that features a local quadratic convergence rate. As a specific instance of our framework, w…

We propose a communication- and computation-efficient distributed optimization algorithm using second-order information for solving empirical risk minimization (ERM) problems with a nonsmooth regularization term. Our algorithm is applicable to both the primal and the dual ERM problem. Current second-order and quasi-New…

We show that link Floer homology detects the Thurston norm of a link complement. As an application, we show that the Thurston polytope of an alternating link is dual to the Newton polytope of its multi-variable Alexander polynomial. To illustrate these techniques, we also compute the Thurston polytopes of several speci…

In many learning tasks, structural models usually lead to better interpretability and higher generalization performance. In recent years, however, the simple structural models such as lasso are frequently proved to be insufficient. Accordingly, there has been a lot of work on "superposition-structured" models where mul…

We present ADMM-Softmax, an alternating direction method of multipliers (ADMM) for solving multinomial logistic regression (MLR) problems. Our method is geared toward supervised classification tasks with many examples and features. It decouples the nonlinear optimization problem in MLR into three steps that can be solv…

Improved solver maintains positivity and accuracy across all time steps.

New method solves constrained stochastic optimization problems efficiently.

In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual subspace ascent. This is the first time momentum variants of several of these metho…

Clustering is a fundamental problem in unsupervised learning. Popular methods like K-means, may suffer from poor performance as they are prone to get stuck in its local minima. Recently, the sum-of-norms (SON) model (also known as the clustering path) has been proposed in Pelckmans et al. (2005), Lindsten et al. (2011)…

A new method for optimization in probability space using Newton's flows.

We describe stochastic Newton and stochastic quasi-Newton approaches to efficiently solve large linear least-squares problems where the very large data sets present a significant computational burden (e.g., the size may exceed computer memory or data are collected in real-time). In our proposed framework, stochasticity…

Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.

RNN operators solve Newton's equations with large timesteps for molecular dynamics.

We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing s…

New algorithm improves convergence of gradient boosting trees.

Boosting algorithms are frequently used in applied data science and in research. To date, the distinction between boosting with either gradient descent or second-order Newton updates is often not made in both applied and methodological research, and it is thus implicitly assumed that the difference is irrelevant. The g…

Newton's method tackles nonlinear mappings into vector bundles with connections and retractions.

Study uses Newton polytopes to distinguish Lagrangian fillings of Legendrian submanifolds.

EGMU optimizes portfolios using KL divergence, ensuring positive solutions.

Unified approach to Bayesian inference with guarantees on covariance matrices.

Paper proposes an online covariance estimator for sketched Newton methods.

New Q-Newton's method avoids saddle points and converges quadratically.

Using elementary ideas from Tropical Geometry, we assign a a tropical curve to every $q$-holonomic sequence of rational functions. In particular, we assign a tropical curve to every knot which is determined by the Jones polynomial of the knot and its parallels. The topical curve explains the relation between the AJ Con…

A new optimization method improves deep learning accuracy without hyper-parameter tuning.