arXiv research

We find a convex model for traditional nonlinear regression under L2 loss.

Spectrahedral regression fits convex functions via a non-convex optimization problem.

New method uses DC functions for piecewise linear regression.

The study establishes risk bounds for distributional regression estimators.

Faster algorithms solve convex function learning problems.

We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that…

New algorithms solve large-scale convex regression problems.

Paper introduces $\ell$-DER for regression tasks using morphological operators and convex-concave procedure.

Convex regression is a promising area for bridging statistical estimation and deterministic convex optimization. New piecewise linear convex regression methods are fast and scalable, but can have instability when used to approximate constraints or objective functions for optimization. Ensemble methods, like bagging, sm…

ICCNLS models complex relationships as convex and concave components.

We propose a computationally efficient estimator, formulated as a convex program, for a broad class of non-linear regression problems that involve difference of convex (DC) non-linearities. The proposed method can be viewed as a significant extension of the "anchored regression" method formulated and analyzed in [10] f…

Optimal convex loss function improves regression coefficient estimation.

Optimizes binary regression models with gradient ascent-descent methods.

In this work we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the $\ell_0$ pseudo norm is able to better induce sparsity than the commonly used $\ell_1$ norm. For a cl…

Short proof shows how ridge regression works with random data.

SAGA is a fast incremental gradient method on the finite sum problem and its effectiveness has been tested on a vast of applications. In this paper, we analyze SAGA on a class of non-strongly convex and non-convex statistical problem such as Lasso, group Lasso, Logistic regression with $\ell_1$ regularization, linear r…

Least Squares Estimators are suboptimal for 5D convex functions.

Efficiently finds sparse solutions to max-plus equations for convex regression.

Max-linear regression problem solved with convex programming.

CNR uses convex optimization to estimate conditional distributions.

In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…

A new method for nonparametric regression using mesh-based solutions.

A new algorithm solves signed Fréchet regression on manifolds with bounded curvature.

New estimator avoids overfitting in convex regression.

Active-set algorithm improves Cox regression for shape-restricted covariates.

In kernel methods, the kernels are often required to be positive definite, which restricts the use of many indefinite kernels. To consider those non-positive definite kernels, in this paper, we aim to build an indefinite kernel learning framework for kernel logistic regression. The proposed indefinite kernel logistic r…

A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.

Support vector regression (SVR) is one of the most popular machine learning algorithms aiming to generate the optimal regression curve through maximizing the minimal margin of selected training samples, i.e., support vectors. Recent researchers reveal that maximizing the margin distribution of whole training dataset ra…

Sparse regression models are increasingly prevalent due to their ease of interpretability and superior out-of-sample performance. However, the exact model of sparse regression with an $\ell_0$ constraint restricting the support of the estimators is a challenging (\NP-hard) non-convex optimization problem. In this paper…

Almost all local minima in neural networks are strongly convex.

Develops a new fairness learning approach for multi-task regression models.

This paper introduces Jensen, an easily extensible and scalable toolkit for production-level machine learning and convex optimization. Jensen implements a framework of convex (or loss) functions, convex optimization algorithms (including Gradient Descent, L-BFGS, Stochastic Gradient Descent, Conjugate Gradient, etc.), …

Global convergence for robust regression problems via IRLS with enhancements.

In this paper, we consider stochastic dual coordinate (SDCA) {\em without} strongly convex assumption or convex assumption. We show that SDCA converges linearly under mild conditions termed restricted strong convexity. This covers a wide array of popular statistical models including Lasso, group Lasso, and logistic reg…

Improved SGD for non-strongly-convex regression with faster convergence.

To construct flexible nonlinear predictive distributions, the paper introduces a family of softplus function based regression models that convolve, stack, or combine both operations by convolving countably infinite stacked gamma distributions, whose scales depend on the covariates. Generalizing logistic regression that…

Efficient algorithms speed up adversarial training for linear models.

Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning and optimization theory. A popular approach used to tackle non-convex problems is to use convex relaxation techniques to find a convex surrogate for the problem. Unfortunately, convex relaxat…

Brenier isotonic regression extends multi-output isotonic regression using optimal transport.

We consider new formulations and methods for sparse quantile regression in the high-dimensional setting. Quantile regression plays an important role in many applications, including outlier-robust exploratory analysis in gene selection. In addition, the sparsity consideration in quantile regression enables the explorati…

Estimates variance function using aggregation methods in regression models.

We introduce a convex approach for mixed linear regression over $d$ features. This approach is a second-order cone program, based on L1 minimization, which assigns an estimate regression coefficient in $\mathbb{R}^{d}$ for each data point. These estimates can then be clustered using, for example, $k$-means. For problem…

Paper proposes a working set algorithm for non-convex sparse regression with provable convergence.

Paper introduces methods to create fair and accurate regression models.

SA algorithms control dynamic regret in non-stationary settings with strong convexity or exp-concavity.

Shape-constrained convex regression problem deals with fitting a convex function to the observed data, where additional constraints are imposed, such as component-wise monotonicity and uniform Lipschitz continuity. This paper provides a comprehensive mechanism for computing the least squares estimator of a multivariate…

New methods for sketching non-PSD matrices improve regression and optimization tasks.

New algorithm for robust high-dimensional linear regression is both fast and statistically optimal.