arXiv research

Max-affine regression method converges linearly using GD and SGD.

New AMP algorithm estimates signals and latent variables in mixed regression models.

Paper presents ABGD for efficient piecewise linear regression in high dimensions.

Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.

Paper proposes Sp-GD for sparse max-affine regression with theoretical guarantees.

Max-affine regression refers to a model where the unknown regression function is modeled as a maximum of $k$ unknown affine functions for a fixed $k \geq 1$. This generalizes linear regression and (real) phase retrieval, and is closely related to convex regression. Working within a non-asymptotic framework, we study th…

New method uses DC functions for piecewise linear regression.

Spectrahedral regression fits convex functions via a non-convex optimization problem.

Tropical Geometry and Mathematical Morphology share the same max-plus and min-plus semiring arithmetic and matrix algebra. In this chapter we summarize some of their main ideas and common (geometric and algebraic) structure, generalize and extend both of them using weighted lattices and a max-$\star$ algebra with an ar…

Single-head attention approximates any function under various norms.

This paper shows neural networks can solve complex graph problems efficiently.

We build a rigorous bridge between deep networks (DNs) and approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs), which provide a powerful portal through which to view and analyze their inner workings. …

We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrat…

Generative networks are analyzed using spline operators to understand their properties and limitations.

Nonlinearity is crucial to the performance of a deep (neural) network (DN). To date there has been little progress understanding the menagerie of available nonlinearities, but recently progress has been made on understanding the rôle played by piecewise affine and convex nonlinearities like the ReLU and absolute value …

Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine…

Efficiently private regression for unbounded data.

This paper studies robust regression in the settings of Huber's $ε$-contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of $ε$-contamination models for various regression problems including nonpa…

This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…

Proposes FARM model combining latent factor and sparse regression.

Survey of SDR methods for high-dimensional regression and embedding.

This paper reviews SDR methods for multivariate response regression.

Paper introduces semi-supervised linear extremile regression for high-dimensional data.

Improves logistic regression performance with nonconvex programming.

Study improves $H$-consistency bounds for regression analysis.

Prevalidated ridge regression simplifies logistic regression for high-dimensional data.

Introduces a new model for mapping matrices to matrices, subsuming linear regression.

Meta-theorems validate fair regression algorithms under demographic parity constraints.

We analyzed optimism in linear and kernel regression models.

A new optimizer, MVO, improves nonlinear regression performance.

Brenier isotonic regression extends multi-output isotonic regression using optimal transport.

We simplify complex regression coefficients using linearization and feature comparison.

Unified framework for fair regression under demographic parity.

Locally adaptive interpretable regression improves linear regression's predictability.

Huber regression assessed for robustness in statistical learning.

Least Angle Regression is a promising technique for variable selection applications, offering a nice alternative to stepwise regression. It provides an explanation for the similar behavior of LASSO ($\ell_1$-penalized regression) and forward stagewise regression, and provides a fast implementation of both. The idea has…

We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.

Study uniform consistency in nonparametric mixture models and mixed regression.

We find a convex model for traditional nonlinear regression under L2 loss.

Collider regression improves predictive performance in regression tasks.

We describe dimensionally constrained symbolic regression which has been developed for mass measurement in certain classes of events in high-energy physics (HEP). With symbolic regression, we can derive equations that are well known in HEP. However, in problems with large number of variables, we find that by constraini…

New GP model estimates piecewise continuous functions.

PARC uses piecewise linear predictors for regression and classification.

This work analyzes Fréchet regression using comparison geometry, providing theoretical and practical insights.

Study on reducing dimensionality in high-dimensional regression with kernel methods and stability analysis.

Paper studies ensemble probabilistic regression trees for smooth approximations.

Modal regression estimates the local modes of the distribution of $Y$ given $X=x$, instead of the mean, as in the usual regression sense, and can hence reveal important structure missed by usual regression methods. We study a simple nonparametric method for modal regression, based on a kernel density estimate (KDE) of …

pGMM kernel outperforms ordinary ridge regression and RBF kernel ridge regression without tuning.