arXiv research

Study introduces a new method for multiple parameter regularization in polynomial functional regression.

The study optimizes polynomial regression for learning under Gaussian distributions.

Paper proposes a robust LPR method using similarity kernels.

A mathematical framework connects neural networks and polynomial regression for better model understanding.

Shallow neural networks can represent polynomials efficiently.

This article describes a multivariate polynomial regression method where the uncertainty of the input parameters are approximated with Gaussian distributions, derived from the central limit theorem for large weighted sums, directly from the training sample. The estimated uncertainties can be propagated into the optimal…

This paper describes a novel method to approximate the polynomial coefficients of regression functions, with particular interest on multi-dimensional classification. The derivation is simple, and offers a fast, robust classification technique that is resistant to over-fitting.

This paper provides mathematical foundations for regression methods used in forward initial margin approximation.

New estimator adapts to various error distributions.

In this paper we develop the theory of parametric polynomial regression in Riemannian manifolds and Lie groups. We show application of Riemannian polynomial regression to shape analysis in Kendall shape space. Results are presented, showing the power of polynomial regression on the classic rat skull growth data of Book…

Paper tackles blind polynomial regression for unknown inputs.

New approach to adaptively select bandwidths in nonparametric regression.

Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.

We solve principal component regression (PCR), up to a multiplicative accuracy $1+γ$, by reducing the problem to $\tilde{O}(γ^{-1})$ black-box calls of ridge regression. Therefore, our algorithm does not require any explicit construction of the top principal components, and is suitable for large-scale PCR instances. In…

New model handles complex non-linear relationships with hidden graph structures.

Proposed by Donoho (1997), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the contex…

Efficiently finds sparse solutions to max-plus equations for convex regression.

Gaussian processes struggle with compositional functions, but deep Gaussian processes can outperform.

BPR matches NN accuracy in crop classification while being more transparent.

Volterra and polynomial regression models play a major role in nonlinear system identification and inference tasks. Exciting applications ranging from neuroscience to genome-wide association analysis build on these models with the additional requirement of parsimony. This requirement has high interpretative value, but …

This paper introduces a novel mixture model-based approach for simultaneous clustering and optimal segmentation of functional data which are curves presenting regime changes. The proposed model consists in a finite mixture of piecewise polynomial regression models. Each piecewise polynomial regression model is associat…

This paper studies how to sketch element-wise functions of low-rank matrices. Formally, given low-rank matrix A = [Aij] and scalar non-linear function f, we aim for finding an approximated low-rank representation of the (possibly high-rank) matrix [f(Aij)]. To this end, we propose an efficient sketching-based algorithm…

Develops nonparametric regression for non-smooth functions using fractional Laplacian.

Study shows RFRR's effectiveness with nearly orthogonal data in overparameterized settings.

New algorithm improves gradient-based ERM for smooth convex losses.

Polynomial-time RL algorithm for constant actions under linear Bellman completeness.

We introduce a new principle for model selection in regression and classification. Many regression models are controlled by some smoothness or flexibility or complexity parameter c, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polyno…

We show how to efficiently project a vector onto the top principal components of a matrix, without explicitly computing these components. Specifically, we introduce an iterative algorithm that provably computes the projection using few calls to any black-box routine for ridge regression. By avoiding explicit principal …

We present a regression technique for data-driven problems based on polynomial chaos expansion (PCE). PCE is a popular technique in the field of uncertainty quantification (UQ), where it is typically used to replace a runnable but expensive computational model subject to random inputs with an inexpensive-to-evaluate po…

We propose a method called ideal regression for approximating an arbitrary system of polynomial equations by a system of a particular type. Using techniques from approximate computational algebraic geometry, we show how we can solve ideal regression directly without resorting to numerical optimization. Ideal regression…

Local polynomial regression (Fan and Gijbels 1996) is an important class of methods for nonparametric density estimation and regression problems. However, straightforward implementation of local polynomial regression has quadratic time complexity which hinders its applicability in large-scale data analysis. In this pap…

Optimal AFs minimize RFR test error and sensitivity.

Robust learning mixtures of linear regressions improve robustness.

Gradient Descent with Projection learns low-degree polynomials efficiently.

GD outperforms ridge regression and SGD in linear regression problems.

Paper proves convergence rates for Gaussian kernel ridge regression.

New matching estimators correct bias in multivariate settings without smoothing parameters.

We present a novel method for exact hierarchical sparse polynomial regression. Our regressor is that degree $r$ polynomial which depends on at most $k$ inputs, counting at most $\ell$ monomial terms, which minimizes the sum of the squares of its prediction errors. The previous hierarchical sparse specification aligns w…

This article proposes a novel solution for stretchy polynomial regression learning. The solution comes in primal and dual closed-forms similar to that of ridge regression. Essentially, the proposed solution stretches the covariance computation via a power term thereby compresses or amplifies the estimation. Our experim…

Polynomial-time algorithm for list-decodable linear regression with batches.

Cryptocurrency prices predicted using LSTM, SVM, and polynomial regression.

Develops fast approximations for conditional Shapley values in linear and polynomial models.

Proposes a new regression method using $L_p$-norms for non-Gaussian noise.

Estimates functions on unknown manifolds using multiscale regression.

Efficiently estimates prediction error in regression with Gaussian covariates under privacy constraints.

TensorSketch is an oblivious linear sketch introduced in Pagh'13 and later used in Pham, Pagh'13 in the context of SVMs for polynomial kernels. It was shown in Avron, Nguyen, Woodruff'14 that TensorSketch provides a subspace embedding, and therefore can be used for canonical correlation analysis, low rank approximation…

We give the first polynomial-time algorithm for performing linear or polynomial regression resilient to adversarial corruptions in both examples and labels. Given a sufficiently large (polynomial-size) training set drawn i.i.d. from distribution D and subsequently corrupted on some fraction of points, our algorithm out…

This article considers algorithmic and statistical aspects of linear regression when the correspondence between the covariates and the responses is unknown. First, a fully polynomial-time approximation scheme is given for the natural least squares optimization problem in any constant dimension. Next, in an average-case…