arXiv research

Proof of Gaussian ML estimator consistency in linear auto-regressive models.

Estimates CATEs using high-dimensional linear regression models.

We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, e…

Paper introduces semi-supervised linear extremile regression for high-dimensional data.

Paper introduces new regression methods for consistent estimation of biophysical parameters.

The standard linear and logistic regression models assume that the response variables are independent, but share the same linear relationship to their corresponding vectors of covariates. The assumption that the response variables are independent is, however, too strong. In many applications, these responses are collec…

Kernel ridge regression imputation with consistent variance estimation for handling missing data.

Proposes a novel network-based neighborhood regression for biological systems.

The paper examines the consistency of Lasso regression applied to signature analysis of time series data.

Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for a mixture of linear regressions, a simple instance of a discriminative latent-variable mode…

High signal to noise ratio (SNR) consistency of model selection criteria in linear regression models has attracted a lot of attention recently. However, most of the existing literature on high SNR consistency deals with model order selection. Further, the limited literature available on the high SNR consistency of subs…

This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…

Develops privacy-preserving methods for longitudinal linear regression.

Analysis of cross-validation for early-stopped gradient descent in high-dimensional regression.

New robust regression method works with fewer data points than previous methods.

Study improves statistical inference for CATEs using Lasso and DML.

New findings show some designs can't be robustly recovered in linear regression.

A non linear regression approach which consists of a specific regression model incorporating a latent process, allowing various polynomial regression models to be activated preferentially and smoothly, is introduced in this paper. The model parameters are estimated by maximum likelihood performed via a dedicated expeca…

Scaling laws in linear regression explain model performance improvements with size and data.

Corrects mismatch in consistency of nuisance estimators for doubly robust methods.

Proposes a robust estimator for RD designs.

Bayesian method improves predictions in overparameterized nonlinear regression.

Unified framework for fair regression under demographic parity.

We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…

High-dimensional regression models struggle with resampling methods.

We study the residual bootstrap (RB) method in the context of high-dimensional linear regression. Specifically, we analyze the distributional approximation of linear contrasts $c^{\top} (\hatβ_ρ-β)$, where $\hatβ_ρ$ is a ridge-regression estimator. When regression coefficients are estimated via least squares, classical…

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

In this paper we explore different regression models based on Clusterwise Linear Regression (CLR). CLR aims to find the partition of the data into $k$ clusters, such that linear regressions fitted to each of the clusters minimize overall mean squared error on the whole data. The main obstacle preventing to use found re…

Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.

We demonstrate that the primal-dual witness proof method may be used to establish variable selection consistency and $\ell_\infty$-bounds for sparse regression problems, even when the loss function and/or regularizer are nonconvex. Using this method, we derive two theorems concerning support recovery and $\ell_\infty$-…

A new method, Residual-Permuted Sums, improves confidence region construction for linear regression models.

We show that the two-stage adaptive Lasso procedure (Zou, 2006) is consistent for high-dimensional model selection in linear and Gaussian graphical models. Our conditions for consistency cover more general situations than those accomplished in previous work: we prove that restricted eigenvalue conditions (Bickel et al.…

We consider the least-square linear regression problem with regularization by the l1-norm, a problem usually referred to as the Lasso. In this paper, we present a detailed asymptotic analysis of model consistency of the Lasso. For various decays of the regularization parameter, we compute asymptotic equivalents of the …

New method for high-dimensional linear regression using empirical Bayes.

The SPS method constructs confidence regions for true parameters with optimal sample complexity.

Scaled sparse linear regression jointly estimates the regression coefficients and noise level in a linear model. It chooses an equilibrium with a sparse regression method by iteratively estimating the noise level via the mean residual square and scaling the penalty in proportion to the estimated noise level. The iterat…

New bounds for transfer learning in linear models, improving generalization.

Paper uses SGD for solving linear inverse problems, improving empirical performance.

New neural network models for complex functional data analysis.

Study improves $H$-consistency bounds for regression analysis.

Reducing ICD-10 code granularity improves cost model accuracy and stability.

Recovering the support of sparse vectors in underdetermined linear regression models, \textit{aka}, compressive sensing is important in many signal processing applications. High SNR consistency (HSC), i.e., the ability of a support recovery technique to correctly identify the support with increasing signal to noise rat…

We study the problem of robust time series analysis under the standard auto-regressive (AR) time series model in the presence of arbitrary outliers. We devise an efficient hard thresholding based algorithm which can obtain a consistent estimate of the optimal AR model despite a large fraction of the time series points …

Method estimates parameters of complex nonlinear systems.

Inflating the minimum norm interpolator improves linear regression generalization error.

The paper revisits discriminative vs. generative classifiers, showing naive Bayes requires fewer samples.

The paper provides risk bounds for learning many response functions using linear regression.

This manuscript studies statistical properties of linear classifiers obtained through minimization of an unregularized convex risk over a finite sample. Although the results are explicitly finite-dimensional, inputs may be passed through feature maps; in this way, in addition to treating the consistency of logistic reg…