arXiv research

Improved algorithm for conditional linear regression with heterogeneous covariances.

Develops fast approximations for conditional Shapley values in linear and polynomial models.

Study high-dimensional Bayesian linear regression using variational inference.

Robust learning mixtures of linear regressions improve robustness.

GD outperforms ridge regression and SGD in linear regression problems.

Estimates CATEs using high-dimensional linear regression models.

Sharp risk bounds for early-stopping in Gaussian linear regression are derived.

Work in machine learning and statistics commonly focuses on building models that capture the vast majority of data, possibly ignoring a segment of the population as outliers. However, there does not often exist a good model on the whole dataset, so we seek to find a small subset where there exists a useful model. We ar…

New algorithm reduces sample complexity for sparse linear regression.

Study robust linear regression without distributional assumptions for heavy-tailed responses.

Paper proposes a new method for selective inference in robust regression.

Robustly estimates linear regression coefficients with adversarial and noisy data.

The paper analyzes the generalizability of linear autoencoders and multivariate linear regression.

We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a generalized linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean fu…

In this paper, non-linear time series models are used to describe volatility in financial time series data. To describe volatility, two of the non-linear time series are combined into form TAR (Threshold Auto-Regressive Model) with AARCH (Asymmetric Auto-Regressive Conditional Heteroskedasticity) error term and its par…

This paper analyzes divide-and-conquer estimators for functional linear regression without assuming target function in the RKHS.

Study improves error bounds for sparse regression with heavy-tailed covariates.

CNR uses convex optimization to estimate conditional distributions.

Proposes a new regression method using conditional GANs.

Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-…

Proposes an efficient shrinkage path for ridge regression.

Improved estimator reduces bias in statistical learning models.

The problem of subgroups is ubiquitous in scientific research (ex. disease heterogeneity, spatial distributions in ecology...), and piecewise regression is one way to deal with this phenomenon. Morse-Smale regression offers a way to partition the regression function based on level sets of a defined function and that fu…

We consider the following conditional linear regression problem: the task is to identify both (i) a $k$-DNF condition $c$ and (ii) a linear rule $f$ such that the probability of $c$ is (approximately) at least some given bound $μ$, and $f$ minimizes the $\ell_p$ loss of predicting the target $z$ in the distribution of …

We develop a framework for post model selection inference, via marginal screening, in linear regression. At the core of this framework is a result that characterizes the exact distribution of linear functions of the response $y$, conditional on the model being selected (``condition on selection" framework). This allows…

Paper analyzes agnostic learning of mixed linear regression without generative models.

Paper studies IPG method's robustness in noisy distributed linear regression.

Improved efficient robust regression with near-linear time and subquadratic samples.

The network Lasso (nLasso) has been proposed recently as an efficient learning algorithm for massive networked data sets (big data over networks). It extends the well-known least absolute shrinkage and selection operator (Lasso) from learning sparse (generalized) linear models to network models. Efficient implementatio…

Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.

New bounds for transfer learning in linear models, improving generalization.

This paper compares linear regression and neural networks for pricing swing options.

Bayesian model estimates treatment effects near cutoffs in regression discontinuity designs.

Scaled sparse linear regression jointly estimates the regression coefficients and noise level in a linear model. It chooses an equilibrium with a sparse regression method by iteratively estimating the noise level via the mean residual square and scaling the penalty in proportion to the estimated noise level. The iterat…

New method for private linear regression under privacy constraints, achieving optimal rates.

New method speeds up distributed linear regression.

Paper addresses global convergence of MLR estimation under weak data conditions.

Paper proposes EILLS for invariant linear regression across environments.

In this paper, we propose a non-parametric conditional factor regression (NCFR)model for domains with high-dimensional input and response. NCFR enhances linear regression in two ways: a) introducing low-dimensional latent factors leading to dimensionality reduction and b) integrating an Indian Buffet Process as a prior…

SLR tackles sparse linear regression problems, showing hardness for efficient algorithms.

Proposes online debiasing estimators for adaptive linear regression.

Machine learning and statistics typically focus on building models that capture the vast majority of the data, possibly ignoring a small subset of data as "noise" or "outliers." By contrast, here we consider the problem of jointly identifying a significant (but perhaps small) segment of a population in which there is a…

Study optimizes linear regression analysis for high-dimensional settings.

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrtκ)$ and thus achieves the optimal …

Study on online regression with noise, achieving near-optimal regret bounds.

Ensemble of regression trees have become popular statistical tools for the estimation of conditional mean given a set of predictors. However, quantile regression trees and their ensembles have not yet garnered much attention despite the increasing popularity of the linear quantile regression model. This work proposes a…

The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regre…

New method for robust linear regression in nearly linear time.