arXiv research

Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.

Flexible empirical Bayes for large-scale multiple linear regression.

In this paper, we propose a one-pass algorithm on MapReduce for penalized linear regression \[f_λ(α, β) = \|Y - α\mathbf{1} - Xβ\|_2^2 + p_λ(β)\] where $α$ is the intercept which can be omitted depending on application; $β$ is the coefficients and $p_λ$ is the penalized function with penalizing parameter $λ$. $f_λ(α, β…

The paper accelerates regression algorithms by identifying saturated coordinates.

The least absolute shrinkage and selection operator (lasso) and ridge regression produce usually different estimates although input, loss function and parameterization of the penalty are identical. In this paper we look for ridge and lasso models with identical solution set. It turns out, that the lasso model with shri…

New scalable algorithm for non-negative linear regression with entropy-regularized OT loss.

Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.

The $\ell_1$-penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of high-dimensional statistical problems. In this article, we consider a class of weigh…

Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distanc…

Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamenta…

In recent years, there has been considerable theoretical development regarding variable selection consistency of penalized regression techniques, such as the lasso. However, there has been relatively little work on quantifying the uncertainty in these selection procedures. In this paper, we propose a new method for inf…

CP degeneracy affects tensor regression solutions, especially in high dimensions.

Paper introduces structured sparsity estimators for Generalized Linear Models.

Proposes a new robust expectile regression method for high-dimensional data.

Study improves error bounds for sparse regression with heavy-tailed covariates.

This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Mi…

The paper discusses methods for interval estimation of coefficients in penalized regression models for insurance data.

A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.

An AI approach selects variables in linear models.

New model handles complex non-linear relationships with hidden graph structures.

FILTER model uses fusion penalized logistic threshold regression for high-dimensional data with unknown cut points.

Paper discusses prediction errors for penalized regressions using GAMP and LOOCV.

It has been shown that AIC-type criteria are asymptotically efficient selectors of the tuning parameter in non-concave penalized regression methods under the assumption that the population variance is known or that a consistent estimator is available. We relax this assumption to prove that AIC itself is asymptotically …

We consider first order expansions of convex penalized estimators in high-dimensional regression problems with random designs. Our setting includes linear regression and logistic regression as special cases. For a given penalty function $h$ and the corresponding penalized estimator $\hatβ$, we construct a quantity $η$,…

PROD method improves high-dimensional regression by handling strong correlations.

Heavy Lasso improves robustness in high-dimensional linear regression with heavy-tailed errors.

New method for multiclass classification reduces error bounds.

Paper optimizes prediction in semi-functional linear models using kernel methods.

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted $\ell_1…

Survey of machine learning methods for time series forecasting.

We introduce a Bernstein-type inequality which serves to uniformly control quadratic forms of gaussian variables. The latter can for example be used to derive sharp model selection criteria for linear estimation in linear regression and linear inverse problems via penalization, and we do not exclude that its scope of a…

Solution to sparse PCA tuning problem using Empirical Bayes.

Sparse-penalized deep neural networks improve performance in weakly dependent processes.

A new robust regression method handles outliers in high-dimensional data.

In this paper we purpose a blockwise descent algorithm for group-penalized multiresponse regression. Using a quasi-newton framework we extend this to group-penalized multinomial regression. We give a publicly available implementation for these in R, and compare the speed of this algorithm to a competing algorithm --- w…

Count data take on non-negative integer values and are challenging to properly analyze using standard linear-Gaussian methods such as linear regression and principal components analysis. Generalized linear models enable direct modeling of counts in a regression context using distributions such as the Poisson and negati…

Develops a method to predict stock returns with time-varying risk premia.

Penalized regression is an attractive framework for variable selection problems. Often, variables possess a grouping structure, and the relevant selection problem is that of selecting groups, not individual variables. The group lasso has been proposed as a way of extending the ideas of the lasso to the problem of group…

New method for inference on strongly identified functionals even when nuisance functions are weakly identified.

Interpretable text-response modelling for structured outcomes

In this paper, we study the performance of extremum estimators from the perspective of generalization ability (GA): the ability of a model to predict outcomes in new samples from the same population. By adapting the classical concentration inequalities, we derive upper bounds on the empirical out-of-sample prediction e…

We extend the analysis of investment strategies derived from penalized quantile regression models, introducing alternative approaches to improve state\textendash of\textendash art asset allocation rules. First, we use a post\textendash penalization procedure to deal with overshrinking and concentration issues. Second, …

Unified analysis of multi-task functional linear regression with manifold and composite penalties.

In this paper, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, (2) approximate the estimator using a few variables by l1-type penalized estimation. We see that the…

New priors improve robustness and interpretability in penalized regression.

Least Angle Regression is a promising technique for variable selection applications, offering a nice alternative to stepwise regression. It provides an explanation for the similar behavior of LASSO ($\ell_1$-penalized regression) and forward stagewise regression, and provides a fast implementation of both. The idea has…

The paper develops methods to create reliable prediction sets for complex mixture models in high-dimensional data.

We consider the high-dimensional heteroscedastic regression model, where the mean and the log variance are modeled as a linear combination of input variables. Existing literature on high-dimensional linear regres- sion models has largely ignored non-constant error variances, even though they commonly occur in a variety…