arXiv research

Equivalence found between algorithmic regularization and convex penalization for convex losses.

Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.

Unified analysis of multi-attribute graph learning with non-convex penalties.

A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.

We develop a first order expansion for convex penalized estimators in high-dimensional regression.

The MM algorithm improves robust penalized estimation for outlier-contaminated data.

In high-dimensional data analysis, penalized likelihood estimators are shown to provide superior results in both variable selection and parameter estimation. A new algorithm, APPLE, is proposed for calculating the Approximate Path for Penalized Likelihood Estimators. Both the convex penalty (such as LASSO) and the nonc…

A new method for nonparametric regression using mesh-based solutions.

Convex clustering solves a stable optimization problem for clustering.

Recently, there has been focus on penalized log-likelihood covariance estimation for sparse inverse covariance (precision) matrices. The penalty is responsible for inducing sparsity, and a very common choice is the convex $l_1$ norm. However, the best estimator performance is not always achieved with this penalty. The …

PEREGRiNN verifies safety of ReLU NNs by penalizing relaxation in a greedy manner.

We consider the problem of learning the inhomogeneous intensity of a counting process, under a sparse segmentation assumption. We introduce a weighted total-variation penalization, using data-driven weights that correctly scale the penalization along the observation interval. We prove that this leads to a sharp tuning …

The $\ell_1$-penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of high-dimensional statistical problems. In this article, we consider a class of weigh…

We extend the analysis of investment strategies derived from penalized quantile regression models, introducing alternative approaches to improve state\textendash of\textendash art asset allocation rules. First, we use a post\textendash penalization procedure to deal with overshrinking and concentration issues. Second, …

This paper proposes a mechanism to produce equivalent Lipschitz surrogates for zero-norm and rank optimization problems by means of the global exact penalty for their equivalent mathematical programs with an equilibrium constraint (MPECs). Specifically, we reformulate these combinatorial problems as equivalent MPECs by…

Paper introduces structured sparsity estimators for Generalized Linear Models.

Let $\cF$ be a set of $M$ classification procedures with values in $[-1,1]$. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in $\cF$. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various …

Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distanc…

Data-driven optimization improves mean-variance portfolios by penalizing norms.

The problem of low-rank matrix estimation recently received a lot of attention due to challenging applications. A lot of work has been done on rank-penalized methods and convex relaxation, both on the theoretical and applied sides. However, only a few papers considered Bayesian estimation. In this paper, we review the …

New algorithms improve sampling from constrained distributions.

The subdifferential of convex functions of the singular spectrum of real matrices has been widely studied in matrix analysis, optimization and automatic control theory. Convex analysis and optimization over spaces of tensors is now gaining much interest due to its potential applications to signal processing, statistics…

A framework estimates multiple precision matrices with shared structures.

We propose the nuclear norm penalty as an alternative to the ridge penalty for regularized multinomial regression. This convex relaxation of reduced-rank multinomial regression has the advantage of leveraging underlying structure among the response categories to make better predictions. We apply our method, nuclear pen…

In this paper the robust utility maximization problem for a market model based on Lévy processes is analyzed. The interplay between the form of the utility function and the penalization function required to have a well posed problem is studied, and for a large class of utility functions it is proved that the dual probl…

Proposes a new robust expectile regression method for high-dimensional data.

We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal $\ell_{1,\infty}$-penalized recursive least squares (R…

Range penalization enhances statistical accuracy and resource efficiency in federated learning.

Derives error formula for convex regression in high dimensions.

In this paper we present nonparametric estimators for coefficients in stochastic differential equation if the data are described by independent, identically distributed random variables. The problem is formulated as a nonlinear ill-posed operator equation with a deterministic forward operator described by the Fokker-Pl…

Study dynamic risk measures with distributional uncertainty using optimal transport.

A new method for high-dimensional classification using Bernstein polynomials.

New method approximates sampling from smooth potential distributions using a vanishing penalty.

Develops exact convex optimization formulations for neural networks.

Paper estimates differences in conditional independence graphs from time-dependent data.

Unified analysis for graph learning from multi-attribute Gaussian time series.

Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.

This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…

We study the risk assessment of uncertain cash flows in terms of dynamic convex risk measures for processes as introduced in Cheridito, Delbaen, and Kupper (2006). These risk measures take into account not only the amounts but also the timing of a cash flow. We discuss their robust representation in terms of suitably p…

New method resolves inconsistency in learning directed acyclic graphs using penalized likelihood.

Sparse high dimensional graphical model selection is a popular topic in contemporary machine learning. To this end, various useful approaches have been proposed in the context of $\ell_1$-penalized estimation in the Gaussian framework. Though many of these inverse covariance estimation approaches are demonstrably scala…

Following the publication of an attack on genome-wide association studies (GWAS) data proposed by Homer et al., considerable attention has been given to developing methods for releasing GWAS data in a privacy-preserving way. Here, we develop an end-to-end differentially private method for solving regression problems wi…

We consider the high-dimensional heteroscedastic regression model, where the mean and the log variance are modeled as a linear combination of input variables. Existing literature on high-dimensional linear regres- sion models has largely ignored non-constant error variances, even though they commonly occur in a variety…

The abundance of high-dimensional data in the modern sciences has generated tremendous interest in penalized estimators such as the lasso, scaled lasso, square-root lasso, elastic net, and many others. In this paper, we establish a general oracle inequality for prediction in high-dimensional linear regression with such…

The paper analyzes LASSO penalization for high-dimensional Beta regression models.

This paper consider penalized empirical loss minimization of convex loss functions with unknown non-linear target functions. Using the elastic net penalty we establish a finite sample oracle inequality which bounds the loss of our estimator from above with high probability. If the unknown target is linear this inequali…

We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…

Proposes a new feature selection method integrating feature relationships.