arXiv research

The MM algorithm improves robust penalized estimation for outlier-contaminated data.

A new Branch-and-Bound solver tackles L0-penalized problems with flexible loss functions.

In this work we establish the equivalence of algorithmic regularization and explicit convex penalization for generic convex losses. We introduce a geometric condition for the optimization path of a convex function, and show that if such a condition is satisfied, the optimization path of an iterative algorithm on the un…

Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.

Improved DPO framework penalizes preference uncertainty to avoid overoptimization.

Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.

The paper develops a deep neural network estimator for weakly dependent processes with various loss functions.

Adam's hyperparameters implicitly regularize solutions, penalizing or impeding loss gradients' norms.

Increasing variance of losses improves learning with noisy labels.

Paper introduces structured sparsity estimators for Generalized Linear Models.

The paper develops robust risk measures for uncertain loss positions.

New risk class penalizes loss deviations from mean on both sides.

Improved convergence rates for MLE in mixture models using penalized log-likelihood.

Let $\cF$ be a set of $M$ classification procedures with values in $[-1,1]$. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in $\cF$. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various …

This paper consider penalized empirical loss minimization of convex loss functions with unknown non-linear target functions. Using the elastic net penalty we establish a finite sample oracle inequality which bounds the loss of our estimator from above with high probability. If the unknown target is linear this inequali…

We consider first order expansions of convex penalized estimators in high-dimensional regression problems with random designs. Our setting includes linear regression and logistic regression as special cases. For a given penalty function $h$ and the corresponding penalized estimator $\hatβ$, we construct a quantity $η$,…

A new method for high-dimensional classification using Bernstein polynomials.

Heavy Lasso improves robustness in high-dimensional linear regression with heavy-tailed errors.

This paper introduces a gradient analysis framework to improve language model performance by rewarding good examples and penalizing bad ones.

We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneousl…

Matrix factorization methods are extensively employed to understand complex data. In this paper, we introduce the cross-product penalized component analysis (XCAN), a sparse matrix factorization based on the optimization of a loss function that allows a trade-off between variance maximization and structural preservatio…

In this paper, we introduce a novel combined reward cum penalty loss function to handle the regression problem. The proposed combined reward cum penalty loss function penalizes the data points which lie outside the $ε$-tube of the regressor and also assigns reward for the data points which lie inside of the $ε$-tube of…

We consider the problem of unveiling the implicit network structure of node interactions (such as user interactions in a social network), based only on high-frequency timestamps. Our inference is based on the minimization of the least-squares loss associated with a multivariate Hawkes model, penalized by $\ell_1$ and t…

Understanding efficiency in high dimensional linear models is a longstanding problem of interest. Classical work with smaller dimensional problems dating back to Huber and Bickel has illustrated the benefits of efficient loss functions. When the number of parameters $p$ is of the same order as the sample size $n$, $p \…

In this paper, we establish the non-positivity of the second eigenvalue of the Schrödinger operator $-\textrm{div}\big( P_r \nabla\cdot\big) - W_r^2$ on a closed hypersurface $Σ^n$ of $\mathbb{R}^{n+1}$, where $W_r$ is a power of the $(r+1)$-th mean curvature of $Σ^n$. In the case that this eigenvalue is null we have a…

Failing to distinguish between a sheepdog and a skyscraper should be worse and penalized more than failing to distinguish between a sheepdog and a poodle; after all, sheepdogs and poodles are both breeds of dogs. However, existing metrics of failure (so-called "loss" or "win") used in textual or visual classification/r…

Proposes a new robust expectile regression method for high-dimensional data.

The $\ell_1$-penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of high-dimensional statistical problems. In this article, we consider a class of weigh…

We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the sample size n. The bounds lead us to consider sample variance penalization, a nov…

Study nonparametric density estimation via measure transport, achieving optimal rates.

New fair regression method improves fairness in chronic kidney disease classification.

The least absolute shrinkage and selection operator (lasso) and ridge regression produce usually different estimates although input, loss function and parameterization of the penalty are identical. In this paper we look for ridge and lasso models with identical solution set. It turns out, that the lasso model with shri…

Early training phase affects deep neural network optimization and generalization.

Supervised linear feature extraction can be achieved by fitting a reduced rank multivariate model. This paper studies rank penalized and rank constrained vector generalized linear models. From the perspective of thresholding rules, we build a framework for fitting singular value penalized models and use it for feature …

Label noise in SGD helps converge to flatter minima.

Gradient descent implicitly regularizes neural networks by penalizing large loss gradients.

A new learning method using hyperbolic geometry for class labels.

The paper optimizes bridge-type estimators for sparse models using pathwise methods.

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted $\ell_1…

The paper proposes a method for predicting equity premium using penalized quantile regression.

Paper tackles efficient SVM classification over decentralized networks.

DYNOTEARS learns connections between variables over time, outperforming other methods.

This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded. The bound applies to many loss functions besides the squared loss, and scales only with the sub-Gaussian or …

Feature selection is a technique to screen out less important features. Many existing supervised feature selection algorithms use redundancy and relevancy as the main criteria to select features. However, feature interaction, potentially a key characteristic in real-world problems, has not received much attention. As a…

Deep learning has become the method of choice in many application domains of machine learning in recent years, especially for multi-class classification tasks. The most common loss function used in this context is the cross-entropy loss, which reduces to the log loss in the typical case when there is a single correct r…

We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…

We propose an AdaPtive Noise Augmentation (PANDA) technique to regularize the estimation and construction of undirected graphical models. PANDA iteratively optimizes the objective function given the noise augmented data until convergence to achieve regularization on model parameters. The augmented noises can be designe…

Paper proposes AXE loss for non-autoregressive machine translation, improving performance.