arXiv research

New sparse penalty improves biclustering for gene expression data.

Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.

In the multiple changepoint setting, various search methods have been proposed which involve optimising either a constrained or penalised cost function over possible numbers and locations of changepoints using dynamic programming. Such methods are typically computationally intensive. Recent work in the penalised optimi…

One-bit measurements widely exist in the real world, and they can be used to recover sparse signals. This task is known as the problem of learning halfspaces in learning theory and one-bit compressive sensing (1bit-CS) in signal processing. In this paper, we propose novel algorithms based on both convex and nonconvex s…

Study evaluates various regularization methods for electricity price forecasting.

A new method for combining multiple data views in supervised learning.

Proposes an alternative invariance penalty to address domain generalization issues.

A new $K$-means method HT $K$-means uses $\ell_0$ penalty for sparsity.

Nonconvex penalty methods for sparse modeling in linear regression have been a topic of fervent interest in recent years. Herein, we study a family of nonconvex penalty functions that we call the trimmed Lasso and that offers exact control over the desired level of sparsity of estimators. We analyze its structural prop…

Study ablated data augmentation techniques and their mathematical equivalence to penalties.

In this paper, we propose a framework for automatic classification of patients from multimodal genetic and brain imaging data by optimally combining them. Additive models with unadapted penalties (such as the classical group lasso penalty or $L_1$-multiple kernel learning) treat all modalities in the same manner and ca…

Gradient descent training of neural networks leads to solutions close to natural cubic splines.

Paper solves convertible bond valuation using finite elements with penalty method.

Efficient cross-validation for multi-penalty ridge regression.

Optimal subset selection for hypothesis testing with penalties.

Paper proposes a method to improve circular coordinate representation for detecting changes in high-dimensional datasets.

There are different problems for resolution of complex LC-MS or GC-MS data, such as the existence of embedded chromatographic peaks, continuum background and overlapping in mass channels for different components. These problems cause rotational ambiguity in recovered profiles calculated using multivariate curve resolut…

We introduce an iterative optimization scheme for convex objectives consisting of a linear loss and a non-separable penalty, based on the expectation-consistent approximation and the vector approximate message-passing (VAMP) algorithm. Specifically, the penalties we approach are convex on a linear transformation of the…

Accelerated gradient method tackles nonconvex penalties in sparse learning.

We establish adaptive results for trend filtering: least squares estimation with a penalty on the total variation of $(k-1)^{\rm th}$ order differences. Our approach is based on combining a general oracle inequality for the $\ell_1$-penalized least squares estimator with "interpolating vectors" to upper-bound the "effe…

We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets. Our study identifies key differences in the tightness of the resulting relaxations through the notion of the lower combinatorial envelope of a set-function along with new necessary conditi…

We demonstrate the existence of universal adversarial perturbations, which can fool a family of audio classification architectures, for both targeted and untargeted attack scenarios. We propose two methods for finding such perturbations. The first method is based on an iterative, greedy approach that is well-known in c…

Functional brain networks are well described and estimated from data with Gaussian Graphical Models (GGMs), e.g. using sparse inverse covariance estimators. Comparing functional connectivity of subjects in two populations calls for comparing these estimated GGMs. Our goal is to identify differences in GGMs known to hav…

A new framework selects information sources to test hypotheses robustly, even with misclassifications.

Wasserstein GANs with Gradient Penalty compute a different optimal transport problem called congested transport.

We consider the problem of selecting the best estimator among a family of Tikhonov regularized estimators, or, alternatively, to select a linear combination of these regularizers that is as good as the best regularizer in the family. Our theory reveals that if the Tikhonov regularizers share the same penalty matrix wit…

New method predicts customer churn using mixed-penalty logistic regression.

Recent developments in linear system identification have proposed the use of non-parameteric methods, relying on regularization strategies, to handle the so-called bias/variance trade-off. This paper introduces an impulse response estimator which relies on an $\ell_2$-type regularization including a rank-penalty derive…

Molecular profiling data (e.g., gene expression) has been used for clinical risk prediction and biomarker discovery. However, it is necessary to integrate other prior knowledge like biological pathways or gene interaction networks to improve the predictive ability and biological interpretability of biomarkers. Here, we…

Two new regularization methods improve neural network performance and complexity control.

We investigate the difference between using an $\ell_1$ penalty versus an $\ell_1$ constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an $\ell_1$ penalty may fail to provide very sparse solutions; a severe disadvantage for variable sel…

Unified framework for fair regression in aware and unaware settings.

SP-SPCA improves sparse PCA by adaptively adjusting variable penalties, enhancing interpretability and stability.

As surrogate functions of $L_0$-norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…

HALO learns to prune neural networks by adaptively shrinking weights.

Systematic trading strategies are rule-based procedures which choose portfolios and allocate assets. In order to attain certain desired return profiles, quantitative strategists must determine a large array of trading parameters. Backtesting, the attempt to identify the appropriate parameters using historical data avai…

Regularization methods are often employed in deep learning neural networks (DNNs) to prevent overfitting. For penalty based DNN regularization methods, convex penalties are typically considered because of their optimization guarantees. Recent theoretical work have shown that nonconvex penalties that satisfy certain reg…

We present a simple and easy to implement method for the numerical solution of a rather general class of Hamilton-Jacobi-Bellman (HJB) equations. In many cases, the considered problems have only a viscosity solution, to which, fortunately, many intuitive (e.g. finite difference based) discretisations can be shown to co…

Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.

Study on Transfer Elastic Net error bounds and grouping effect.

This paper presents numerical algorithm and results for pricing a capital protection option offered by many asset managers for investment portfolios to take advantage of market growth and protect savings. Under optimal withdrawal policyholder behaviour the pricing of such a product is an optimal stochastic control prob…

R package `mvs` handles multi-view data for better model performance.

The paper studies robust risk measures with linear penalties under uncertain distributions.

Identifying homogeneous subgroups of variables can be challenging in high dimensional data analysis with highly correlated predictors. We propose a new method called Hexagonal Operator for Regression with Shrinkage and Equality Selection, HORSES for short, that simultaneously selects positively correlated variables and…

The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…

Paper develops a new algorithm for distribution regression with optimal learning rates.

We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. This problem is relevant in machine learning, statistics and signal processing. It is well known that a linear regression can benefit from knowledge that the underlying regression vec…

The paper explores MMPR to select diverse models for scientific insight.