arXiv research

Regularized least-squares approaches have been successfully applied to linear system identification. Recent approaches use quadratic penalty terms on the unknown impulse response defined by stable spline kernels, which control model space complexity by leveraging regularity and bounded-input bounded-output stability. T…

We propose a penalized likelihood method to jointly estimate multiple precision matrices for use in quadratic discriminant analysis and model based clustering. A ridge penalty and a ridge fusion penalty are used to introduce shrinkage and promote similarity between precision matrix estimates. Block-wise coordinate desc…

Piecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ penalties include least squares, Huber, Vapnik, 1-norm, and their asymmetric generalizations. Properties of these estimators depend on the choice of pena…

Global minima found for multidimensional scaling with penalties.

Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the pe…

Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation (Eskin et al, 2004), and this view is consistent with the motivation given by Kirkpatrick et al (2017). In…

This paper studies the risk-adjusted optimal timing to liquidate an option at the prevailing market price. In addition to maximizing the expected discounted return from option sale, we incorporate a path-dependent risk penalty based on shortfall or quadratic variation of the option price up to the liquidation time. We …

This paper introduces a method to incorporate risk sensitivity in RL using quadratic variation penalties.

Insider trading is reduced when penalized, affecting expected penalties in a non-monotone way.

Proposes a new framework for invariant quadratic P&L predictions in option books.

In this paper, we present a novel penalty approach for the numerical solution of continuously controlled HJB equations and HJB obstacle problems. Our results include estimates of the penalisation error for a class of penalty terms, and we show that variations of Newton's method can be used to obtain globally convergent…

A new spline method for manifold learning using Hessian-based curvature penalties.

Unified analysis of multi-task functional linear regression with manifold and composite penalties.

We propose $\ell_1$ norm regularized quadratic surface support vector machine models for binary classification in supervised learning. We establish their desired theoretical properties, including the existence and uniqueness of the optimal solution, reduction to the standard SVMs over (almost) linearly separable data s…

Quantum algorithm speeds up Lasso regression by quadratically faster per iteration.

Study uses deep learning for efficient hedging of long-term financial derivatives.

New insights into convergence of optimization methods for DAG structure learning.

A new penalty-free method optimizes portfolios without quantum annealing penalties.

Data-driven optimization improves mean-variance portfolios by penalizing norms.

State-space smoothing has found many applications in science and engineering. Under linear and Gaussian assumptions, smoothed estimates can be obtained using efficient recursions, for example Rauch-Tung-Striebel and Mayne-Fraser algorithms. Such schemes are equivalent to linear algebraic techniques that minimize a conv…

New screening rules speed up optimal design calculations.

We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…

Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.

Optimal dynamic allocation of carbon allowances reduces emissions efficiently.

New algorithms reduce complexity for solving nonconvex optimization problems with stochastic objectives and constraints.

Low-rank factorization is a standard way to make structured optimization problems in machine learning more tractable by replacing matrix variables with compact factors. For positive semidefinite (PSD) variables, the symmetric Burer--Monteiro factorization (sBMF) writes $Z=XX^\top$ with a single low-rank factor $X$. A r…

The least absolute shrinkage and selection operator (lasso) and ridge regression produce usually different estimates although input, loss function and parameterization of the penalty are identical. In this paper we look for ridge and lasso models with identical solution set. It turns out, that the lasso model with shri…

Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…

We propose a new method for supervised learning, especially suited to wide data where the number of features is much greater than the number of observations. The method combines the lasso ($\ell_1$) sparsity penalty with a quadratic penalty that shrinks the coefficient vector toward the leading principal components of …

In this paper we consider $l_0$ regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving $l_0$ regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…

A new Branch-and-Bound solver tackles L0-penalized problems with flexible loss functions.

Improved continual learning for neural networks with BN layers using K-FAC extension.

Paper proposes a novel optimization method for disaggregating smart meter data.

We analyze dropout in deep networks with rectified linear units and the quadratic loss. Our results expose surprising differences between the behavior of dropout and more traditional regularizers like weight decay. For example, on some simple data sets dropout training produces negative weights even though the output i…

Proposes sparse QSVM for better generalization and interpretability.

In this paper we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian. In particular, we study and compare the Diagonal Quadratic Approximation Method (DQAM) of Mulvey and Ruszczyński and the Parallel Coordinate Descent Method (PCDM) of Richtárik and Takáč. We show that …

This is a follow up of our previous paper - Trybuła and Zawisza \cite{TryZaw}, where we considered a modification of a monotone mean-variance functional in continuous time in stochastic factor model. In this article we address the problem of optimizing the mentioned functional in a market with a stochastic interest rat…

We consider the problem of learning a structured multi-task regression, where the output consists of multiple responses that are related by a graph and the correlated response variables are dependent on the common inputs in a sparse but synergistic manner. Previous methods such as l1/l2-regularized multi-task regressio…

Sparse regression models are increasingly prevalent due to their ease of interpretability and superior out-of-sample performance. However, the exact model of sparse regression with an $\ell_0$ constraint restricting the support of the estimators is a challenging (\NP-hard) non-convex optimization problem. In this paper…

The paper optimizes portfolios with transaction costs in a large asset universe.

This article is concerned with the Bridge Regression, which is a special family in penalized regression with penalty function $\sum_{j=1}^{p}|β_j|^q$ with $q>0$, in a linear model with linear restrictions. The proposed restricted bridge (RBRIDGE) estimator simultaneously estimates parameters and selects important varia…

We study the problem of learning similarity functions over very large corpora using neural network embedding models. These models are typically trained using SGD with sampling of random observed and unobserved pairs, with a number of samples that grows quadratically with the corpus size, making it expensive to scale to…

HAMD optimizes cubic portfolios without quadratization, achieving better results.

Many problems in machine learning and other fields can be (re)for-mulated as linearly constrained separable convex programs. In most of the cases, there are multiple blocks of variables. However, the traditional alternating direction method (ADM) and its linearized version (LADM, obtained by linearizing the quadratic p…

New algorithm tackles stochastic optimization with inequality constraints.

In this paper we present nonparametric estimators for coefficients in stochastic differential equation if the data are described by independent, identically distributed random variables. The problem is formulated as a nonlinear ill-posed operator equation with a deterministic forward operator described by the Fokker-Pl…

A geometric theory explains loss functions for robust representation learning.

In this paper, we consider a risk-based optimal investment problem of an insurer in a regime-switching jump diffusion model with noisy memory. Using the model uncertainty modeling, we formulate the investment problem as a zero-sum, stochastic differential delay game between the insurer and the market, with a convex ris…