arXiv research

New method uses DC functions for piecewise linear regression.

In the article the necessary and sufficient conditions for a representation of Lipschitz function of two variables as a difference of two convex functions are formulated. An algorithm of this representation is given. The outcome of this algorithm is a sequence of pairs of convex functions that converge uniformly to a p…

This paper studies quasar-convex functions to improve optimization methods.

We carefully study how well minimizing convex surrogate loss functions, corresponds to minimizing the misclassification error rate for the problem of binary classification with linear predictors. In particular, we show that amongst all convex surrogate losses, the hinge loss gives essentially the best possible bound, o…

In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable. This problem include difference of convex (DC) functions and a family of bi-convex functions as special cas…

Optimally shows the distance between perturbed convex functions and their Γ-regularizations.

In this paper, we propose a successive convex approximation framework for sparse optimization where the nonsmooth regularization function in the objective function is nonconvex and it can be written as the difference of two convex functions. The proposed framework is based on a nontrivial combination of the majorizatio…

An Euler discretization of the Langevin diffusion is known to converge to the global minimizers of certain convex and non-convex optimization problems. We show that this property holds for any suitably smooth diffusion and that different diffusions are suitable for optimizing different classes of convex and non-convex …

Manifolds uniquely identified by boundary distance differences.

Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.

Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convexity/concavity and their extensions, can be integrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can est…

Paper proposes DC functions for better regularization of inverse problems with theoretical guarantees.

Boosted Difference of Convex Functions Algorithm solves VaR constrained portfolio optimization.

We find a different approach to define convex functions in the sub-Riemannian setting. A function on a sub-Riemannian manifold is nonholonomically geodesic convex if its restriction to any nonholonomic (straightest) geodesic is convex. In the case of Carnot groups, this definition coincides with that by Danniell-Garofa…

New algorithm solves complex non-convex problems efficiently.

In this two-part work, we propose an algorithmic framework for solving non-convex problems whose objective function is the sum of a number of smooth component functions plus a convex (possibly non-smooth) or/and smooth (possibly non-convex) regularization function. The proposed algorithm incorporates ideas from several…

Distributed machine learning is an approach allowing different parties to learn a model over all data sets without disclosing their own data. In this paper, we propose a weighted distributed differential privacy (WD-DP) empirical risk minimization (ERM) method to train a model in distributed setting, considering differ…

In this paper we study integer multiplicity rectifiable currents carried by the subgradient (subdifferential) graphs of semi-convex functions on a $n$-dimensional convex domain, and show a weak continuity theorem with respect to pointwise convergence for such currents. As an application, the $k$-Hessian measures are ca…

In this paper, a new approach of defining Steiner symmetrization of coercive convex functions is proposed and some fundamental properties of the new Steiner symmetrization are proved. Further, using the new Steiner symmetrization, we give a different approach to prove a functional version of the Blaschke-Santalo inequa…

The paper explores different smooth map notions on convex sets and their relationships.

We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consi…

Empirical risk minimization frequently employs convex surrogates to underlying discrete loss functions in order to achieve computational tractability during optimization. However, classical convex surrogates can only tightly bound modular loss functions, sub-modular functions or supermodular functions separately while …

We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…

We study convexity and monotonicity properties of option prices in a model with jumps using the fact that these prices satisfy certain parabolic integro-differential equations. Conditions are provided under which preservation of convexity holds, i.e. under which the value, calculated under a chosen martingale measure, …

We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets. Our study identifies key differences in the tightness of the resulting relaxations through the notion of the lower combinatorial envelope of a set-function along with new necessary conditi…

This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement…

Optimizes portfolios using CPT utility via convex optimization.

SGD converges to global minimum for structured non-convex functions.

Improved dynamic regret analysis for strongly convex and smooth functions.

Unified stability bounds for noisy SGD across convex and non-convex losses.

Constructs new elicitable risk measures with multiplicative scoring functions.

In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared …

Proposes a new sparse recovery method using generalized error function.

We investigate online convex optimization in changing environments, and choose the adaptive regret as the performance measure. The goal is to achieve a small regret over every interval so that the comparator is allowed to change over time. Different from previous works that only utilize the convexity condition, this pa…

New method improves MAP inference for CGMs on path graphs, avoiding approximation and maintaining integrality.

The paper uses distance correlation for brain connectivity and a novel multi-task learning model for age prediction.

In this paper, we study adaptive online convex optimization, and aim to design a universal algorithm that achieves optimal regret bounds for multiple common types of loss functions. Existing universal methods are limited in the sense that they are optimal for only a subclass of loss functions. To address this limitatio…

We study dual-based algorithms for distributed convex optimization problems over networks, where the objective is to minimize a sum $\sum_{i=1}^{m}f_i(z)$ of functions over in a network. We provide complexity bounds for four different cases, namely: each function $f_i$ is strongly convex and smooth, each function is ei…

Faster algorithms solve convex function learning problems.

We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises …

In this paper, we further study the forward-backward envelope first introduced in [28] and [30] for problems whose objective is the sum of a proper closed convex function and a twice continuously differentiable possibly nonconvex function with Lipschitz continuous gradient. We derive sufficient conditions on the origin…

New insights into risk aversion for complex decision models.

Adaptive exploration scheme for evaluating multiple policies with different rewards.

Convex neural networks enforce convex constraints on weights and activations, improving generalization.

By exploiting the property that the RBM log-likelihood function is the difference of convex functions, we formulate a stochastic variant of the difference of convex functions (DC) programming to minimize the negative log-likelihood. Interestingly, the traditional contrastive divergence algorithm is a special case of th…

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes o…

Difference of convex (DC) functions cover a broad family of non-convex and possibly non-smooth and non-differentiable functions, and have wide applications in machine learning and statistics. Although deterministic algorithms for DC functions have been extensively studied, stochastic optimization that is more suitable …

Unified framework for analyzing neural networks trained by gradient descent.