arXiv research

New method uses DC functions for piecewise linear regression.

This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement…

New algorithms improve submodular minimization via DC programming.

This paper tackles multi-marginal optimal transport problems using DC programming.

We consider the problem of decomposing a multivariate polynomial as the difference of two convex polynomials. We introduce algebraic techniques which reduce this task to linear, second order cone, and semidefinite programming. This allows us to optimize over subsets of valid difference of convex decompositions (dcds) a…

Boosted Difference of Convex Functions Algorithm solves VaR constrained portfolio optimization.

Paper uses integer programming for non-convex boosting in classification.

Extends DCP framework to Hadamard manifolds for geodesically convex functions.

Paper develops exact convex optimization for neural networks with polynomial activations.

Differentiable layers for convex optimization problems.

By exploiting the property that the RBM log-likelihood function is the difference of convex functions, we formulate a stochastic variant of the difference of convex functions (DC) programming to minimize the negative log-likelihood. Interestingly, the traditional contrastive divergence algorithm is a special case of th…

Polynomial-time convex optimization for CNNs with ReLU activations.

In this paper we study a broad class of structured nonlinear programming (SNLP) problems. In particular, we first establish the first-order optimality conditions for them. Then we propose sequential convex programming (SCP) methods for solving them in which each iteration is obtained by solving a convex programming pro…

We propose a computationally efficient estimator, formulated as a convex program, for a broad class of non-linear regression problems that involve difference of convex (DC) non-linearities. The proposed method can be viewed as a significant extension of the "anchored regression" method formulated and analyzed in [10] f…

In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search step (gradient descent or Quasi-Newton iteration) into these uniformly optimal conv…

Novel approximation hierarchy for sparse quadratic programs.

In this paper we consider $l_0$ regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving $l_0$ regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…

Convex optimisation solves inverse kinematics problems more reliably.

Convex optimization refines neural network training, improving model performance and reducing hyperparameter sensitivity.

It has been observed that deep learning architectures tend to make erroneous decisions with high reliability for particularly designed adversarial instances. In this work, we show that the perturbation analysis of these architectures provides a framework for generating adversarial instances by convex programming which,…

This paper tackles minimizing clipped convex functions with heuristics and mixed-integer convex programming.

Neural networks solve copositive programs, revealing insights into training problems.

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statist…

Motivated by big data applications, first-order methods have been extremely popular in recent years. However, naive gradient methods generally converge slowly. Hence, much efforts have been made to accelerate various first-order methods. This paper proposes two accelerated methods towards solving structured linearly co…

Maximizes determinant of vector sums under matroid constraints.

Max-linear regression problem solved with convex programming.

We develop methods to estimate lag and parameters for multiple stable autoregressive processes.

In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of the solution to the generalized eigenvalue problem and obtain sparse principal component analysis (PCA), sparse canonical cor…

Bayesian method approximates intractable stochastic programs with chance constraints.

New formulation of MIL using shapelets for better classifier of bags.

New method finds arbitrage opportunities in fluctuating asset bands.

Demixing problems in many areas such as hyperspectral imaging and differential optical absorption spectroscopy (DOAS) often require finding sparse nonnegative linear combinations of dictionary elements that match observed data. We show how aspects of these problems, such as misalignment of DOAS references and uncertain…

Convex regression is a promising area for bridging statistical estimation and deterministic convex optimization. New piecewise linear convex regression methods are fast and scalable, but can have instability when used to approximate constraints or objective functions for optimization. Ensemble methods, like bagging, sm…

We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, wi…

We consider the problem of the recovery of a k-sparse vector from compressed linear measurements when data are corrupted by a quantization noise. When the number of measurements is not sufficiently large, different $k$-sparse solutions may be present in the feasible set, and the classical l1 approach may be unsuccessfu…

Develops consistent approximations for composite optimization problems.

We introduce a convex approach for mixed linear regression over $d$ features. This approach is a second-order cone program, based on L1 minimization, which assigns an estimate regression coefficient in $\mathbb{R}^{d}$ for each data point. These estimates can then be clustered using, for example, $k$-means. For problem…

Bayesian optimization tackles non-convex, two-stage stochastic problems efficiently.

Paper improves robust PCA for noisy, outlier, and missing data.

We consider the problem of finding model-independent bounds on the price of an Asian option, when the call prices at the maturity date of the option are known. Our methods differ from most approaches to model-independent pricing in that we consider the problem as a dynamic programming problem, where the controlled proc…

Paper introduces $\ell$-DER for regression tasks using morphological operators and convex-concave procedure.

This paper studies the estimation of low-rank Markov chains from empirical trajectories. We propose a non-convex estimator based on rank-constrained likelihood maximization. Statistical upper bounds are provided for the Kullback-Leiber divergence and the $\ell_2$ risk between the estimator and the true transition matri…

Develops exact convex optimization formulations for neural networks.

Proposes Robust Matrix Factorization with Grouping Effect (GRMF) for better performance and robustness.

We study a distributionally robust mean square error estimation problem over a nonconvex Wasserstein ambiguity set containing only normal distributions. We show that the optimal estimator and the least favorable distribution form a Nash equilibrium. Despite the non-convex nature of the ambiguity set, we prove that the …

The closed string field theory minimal-area problem asks for the conformal metric of least area on a Riemann surface with the condition that all non-contractible closed curves have length at least 2π. This is an extremal length problem in conformal geometry as well as a problem in systolic geometry. We consider the ana…

This paper introduces a general multi-class approach to weakly supervised classification. Inferring the labels and learning the parameters of the model is usually done jointly through a block-coordinate descent algorithm such as expectation-maximization (EM), which may lead to local minima. To avoid this problem, we pr…

Random projection (RP) is a classical technique for reducing storage and computational costs. We analyze RP-based approximations of convex programs, in which the original optimization problem is approximated by the solution of a lower-dimensional problem. Such dimensionality reduction is essential in computation-limite…