Paper uses integer programming for non-convex boosting in classification.
problem Improving classification performance using non-convex optimization.
method Non-convex boosting via integer programming.
result Results comparable to or better than state-of-the-art.
Extends DCP framework to Hadamard manifolds for geodesically convex functions.
problem Verifying convexity in nonlinear programs on Hadamard manifolds.
method Introduces Disciplined Geodesically Convex Programming (DGCP) framework, defining compositions and transformations for geodesically convex functions.
result Allows verification of geodesic convexity for a broader range of functions, including statistical estimators and matrix-valued optimization.
Differentiable layers for convex optimization problems.
problem Rigidity of existing differentiable optimization layers.
method Disciplined parametrized programming and affine-solver-affine form.
result Efficient analytical differentiation through convex programs.
Convex program recovers mixture components in well-separated data.
problem Mixed linear regression with well-separated classes.
method Second-order cone program based on L1 minimization.
result The convex program exactly recovers mixture components under well-separation assumptions.
In this paper we study a broad class of structured nonlinear programming (SNLP) problems. In particular, we first establish the first-order optimality conditions for them. Then we propose sequential convex programming (SCP) methods for solving them in which each iteration is obtained by solving a convex programming pro…
Convex program for estimating nonlinear recurrent models with stability conditions.
problem Estimating parameters in nonlinear recurrent models with stability conditions.
method Formulated a convex program for the estimator of nonlinear recurrent models under stability conditions.
result Sample complexity for the convex program estimator under stable dynamics.
In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search step (gradient descent or Quasi-Newton iteration) into these uniformly optimal conv…
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
In this paper we consider l0 regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving l0 regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…
Established strong geodesic convex functions and their properties.
problem Geodesic convex functions and monotone vector fields on Riemannian manifolds.
method Characterization and relation establishment for strong geodesic convex functions.
result Relation between variational inequality solutions and strict minimizers for multiobjective programming.
Convex optimization refines neural network training, improving model performance and reducing hyperparameter sensitivity.
problem Training deep neural networks using non-convex optimization methods often leads to suboptimal solutions and requires extensive tuning.
method Formulate neural network training as convex programs with regularization terms, leveraging sparse recovery models and semi-infinite programming theory.
result Convex models can achieve global optima and outperform traditional non-convex methods, with improved robustness to hyperparameters.
Neural networks solve copositive programs, revealing insights into training problems.
problem Training two-layer vector-output ReLU neural networks.
method Convex analysis and copositive programming.
result Neural networks solve copositive programs, providing insights into training problems.
This paper tackles minimizing clipped convex functions with heuristics and mixed-integer convex programming.
problem Minimizing a sum of clipped convex functions.
method Heuristics and mixed-integer convex programming.
result Heuristics can find good solutions, and the perspective transformation yields tractable lower bounds.
New method solves convex equations via convex programming.
problem Estimating solutions to systems of equations involving convex nonlinearities.
method Anchored regression, a convex programming approach.
result Guarantees on the accuracy of the estimator in terms of various parameters.
Efficient estimator for non-linear regression problems using convex programming.
problem Non-linear regression problems with difference of convex (DC) non-linearities.
method Formulated as a convex program, using an approximation oracle for gradients.
result Produces accurate estimates with high probability under certain assumptions.
Convex programming framework generates adversarial examples for deep learning models.
problem Generating robust adversarial examples for deep learning models.
method Convex programming for perturbation analysis of deep learning architectures.
result Framework can generate adversarial examples with competitive performance.
Max-linear regression problem solved with convex programming.
problem Estimating parameters in max-linear regression models.
method Formulated and analyzed a scalable convex program called anchored regression (AR).
result AR provides high probability recovery of parameters with a sample complexity of k4p. New convex programs solve minimal-area problems on Riemann surfaces.
problem Finding the conformal metric of least area with constraints on curve lengths.
method Formulated as a local convex program using calibrations and max flow-min cut theorem.
result Derives new insights and numerical solutions for minimal-area metrics.
Bayesian method approximates intractable stochastic programs with chance constraints.
problem Designing systems with stochastic constraints and chance constraints.
method Variational Bayesian approach to approximate posterior predictive integral.
result The solution set converges to the true solution set as the number of observations increases.
Convex relaxations improve CNNs with fixed weights.
problem Improving CNNs with fixed weights.
method Convex relaxations for CNNs with fixed weights using second order cone programs.
result The relaxation recovers the global minimum under a planted model assumption.
Solves multi-objective risk-averse portfolio optimization with convex risk measures.
problem Portfolio optimization under risk and uncertainty.
method Convex vector optimization, Benson's algorithm, Lagrangian duality, scenario-wise decomposition.
result Developed methods to solve complex portfolio optimization problems.
New method finds arbitrage opportunities in fluctuating asset bands.
problem Finding arbitrage opportunities in fluctuating asset bands.
method Formulate as maximizing volatility within a price band, using convex-concave optimization.
result Approximately solves non-convex optimization problem for moving-band arbitrage.
Convex regression is a promising area for bridging statistical estimation and deterministic convex optimization. New piecewise linear convex regression methods are fast and scalable, but can have instability when used to approximate constraints or objective functions for optimization. Ensemble methods, like bagging, sm…
We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, wi…
Paper develops exact convex optimization for neural networks with polynomial activations.
problem Training two-layer neural networks with nonlinear polynomial activations.
method Exact convex optimization using semidefinite programming.
result Global optimization of neural networks is polynomial-time computable.
Bayesian optimization tackles non-convex, two-stage stochastic problems efficiently.
problem Solving non-convex, two-stage stochastic optimization problems with expensive, black-box evaluations.
method Knowledge-gradient-based acquisition function for joint optimization of first- and second-stage variables.
result Comparable and superior empirical results compared to alternatives.
Paper improves robust PCA for noisy, outlier, and missing data.
problem Robust PCA with noise, outliers, and missing data.
method Bridging convex and nonconvex optimization.
result Near-optimal statistical accuracy for robust PCA.
A new spectral clustering algorithm uses convex programming for better cluster identification.
problem Improving spectral clustering for better cluster identification in well-clustered graphs.
method Uses convex programming in the grouping stage of spectral clustering.
result The algorithm can find clusters of nodes with minimal conductance for well-clustered graphs.
Develops exact convex optimization formulations for neural networks.
problem Training two-layer neural networks with rectified linear units.
method Uses semi-infinite duality and minimum norm regularization to develop exact convex optimization formulations.
result Shows equivalence of ReLU networks trained with weight decay to block ℓ1 penalized convex models. Random projection (RP) is a classical technique for reducing storage and computational costs. We analyze RP-based approximations of convex programs, in which the original optimization problem is approximated by the solution of a lower-dimensional problem. Such dimensionality reduction is essential in computation-limite…
SOC-ICNN expands neural network representational capacity by using conic optimization.
problem Restrictive representational capacity of ReLU-based ICNNs.
method Proposes SOC-ICNN architecture that uses Second-Order Cone Programming.
result SOC-ICNN strictly expands representational space without increasing complexity.
A new DC programming approach improves RBM training efficiency.
problem Improving the training efficiency of Restricted Boltzmann Machines (RBMs).
method Formulated a stochastic DC programming approach to minimize RBM log-likelihood.
result The new algorithm achieves higher log-likelihood more rapidly with the same computational budget.
Estimates Markov chains from data with a non-convex rank-constrained approach.
problem Estimating low-rank Markov chains from empirical data.
method Rank-constrained likelihood maximization and DC programming.
result The proposed estimator achieves better empirical performance than other methods.
Proposes a new algorithm for nonconvex sparse learning problems that converges quickly.
problem Nonconvex sparse learning problems in high dimensions.
method Combines proximal Newton algorithm with DC programming for multi-stage convex relaxation.
result Achieves quadratic convergence and finds sparse approximate local optima.
Polynomial-time convex optimization for CNNs with ReLU activations.
problem Training Convolutional Neural Networks (CNNs) with ReLU activations.
method Developed a convex analytic framework using semi-infinite duality to formulate equivalent convex optimization problems for CNN architectures.
result Proved that two-layer CNNs can be globally optimized via an ℓ2 norm regularized convex program. Optimizes distributions robustly with Sinkhorn distance.
problem Distributionally robust optimization with Wasserstein distance.
method Convex programming dual reformulation, stochastic mirror descent algorithm.
result Demonstrates superior performance in synthetic and real data.
Many problems in machine learning and other fields can be (re)for-mulated as linearly constrained separable convex programs. In most of the cases, there are multiple blocks of variables. However, the traditional alternating direction method (ADM) and its linearized version (LADM, obtained by linearizing the quadratic p…
This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement…
Data-driven optimization improves mean-variance portfolios by penalizing norms.
problem Estimation error in mean-variance optimization.
method Augment MVO with norm penalties, use neural networks for optimization, and compute derivatives implicitly.
result Data-driven optimization reduces portfolio risk compared to standard MVO.
This paper tackles multi-marginal optimal transport problems using DC programming.
problem Multi-marginal optimal transport problems in machine learning.
method Promoting structural information in MMOT leads to a DC programming problem.
result Solutions from DC optimization are as qualitative as current methods.
Optimal binning method for numeric targets using mathematical programming.
problem Optimizing the discretization of numeric variables for classification.
method Mathematical programming formulation for binary, continuous, and multi-class targets with constraints.
result Convex mixed-integer programming formulations for all target types.
A hybrid method combines Jacobian and Gauss-Seidel updates for linearly constrained convex programming.
problem Solving linearly constrained convex programming problems with nonseparable constraints.
method Hybrid Jacobian and Gauss-Seidel proximal block coordinate update method.
result The hybrid method converges and performs as well as Gauss-Seidel method and better than a randomized primal-dual BCU method.
We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that…
A new method solves diagonally constrained SDPs quickly and accurately.
problem Solving large-scale diagonally constrained SDPs efficiently.
method Combines momentum from convex optimization with coordinate descent and matrix factorization.
result Local linear convergence and first-order critical point convergence proved.
We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix …
New Kalman filter mitigates model risk by considering normal distributions within a Wasserstein ambiguity set.
problem Model risk in Kalman filtering.
method Distributionally robust optimization over Wasserstein ambiguity set, Nash equilibrium, convex program, Frank-Wolfe algorithm.
result Distributionally robust Kalman filter reduces model risk.
New method approximates neural network training for robustness.
problem Training robust neural networks with adversarial input perturbations.
method Stochastic convex optimization approach to adversarial training.
result Method achieves better adversarial robustness and performance.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.