Two accelerated methods for linearly constrained convex programming are proposed, improving convergence rates.
problem Efficiently solving structured linearly constrained convex programming problems.
method Two accelerated methods: LALM and LADMM, for composite convex objectives.
result Accelerated methods achieve faster convergence rates compared to non-accelerated methods.
A hybrid method combines Jacobian and Gauss-Seidel updates for linearly constrained convex programming.
problem Solving linearly constrained convex programming problems with nonseparable constraints.
method Hybrid Jacobian and Gauss-Seidel proximal block coordinate update method.
result The hybrid method converges and performs as well as Gauss-Seidel method and better than a randomized primal-dual BCU method.
Boosted Difference of Convex Functions Algorithm solves VaR constrained portfolio optimization.
problem Designing VaR optimal portfolios under financial regulations.
method Boosted Difference of Convex Functions Algorithm (BDCA) with a novel line search framework.
result BDCA linearly converges to a Karush-Kuhn-Tucker point for VaR constrained portfolio problems.
Estimates Markov chains from data with a non-convex rank-constrained approach.
problem Estimating low-rank Markov chains from empirical data.
method Rank-constrained likelihood maximization and DC programming.
result The proposed estimator achieves better empirical performance than other methods.
Many problems in machine learning and other fields can be (re)for-mulated as linearly constrained separable convex programs. In most of the cases, there are multiple blocks of variables. However, the traditional alternating direction method (ADM) and its linearized version (LADM, obtained by linearizing the quadratic p…
Accelerated method for solving convex optimization problems with linear constraints.
problem Solving large-scale convex optimization problems with linear constraints.
method Primal-dual proximal block coordinate updating method with accelerated convergence.
result Achieves O(1/t2) convergence rate for strongly convex objectives and linear rate for independent blocks. New method solves constrained stochastic optimization problems efficiently.
problem Online statistical inference of constrained stochastic nonlinear optimization problems.
method Stochastic Sequential Quadratic Programming (StoSQP) with iterative sketching solver.
result The rescaled primal-dual sequence converges to a mean-zero Gaussian distribution.
A new method solves diagonally constrained SDPs quickly and accurately.
problem Solving large-scale diagonally constrained SDPs efficiently.
method Combines momentum from convex optimization with coordinate descent and matrix factorization.
result Local linear convergence and first-order critical point convergence proved.
Paper tackles multivariate shape-constrained convex regression problems.
problem Fitting a convex function to data with component-wise monotonicity and uniform Lipschitz continuity.
method Least squares estimator via solving a constrained convex quadratic programming problem. Efficient algorithms designed: sGS-ADMM and pALM.
result Both proposed algorithms outperform state-of-the-art methods in numerical experiments.
Method approximates efficient frontier of chance-constrained programs.
problem Approximating the efficient frontier of chance-constrained nonlinear programs.
method Stochastic approximation method based on bi-objective viewpoint.
result Converges to stationary solutions of a smooth approximation of the original problem.
Non-linear control rules improve smart inverter performance in fluctuating grids.
problem Optimizing smart inverter control for voltage regulation and energy efficiency in fluctuating grids.
method Customized non-linear control rules designed as a kernel-based regression task, leveraging a linearized grid model and convex optimization.
result Non-linear control rules achieve near-optimal performance in real-world tests, minimizing voltage deviations and ohmic losses.
A new algorithm finds optimal solutions for constrained decision processes.
problem Optimizing state-value functions with constraints in CMDPs.
method Gradient-Aware Search (GAS) exploiting PWLC structure.
result GAS converges faster and more reliably than existing methods.
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
In this paper we consider l0 regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving l0 regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…
New projection techniques reduce the frequency of projections in solving LCPs.
problem Solving linearly constrained problems efficiently with reduced projection frequency.
method Delayed projection technique to call a projection less frequently.
result Theoretical and practical improvements in convergence rates and efficiency.
New method solves complex constrained optimization problems.
problem Constrained nonconvex-nonconcave minimax optimization problems.
method Inexact proximal gradient method using sequential convex programming.
result Established complexity guarantees for approximate stationary points.
New methods for phase estimation in mixed signals, improving source separation.
problem Estimating phases of mixed complex signals from multichannel observations.
method Three approaches: heuristic, alternate minimization, and convex relaxation.
result Convex relaxation approach yields best results, including exact source separation.
The (constrained) minimization of a ratio of set functions is a problem frequently occurring in clustering and community detection. As these optimization problems are typically NP-hard, one uses convex or spectral relaxations in practice. While these relaxations can be solved globally optimally, they are often too loos…
This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…
Paper introduces new norms for rank-constrained optimization problems.
problem Rank-constrained optimization problems in various fields.
method Introduces a family of low-rank inducing norms and regularizers.
result Other low-rank inducing norms outperform nuclear norm in matrix completion problems.
This paper finds a convex optimization solution for improving privacy-protected data accuracy.
problem Improving accuracy of privacy-protected data aggregates using differential privacy.
method Developed a convex optimization program to find the best strategy for computing multiple correlated aggregates.
result The optimal solution can be found by solving a simple convex optimization program, not a non-convex one.
The paper develops methods for time-varying constrained online convex optimization.
problem Time-varying loss and constraint functions in online convex optimization.
method Model-based augmented Lagrangian methods (MALM) for time-varying and delayed feedback.
result Sublinear regret and constraint violation for both time-varying and delayed feedback scenarios.
Bayesian method approximates intractable stochastic programs with chance constraints.
problem Designing systems with stochastic constraints and chance constraints.
method Variational Bayesian approach to approximate posterior predictive integral.
result The solution set converges to the true solution set as the number of observations increases.
The paper proposes a method to escape saddle points in constrained optimization.
problem Escaping saddle points in smooth nonconvex optimization problems with a convex constraint.
method Generic framework that yields convergence to a second-order stationary point under certain conditions.
result The sequence of iterates reaches an (ε,γ)-second order stationary point in polynomial iterations. A new algorithm for solving constrained convex optimization problems efficiently.
problem Constrained convex optimization problems requiring high accuracy solutions.
method Second-Order Conditional Gradient Sliding (SOCGS) algorithm, using projection-free methods to solve quadratic subproblems inexactly.
result Converges quadratically in primal gap after a finite number of linearly convergent iterations.
Unified approach adjusts classifiers to meet system-level constraints.
problem Multi-class classification under system-level constraints.
method Post-processing approach using linearly constrained stochastic program and entropic regularization.
result Finite-sample guarantees for risk and constraint satisfaction.
Paper connects DP and optimization for RL, suggesting new algorithms.
problem Optimizing scalar objectives in RL.
method Drawing connections between DP and optimization algorithms.
result Links between DP schemes and optimization algorithms.
We develop randomized (block) coordinate descent (CD) methods for linearly constrained convex optimization. Unlike most CD methods, we do not assume the constraints to be separable, but let them be coupled linearly. To our knowledge, ours is the first CD method that allows linear coupling constraints, without making th…
We make non-convex optimization problems convex.
problem Learning from constrained labels is non-convex.
method Characterize and compute tightest convex extensions of non-convex objectives.
result We can compute efficient convex approximations of non-convex objectives.
skscope simplifies sparsity-constrained optimization in Python.
problem Tedious mathematical deduction and programming for sparsity-constrained optimization.
method Introduces skscope, a Python library that allows users to solve sparsity-constrained optimization problems by just programming the objective function.
result skscope enables state-of-the-art solvers to quickly attain sparse solutions in high-dimensional spaces, achieving up to 80x speedup.
Develops a theory to make learning solutions fair and safe.
problem Ensuring learning solutions are unbiased and safe in critical applications.
method Generates a generalization theory based on PAC learning framework, introduces constrained learning algorithm.
result Proves that constrained learning is as learnable as unconstrained learning, provides practical algorithm.
New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.
problem Rank-constrained optimization problems with linear matrix inequalities.
method Investigates Dantzig-Wolfe relaxation and develops conditions for exactness.
result Conditions for extreme point, convex hull, and objective exactness.
Random projection (RP) is a classical technique for reducing storage and computational costs. We analyze RP-based approximations of convex programs, in which the original optimization problem is approximated by the solution of a lower-dimensional problem. Such dimensionality reduction is essential in computation-limite…
Optimal neural network approximation for Wasserstein gradient direction via convex optimization.
problem Approximating Wasserstein gradient direction with limited data.
method Two-layer networks with squared-ReLU activations, SDP relaxation.
result Optimal approximation of Wasserstein gradient direction in two-layer networks.
Data-driven optimization improves mean-variance portfolios by penalizing norms.
problem Estimation error in mean-variance optimization.
method Augment MVO with norm penalties, use neural networks for optimization, and compute derivatives implicitly.
result Data-driven optimization reduces portfolio risk compared to standard MVO.
Estimating a constrained relation is a fundamental problem in machine learning. Special cases are classification (the problem of estimating a map from a set of to-be-classified elements to a set of labels), clustering (the problem of estimating an equivalence relation on a set) and ranking (the problem of estimating a …
A new algorithm solves bilevel optimization with linear constraints.
problem Solving bilevel optimization problems with coupled linear constraints.
method Penalty and augmented Lagrangian methods reformulate the problem; a single-loop, first-order algorithm proposed.
result Improved convergence rates compared to prior methods.
A new method solves sparse regularization problems efficiently and robustly.
problem Sparse regularization in machine learning problems.
method Iteratively reweighted least square (IRLS) with bilevel resolution and BFGS solver.
result Achieves top performances on various sparsity and regularization problems.
Paper solves robust optimization with expectation constraints for financial derivatives.
problem Computing robust maximization solutions with expectation constraints.
method Shows a single convex minimization problem for super-replication values.
result No-arbitrage bounds on various financial derivatives.
Proposes a framework for learning constrained motor skills.
problem Learning constrained motor skills in robotic systems.
method Exploits probabilistic properties of multiple demonstrations in a linearly constrained optimization problem.
result Proposes a non-parametric solution for constrained motor skills.
New method solves SDPs for MaxCut and synchronization problems efficiently.
problem Solving semidefinite programs for MaxCut and synchronization problems.
method Established a Grothendieck-type inequality to prove local maxima are close to global maxima, then applied Riemannian trust-region method with rank constraints.
result Proved that SDPs can be solved within known accuracy using rank constraints, achieving approximations for MaxCut and synchronization problems.
Improved Frank-Wolfe algorithm for constrained convex optimization with nearest extreme point oracle.
problem Constrained smooth convex minimization with limited linear optimization oracle access.
method Frank-Wolfe algorithm with nearest extreme point oracle.
result Improved complexity bounds for specific feasible sets, including linear convergence for 0ext−−1 polytopes. Paper introduces a novel algorithm for semi-supervised one-class classification.
problem Detecting linearly separable classes from unlabeled data.
method The approach assumes linear separability based on class mean and uses linear programming.
result The algorithm successfully identifies linearly separable classes.
The paper optimizes policies constrained to Schur stabilizing controllers using a Newton-type algorithm.
problem Optimizing policies under linear constraints in control systems.
method Newton-type algorithm on a manifold of Schur stabilizing controllers with a Riemannian metric.
result Local convergence guarantees for the Newton-type algorithm without relying on exponential mapping or retractions.
This paper extends forecast reconciliation to non-linearly constrained time series.
problem Forecasting time series with non-linear constraints.
method Non-linearly Constrained Reconciliation (NLCR) algorithm that adjusts forecasts to meet non-linear constraints.
result NLCR significantly improves forecast accuracy compared to benchmarks.
Stochastic heavy ball method achieves linear convergence for general loss minimization.
problem Minimizing generalization error in machine learning models.
method SGD steps with heavy ball momentum, focusing on expected loss, not finite-sum minimization.
result Established the first linear convergence result for the stochastic heavy ball method.
In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of the solution to the generalized eigenvalue problem and obtain sparse principal component analysis (PCA), sparse canonical cor…
This work connects neural network training to convex optimization via the NTK.
problem Understanding and optimizing neural network training via convex programs.
method Interpreting gated ReLU network as MKL, showing NTK equivalence, and improving weights.
result The NTK cannot perform better than the optimal MKL kernel on the training set.