Stochastic cutting planes improve data-driven optimization speed.
problem Data-driven Mixed-Integer Nonlinear Optimization problems.
method Stochastic version of cutting-plane method.
result Stochastic algorithm converges to ε-optimal solution with high probability.
Differentiable cutting-plane layers solve parametric mixed-integer linear optimization problems.
problem Solving parametric mixed-integer linear optimization problems with changing data.
method Introducing cutting-plane layers (CPLs) for differentiable cutting-plane generation.
result The algorithm computes solutions with low integrality gaps and generalizes to unseen instances.
Generalizes neural network verification by adding arbitrary cutting planes.
problem Handling general cutting plane constraints in neural network verification.
method Generalized bound propagation method (GCP-CROWN) that allows arbitrary cutting plane constraints.
result GCP-CROWN significantly improves neural network verification performance.
RL improves IP solver performance by learning to select cutting planes.
problem Improving the performance of IP solvers through heuristic optimization.
method Employing reinforcement learning to intelligently select cutting planes in the Cutting Plane Method.
result Trained RL agent significantly outperforms human-designed heuristics across various IP tasks.
NeuralCut learns to select cutting planes by looking ahead, outperforming traditional methods.
problem Selecting effective cutting planes for MILP optimization.
method Imitation learning on a lookahead expert to train a neural network for cut selection.
result NeuralCut outperforms standard baselines in cut selection for MILP benchmarks.
A cutting-plane method learns data manifolds efficiently.
problem Classifying data manifolds with continuous parameters.
method Iterative algorithm M_{CP} based on cutting-plane approach solving a quadratic semi-infinite programming problem.
result M_{CP} provides superior generalization performance compared to conventional methods.
Improved cutting plane method for convex optimization and games.
problem Efficiently finding points in convex sets or proving they do not contain balls.
method Optimal cutting plane algorithm using leverage scores and advanced data structures.
result Significant improvement in time complexity for convex optimization and games.
Book covers tools for zeroth-order convex optimisation.
problem Zeroth-order convex optimisation.
method Cutting plane methods, interior point methods, continuous exponential weights, gradient descent, online Newton step.
result Improved existing bounds and algorithms.
New algorithms solve convex optimization problems with limited memory.
problem Solving convex optimization problems with constrained memory.
method Recursive cutting-plane algorithms dividing variables into blocks.
result Achieves optimal memory usage and oracle complexity in certain regimes.
This dissertation uses ILP to learn Bayesian network structures efficiently.
problem Learning the structure of Bayesian networks from data.
method Integer Linear Programming formulation with cluster constraints and cutting planes.
result The approach finds feasible solutions for Bayesian network structures efficiently.
Method provides bounds for sparse PCA and nuclear norm problems.
problem Semidefinite optimization problems (SDOs).
method Cutting-plane method with focus on initial outer approximation as a second-order cone approximation.
result Method provides bound gaps of 0.5-6.5% for sparse PCA problems with 1000 covariates and solves nuclear norm problems over 500x500 matrices.
New memory-query tradeoffs for convex optimization algorithms.
problem Optimizing memory usage for convex optimization algorithms.
method Analyzing randomized first-order algorithms for minimizing convex functions.
result Cutting plane methods are optimal in terms of memory and query complexity.
Memory-constrained algorithms need superlinear memory for efficient convex optimization.
problem Efficiently minimizing convex functions with limited memory.
method Analyzing first-order algorithms with superlinear memory constraints.
result Superlinear memory is necessary for optimal performance in convex optimization.
MOSS optimizes decision rules for accuracy and stability.
problem Constructing stable sets of decision rules.
method Multi-objective optimization framework incorporating sparsity, accuracy, and stability.
result MOSS outperforms state-of-the-art rule ensembles in predictive performance and stability.
OptComplete efficiently completes matrices with side information, providing insights.
problem Matrix completion with interpretability for side information.
method Binary convex optimization reformulation and stochastic cutting planes.
result OptComplete outperforms state-of-the-art methods in scalability and accuracy.
A Support Vector Method for multivariate performance measures was recently introduced by Joachims (2005). The underlying optimization problem is currently solved using cutting plane methods such as SVM-Perf and BMRM. One can show that these algorithms converge to an eta accurate solution in O(1/Lambda*e) iterations, wh…
Develops algorithms to optimize a partial area under the ROC curve.
problem Optimizing performance measures between specific false positive rates.
method Support vector algorithms based on minimizing convex surrogates for partial AUC.
result Polynomial time algorithm for solving combinatorial optimization problem associated with partial AUC.
Financial portfolios are often optimized for maximum profit while subject to a constraint formulated in terms of the Conditional Value-at-Risk (CVaR). This amounts to solving a linear problem. However, in its original formulation this linear problem has a very large number of linear constraints, too many to be enforced…
Novel convex surrogate for submodular losses with tractable computation.
problem Learning with non-modular losses for set prediction.
method Proposed Lovász hinge loss function for submodular losses.
result First tractable convex surrogates for submodular losses.
New method solves complex optimization problems faster.
problem Minimizing a convex smooth objective over the optimal solution set of another convex smooth problem.
method Uses a cutting plane approach to approximate the lower-level problem and an accelerated gradient method to update the upper-level objective.
result Shows that the method requires at most O ( max { 1 / ε f , 1 / ε g } ) \mathcal{O}(\max\{1/\sqrt{ε_{f}}, 1/ε_g\}) O ( max { 1/ ε f , 1/ ε g }) iterations to achieve ε f ε_f ε f -suboptimality and ε g ε_g ε g -infeasibility. New approach learns risk scores efficiently and optimally.
problem Learning risk scores from data is challenging due to calibration, sparsity, and operational constraints.
method Formulated as a mixed integer nonlinear program and solved using a cutting plane algorithm with specialized techniques.
result Improves risk score learning efficiency and optimality, providing a feasible solution without post-processing.
New method solves sparse regression problems in seconds for millions of variables.
problem Sparse regression problems in high-dimensional settings.
method Binary convex reformulation and cutting plane method.
result Approach solves sparse regression problems for n > 100,000 in seconds, recovering 100% true signal.
Paper proposes a robust method for inferring parameters in multiobjective optimization.
problem Uncertainty in hypothetical decision-making problem, data quality, and parameter space.
method Wasserstein distributionally robust approach for inverse multiobjective optimization.
result WRO-IMOP minimizes worst-case expected loss over a Wasserstein ball of distributions.
A new framework for sparse regression models with slow variations.
problem Parameter estimation for sparse regression models with slow variations.
method Formulated as a mixed-integer optimization problem, then reformulated as a binary convex optimization problem with a novel relaxation technique.
result Efficiently solves the problem to provable optimality using a cutting plane-type algorithm.
New method for sparse polynomial regression with fast input ranking and cutting plane optimization.
problem Sparse polynomial regression with controlled functional complexity.
method Two-step approach: input ranking followed by integer optimization.
result Empirical phase transition in identifying relevant inputs and monomials.
Novel method for bilevel optimization with convex lower-level problem.
problem Minimizing a smooth objective over the optimal solution set of a convex constrained problem.
method Local cutting plane approximation of lower-level solution set combined with conditional gradient updates.
result Achieves optimal iteration complexity for the considered class of bilevel problems.
New algorithm for online portfolio selection with reduced runtime.
problem Maximizing total return in online portfolio selection.
method Minimizes current logarithmic loss regularized by log-determinant of Hessian.
result Achieves regret guarantee similar to Universal Portfolios with reduced runtime.
New methods optimize complex optimization problems with improved efficiency.
problem Optimizing complex problems with a convex lower-level objective.
method Uses stochastic cutting planes and conditional gradient updates.
result Improves complexity for both convex and non-convex upper-level functions.
We introduce an efficient method for training the linear ranking support vector machine. The method combines cutting plane optimization with red-black tree based approach to subgradient calculations, and has O(m*s+m*log(m)) time complexity, where m is the number of training examples, and s the average number of non-zer…
The article generalizes Pearson correlation to Riemannian manifolds.
problem Analyzing statistical models on non-linear manifolds.
method Reconstitutes Pearson correlation properties and derives a nonlinear generalization.
result Developed the Riemann-Pearson Correlation for manifold analysis.
Efficiently simulates risk budgeting portfolios using novel algorithms.
problem Estimating risk contributions in portfolios efficiently.
method Cutting planes algorithm, specialised SGD for Expected Shortfall, numerical simulations.
result Outperforms standard convex optimisation solvers in estimating risk budgeting portfolios.
Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims …
We consider the problem of jointly estimating the parameters as well as the structure of binary valued Markov Random Fields, in contrast to earlier work that focus on one of the two problems. We formulate the problem as a maximization of ℓ 1 \ell_1 ℓ 1 -regularized surrogate likelihood that allows us to find a sparse solution…
In this paper we study output coding for multi-label prediction. For a multi-label output coding to be discriminative, it is important that codewords for different label vectors are significantly different from each other. In the meantime, unlike in traditional coding theory, codewords in output coding are to be predic…
In this paper, a novel learning paradigm is presented to automatically identify groups of informative and correlated features from very high dimensions. Specifically, we explicitly incorporate correlation measures as constraints and then propose an efficient embedded feature selection method using recently developed cu…
This monograph presents the main complexity theorems in convex optimization and their corresponding algorithms. Starting from the fundamental theory of black-box optimization, the material progresses towards recent advances in structural optimization and stochastic optimization. Our presentation of black-box optimizati…
Generalizes leverage score sampling for neural networks, accelerating kernel methods and deep learning.
problem Accelerating kernel methods and deep learning training.
method Generalizes leverage score sampling to neural networks and proves equivalence to neural tangent kernel ridge regression.
result Equivalence between regularized neural network and neural tangent kernel ridge regression under leverage score sampling initialization.
Paper solves globally optimal k-means for low dimensional data.
problem Finding globally optimal k-means solutions for low dimensional data.
method Formulates as a concave assignment problem, iteratively solving small concave and large linear programming problems.
result Solves k-means to global optimality for large data sets with several clusters.
Complex Chern-Simons theory reveals peacock patterns in perturbative series.
problem Understanding the structure of partition functions in complex Chern-Simons theory.
method Analyzing the partition function as a holomorphic function and using resurgence theory.
result Perturbative series are resurgent, with trans-series involving non-perturbative variables.
Quantum algorithms improve perceptron learning efficiency.
problem Improving quantum algorithms for perceptron learning.
method Revisiting and correcting a flawed quantum version space perceptron algorithm, proposing quantum-enhanced cutting-plane algorithms.
result Improved complexity bounds for quantum perceptron learning.
Quadratic memory is essential for optimal convex optimization queries.
problem Optimal query complexity for convex optimization and feasibility problems.
method Lower bounds on query complexity for convex optimization and feasibility problems.
result Center-of-mass algorithms are Pareto-optimal for both convex optimization and feasibility problems.
A biclustering algorithm finds dense disjoint subgraphs in weighted bipartite graphs.
problem Finding dense disjoint bicliques in a weighted bipartite graph.
method Semidefinite programming-based branch-and-cut algorithm with upper and lower bounds.
result The algorithm can solve much larger instances than general-purpose solvers.
New framework for conditional risk minimization using optimal transport.
problem High-stakes decisions with side information, especially economic conditions.
method Universal framework based on union-ball formulation in optimal transport.
result Offers interpretability, tractability, and scalability for various risk functionals.
Unified approach tackles logical constraints in mixed-integer optimization.
problem Logical constraints in mixed-integer optimization problems.
method Express logical constraints non-linearly, reformulate as convex binary optimization, solve using outer-approximation.
result Solves problems faster and at larger scale than existing methods.
The problem of maximizing precision at the top of a ranked list, often dubbed Precision@k (prec@k), finds relevance in myriad learning applications such as ranking, multi-label classification, and learning with severe label imbalance. However, despite its popularity, there exist significant gaps in our understanding of…
Convex geometry explains optimal neural network parameters.
problem Understanding optimal parameters in over-parameterized neural networks.
method Convex geometry, extreme points, linear spline interpolation, kernel matrix, cutting-plane algorithm.
result Optimal network parameters can be characterized as interpretable closed-form formulas.
Paper generalizes VB-FTRL for online learning of quantum states with logarithmic loss.
problem Online learning of quantum states with logarithmic loss.
method Generalizes VB-FTRL algorithm for LL-OLQS with polynomial-time implementation.
result Achieves a regret rate of O ( d 2 log ( d + T ) ) O (d^2 \log (d + T)) O ( d 2 log ( d + T )) for LL-OLQS. FairWASP optimizes training data to reduce disparities across subgroups.
problem Reducing disparities in model outputs across different subgroups in machine learning.
method A novel pre-processing approach that minimizes Wasserstein distance to the original dataset while satisfying demographic parity.
result Integer weights are optimal, allowing FairWASP to be understood as duplicating or eliminating samples.