MIP-GNN uses graph neural networks to predict variable biases for MIP solvers.
problem Improving combinatorial optimization through data-driven insights.
method Encoding MILP interactions as graphs, training a graph neural network to predict variable biases, and guiding the MIP solver with these predictions.
result Significant improvements in solving binary MILPs compared to default settings of state-of-the-art solvers.
New hybrid model reduces MILP solver time by up to 26%.
problem Improving CPU-based MILP solver efficiency.
method Combines GNN and MLP for efficient CPU-based branching.
result Up to 26% reduction in solver running time compared to state-of-the-art methods.
Graph Neural Networks learn to mimic strong branching in MILP solvers.
problem Improving variable selection in MILP solvers using Graph Neural Networks.
method Proposed methods include target smoothing and a Parent-as-Target Lookback regularizer.
result Up to 22% decrease in B&B tree size and 15% improvement in solving times.
NeuralCut learns to select cutting planes by looking ahead, outperforming traditional methods.
problem Selecting effective cutting planes for MILP optimization.
method Imitation learning on a lookahead expert to train a neural network for cut selection.
result NeuralCut outperforms standard baselines in cut selection for MILP benchmarks.
Algorithm finds near-optimal VaR portfolios using MILP, improving risk management.
problem Computing optimal VaR portfolios is hard due to non-convexity and combinatorial nature.
method Formulates VaR portfolio problem as MILP, uses alternate formulations for guarantees.
result Near-optimal VaR portfolios with near-optimality guarantees.
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing strategies. We introduce Checkmate, a system that solves for optimal rematerialization schedules in reasonable times (under an hour) using off-t…
Branch and Bound (B&B) is the exact tree search method typically used to solve Mixed-Integer Linear Programming problems (MILPs). Learning branching policies for MILP has become an active research area, with most works proposing to imitate the strong branching rule and specialize it to distinct classes of problems. We …
ReLU networks trained with MILPs match deep learning accuracy.
problem Training deep neural networks efficiently.
method Iterative training with Mixed Integer Linear Programs (MILPs).
result ReLU networks can be trained with MILPs achieving similar accuracy to deep learning methods.
Binarized Neural Networks (BNNs) have recently attracted significant interest due to their computational efficiency. Concurrently, it has been shown that neural networks may be overly sensitive to "attacks" - tiny adversarial changes in the input - which may be detrimental to their use in safety-critical domains. Desig…
Plain vanilla K-means clustering has proven to be successful in practice, yet it suffers from outlier sensitivity and may produce highly unbalanced clusters. To mitigate both shortcomings, we formulate a joint outlier detection and clustering problem, which assigns a prescribed number of datapoints to an auxiliary outl…
This paper proposes exact and approximation algorithms for Sparse PCA, improving interpretability and scalability.
problem Selecting a prespecified-size principal submatrix from a covariance matrix to maximize its largest eigenvalue.
method Proposes two exact mixed-integer SDPs and a mixed-integer linear program (MILP) for SPCA, analyzes theoretical optimality gaps, and develops approximation algorithms.
result The proposed algorithms achieve strong theoretical optimality and effective scalability, with continuous relaxations close to optimality and MILP solving small to medium-size instances.
The paper verifies the robustness of classifier ensembles against randomized attacks.
problem Ensuring classifier ensembles are robust against arbitrary randomized attacks.
method Formal verification procedure using SMT and MILP encodings to assess robustness.
result Proves the NP-hardness of the robustness-checking problem and provides upper bounds on attack sets.
The paper analyzes how behavioral investors make portfolio decisions using Markowitz Stochastic Dominance criteria.
problem Understanding how behavioral investors make portfolio decisions.
method Developed stochastic optimization problems and MILP models to capture subjective decision weights and probability weighting functions.
result The developed models can be used to formulate computationally tractable portfolio analysis problems.
Optimal design for multinomial logit models improves assortment selection efficiency.
problem Optimal experimental design for multinomial logit models with feedback.
method Two complementary approaches: MILP reformulation and lifted design.
result Achieves statistical efficiency and scalability for MNL bandits.
We can compare the expressiveness of neural networks that use rectified linear units (ReLUs) by the number of linear regions, which reflect the number of pieces of the piecewise linear functions modeled by such networks. However, enumerating these regions is prohibitive and the known analytical bounds are identical for…
Biclustering techniques have been widely used to identify homogeneous subgroups within large data matrices, such as subsets of genes similarly expressed across subsets of patients. Mining a max-sum sub-matrix is a related but distinct problem for which one looks for a (non-necessarily contiguous) rectangular sub-matrix…
Glitches cause unreliable AI decisions with steep boundaries.
problem Glitches impair the reliability of AI models with steep decision boundaries.
method Formal definition of glitches, algorithmic search using MILP encoding.
result Glitches are widespread and indicate potential model inconsistencies.
We study the robustness verification problem for tree-based models, including decision trees, random forests (RFs) and gradient boosted decision trees (GBDTs). Formal robustness verification of decision tree ensembles involves finding the exact minimal adversarial perturbation or a guaranteed lower bound of it. Existin…
We address the problem of verifying neural-based perception systems implemented by convolutional neural networks. We define a notion of local robustness based on affine and photometric transformations. We show the notion cannot be captured by previously employed notions of robustness. The method proposed is based on re…
Efficiently computes quasiconcave envelope with limited data.
problem Approximating unknown quasiconcave function with partial information.
method Solves value problem and interpolation problem with polynomial and logarithmic LPs.
result Efficiently computes quasiconcave envelope with limited data.
Exact solver speeds up Weston-Watkins SVM subproblem significantly.
problem Improving performance of Weston-Watkins multiclass SVM.
method Novel reparametrization for exact subproblem solving.
result Significant speed-up over state-of-the-art solvers for large number of classes.
Optimizes neural networks with blackbox solvers using Time-cost Regularization.
problem Improving neural network performance by integrating efficient solvers for complex problems.
method Optimizes both the primary loss function and the performance of the blackbox solver using Time-cost Regularization. Introduces a hyper-blackbox concept to learn blackbox parameters.
result Significant improvement in neural network performance through optimization of blackbox solvers.
New framework detects model weaknesses in decision tree ensembles.
problem Detecting feature sensitivity in decision tree ensembles.
method Data-aware sensitivity framework using MILP and SMT.
result Realistic and interpretable examples of model weaknesses.
This work analyzes machine learning for Lagrangian Relaxation in MILP.
problem Improving efficiency in solving large-scale MILP problems.
method Data-driven Algorithm Design approach to learn Lagrangian multipliers.
result Stochastic Gradient Ascent achieves the minimax optimal rate for learning multipliers.
Study analyzes 3,171 stocks to pick efficient portfolios using quantum and classical solvers.
problem Creating efficient stock portfolios from a large dataset.
method Used classical and quantum solvers to optimize portfolios of 3,171 US stocks.
result Demonstrated the effectiveness of quantum and classical solvers in portfolio optimization.
This work focuses on support vector machine (SVM) with feature selection. A MILP formulation is proposed for the problem. The choice of suitable features to construct the separating hyperplanes has been modelled in this formulation by including a budget constraint that sets in advance a limit on the number of features …
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
New solver MPLP++ outperforms existing solvers for dense graph models.
problem Efficiently solving dense, discrete Graphical Models with pairwise potentials.
method Dual Block-Coordinate Ascent with MPLP++ modification.
result MPLP++ significantly outperforms existing solvers, including TRWS.
CRA improves UL-based CO solvers by dynamically smoothing and enforcing discreteness.
problem Local optima and artificial rounding issues in UL-based CO solvers.
method Continuous Relaxation Annealing (CRA) strategy that dynamically shifts from continuous to discrete solutions.
result Significantly enhances UL-based CO solver performance and eliminates artificial rounding.
Study compares 5 ODE solvers on 3 case studies, finding varying accuracy.
problem Comparing estimation accuracy of 5 ODE solvers on 3 case studies.
method Used 5 different numerical ODE solvers (Euler's, Heun's, Midpoint, Runge-Kutta 4th order, ODE45) on 3 case studies and compared their results.
result Different solvers have varying accuracy depending on the case study.
A new method combines classical and machine learning PDE solvers efficiently.
problem Combining classical and machine learning PDE solvers to reduce computational cost and improve accuracy.
method Proposes an approximate greedy router to select solvers at each iteration, mimicking a greedy approach.
result Consistently reduces final error and AUC of the error trajectory compared to single-solver baselines and hybrid approaches.
Higher-order ODE solvers improve deep learning performance.
problem Improving deep learning performance using higher-order ODE solvers.
method Evaluation and improvement of Runge-Kutta (RK) methods for deep learning.
result Higher-order RK solvers can improve deep learning performance by incorporating key ingredients of optimizers.
SA-Solver improves stochastic sampling from DPMs.
problem Efficient sampling from Diffusion Probabilistic Models (DPMs) is time-consuming.
method Proposes SA-Solver, an improved stochastic Adams method for solving diffusion SDE.
result SA-Solver achieves improved or comparable performance compared to SOTA methods for few-step sampling.
Accelerates data generation in score-based models.
problem Slow generation of realistic data by score-based models.
method Developed an adaptive step size SDE solver.
result Generates data 2-10 times faster with high quality.
Although optimization is the longstanding algorithmic backbone of machine learning, new models still require the time-consuming implementation of new solvers. As a result, there are thousands of implementations of optimization algorithms for machine learning problems. A natural question is, if it is always necessary to…
New taxonomy and improved solvers for discrete energy minimization.
problem Maximum-a-posteriori inference in discrete graphical models.
method Dual block-coordinate ascent rule, theoretical analysis, new solver variants.
result Improved state-of-the-art solver outperforming existing methods on all test instances.
End-to-end trainable graph matching using improved combinatorial solvers.
problem Graph matching in deep learning.
method Combining deep learning with optimized combinatorial solvers.
result Advances state-of-the-art on deep graph matching benchmarks.
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
The paper improves ODE solvers by integrating diverse information types.
problem Improving accuracy and physical meaningfulness of ODE solutions.
method Leveraging probabilistic solvers to include second-order information and physical conservation laws.
result Solutions become more accurate and physically meaningful with additional information.
DPM-Solver speeds up DPM sampling to 10-20 function evaluations.
problem Slow sampling from Diffusion Probabilistic Models (DPMs).
method Exact formulation of diffusion ODE solutions, using change-of-variable and exponentially weighted integral.
result Generates high-quality samples in 10-20 function evaluations.
We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
ML4CO uses machine learning to improve combinatorial optimization solvers.
problem Solving combinatorial problems in practice often involves related data distributions.
method Replacing heuristic components with machine learning approaches.
result Improved state-of-the-art combinatorial optimization solvers.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-craft…
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i.e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two…
GENIE accelerates DDM synthesis with higher-order solvers.
problem Efficiently solving the differential equation for high-quality generation.
method Higher-order Taylor methods, utilizing Jacobian-vector products.
result GENIE significantly accelerates synthesis compared to previous solvers.
New solver avoids memory issues for long differential equations.
problem Memory constraints in adaptive probabilistic ODE solvers.
method Fixed memory demands adaptive probabilistic solver using robust state estimation.
result Eliminates memory issues for long time series simulations.