Optimizes neural networks with blackbox solvers using Time-cost Regularization.
problem Improving neural network performance by integrating efficient solvers for complex problems.
method Optimizes both the primary loss function and the performance of the blackbox solver using Time-cost Regularization. Introduces a hyper-blackbox concept to learn blackbox parameters.
result Significant improvement in neural network performance through optimization of blackbox solvers.
CRA improves UL-based CO solvers by dynamically smoothing and enforcing discreteness.
problem Local optima and artificial rounding issues in UL-based CO solvers.
method Continuous Relaxation Annealing (CRA) strategy that dynamically shifts from continuous to discrete solutions.
result Significantly enhances UL-based CO solver performance and eliminates artificial rounding.
Study analyzes 3,171 stocks to pick efficient portfolios using quantum and classical solvers.
problem Creating efficient stock portfolios from a large dataset.
method Used classical and quantum solvers to optimize portfolios of 3,171 US stocks.
result Demonstrated the effectiveness of quantum and classical solvers in portfolio optimization.
ML4CO uses machine learning to improve combinatorial optimization solvers.
problem Solving combinatorial problems in practice often involves related data distributions.
method Replacing heuristic components with machine learning approaches.
result Improved state-of-the-art combinatorial optimization solvers.
Although optimization is the longstanding algorithmic backbone of machine learning, new models still require the time-consuming implementation of new solvers. As a result, there are thousands of implementations of optimization algorithms for machine learning problems. A natural question is, if it is always necessary to…
End-to-end trainable graph matching using improved combinatorial solvers.
problem Graph matching in deep learning.
method Combining deep learning with optimized combinatorial solvers.
result Advances state-of-the-art on deep graph matching benchmarks.
This work introduces a new loss function to improve the efficiency of optimization-based PDE solvers.
problem Optimization-based PDE solvers converge slowly and are inefficient compared to classical iterative solvers.
method Proposes a novel Stabilized Gradient Residual (SGR) loss function to modulate the condition number.
result The SGR loss achieves orders-of-magnitude faster convergence than the MSE loss in both ODIL and PINNs frameworks.
MIP-GNN uses graph neural networks to predict variable biases for MIP solvers.
problem Improving combinatorial optimization through data-driven insights.
method Encoding MILP interactions as graphs, training a graph neural network to predict variable biases, and guiding the MIP solver with these predictions.
result Significant improvements in solving binary MILPs compared to default settings of state-of-the-art solvers.
skscope simplifies sparsity-constrained optimization in Python.
problem Tedious mathematical deduction and programming for sparsity-constrained optimization.
method Introduces skscope, a Python library that allows users to solve sparsity-constrained optimization problems by just programming the objective function.
result skscope enables state-of-the-art solvers to quickly attain sparse solutions in high-dimensional spaces, achieving up to 80x speedup.
Higher-order ODE solvers improve deep learning performance.
problem Improving deep learning performance using higher-order ODE solvers.
method Evaluation and improvement of Runge-Kutta (RK) methods for deep learning.
result Higher-order RK solvers can improve deep learning performance by incorporating key ingredients of optimizers.
New taxonomy and improved solvers for discrete energy minimization.
problem Maximum-a-posteriori inference in discrete graphical models.
method Dual block-coordinate ascent rule, theoretical analysis, new solver variants.
result Improved state-of-the-art solver outperforming existing methods on all test instances.
A new method combines classical and machine learning PDE solvers efficiently.
problem Combining classical and machine learning PDE solvers to reduce computational cost and improve accuracy.
method Proposes an approximate greedy router to select solvers at each iteration, mimicking a greedy approach.
result Consistently reduces final error and AUC of the error trajectory compared to single-solver baselines and hybrid approaches.
New solver for MKL-SVM with 0/1 loss function.
problem Optimizing MKL-SVM with (0,1)-loss function. method Developed an ADMM solver for nonconvex, nonsmooth optimization.
result Shows promise in simple numerical experiment.
New method combines QQA and gradient-based sampling for combinatorial optimization.
problem Scalability challenges in learning-based solvers for combinatorial optimization.
method Integrates gradient-based update through continuous relaxation with Quasi-Quantum Annealing (QQA) and parallel communication.
result Achieves superior speed-quality trade-offs for large-scale instances.
CADO optimizes heatmap-based solvers for cost minimization, overcoming performance limitations.
problem Heatmap-based solvers lack objective alignment for cost minimization.
method CADO uses Reinforcement Learning to optimize solution cost directly, introducing Label-Centered Reward and Hybrid Fine-Tuning.
result CADO achieves state-of-the-art performance across diverse benchmarks.
DiffOPF solves multi-valued OPF problems by sampling from system history.
problem Multi-valued and non-convex OPF problems due to system parameter variability.
method DiffOPF treats OPF as a conditional sampling problem, learning from historical data.
result DiffOPF enables statistically credible warm starts with favorable cost and constraint satisfaction trade-offs.
Extends GENO framework for GPU optimization of constrained ML problems.
problem Constrained optimization in classical machine learning.
method Extends GENO framework to GPU optimization, specifying problems in a modeling language.
result Solvers on GPU outperform state-of-the-art approaches by several orders of magnitude.
New EOT solvers estimate both plans and maps efficiently.
problem Difficulty in tuning entropic regularization strength in EOT solvers.
method Time discretization and proper parameter scheduling to optimize EOT computation.
result ProgOT is faster and more robust, outperforming neural networks.
RL accelerates portfolio optimization and option pricing by dynamically adjusting preconditioner sizes.
problem Large linear systems in portfolio optimization and option pricing lead to slow convergence.
method Reinforcement Learning (RL) dynamically adjusts block-preconditioner sizes to accelerate convergence.
result RL-driven solver significantly reduces computational cost and accelerates convergence.
A learning algorithm optimizes SOR solver parameters for a sequence of linear systems efficiently.
problem Optimizing solver parameters for a sequence of related linear systems without extra computations.
method Bandit and contextual bandit algorithms for online learning of optimal parameters.
result The overall cost approaches the best fixed parameter as the sequence length increases.
Deep unfolding accelerates MCMC-based COP solvers.
problem Optimizing combinatorial problems with MCMC and gradient descent.
method Combines MCMC and gradient descent, trains step sizes, uses variance estimation for non-differentiable MCMC.
result Significantly accelerates convergence speed for COPs.
Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-craft…
New method solves constrained optimization problems efficiently.
problem Equality-constrained nonlinear, nonconvex optimization problems.
method Adaptive inexact Newton method with randomized iterative sketching.
result Global almost sure convergence and local linear/superlinear convergence.
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i.e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two…
New LNS framework improves integer program solving.
problem Solving large-scale integer linear programs efficiently.
method Large neighborhood search with imitation and reinforcement learning.
result Framework significantly outperforms commercial solvers.
FIRE extracts interpretable rules from tree ensembles.
problem Building sparse and interpretable rule sets from tree ensembles.
method Optimization-based framework with fusion regularization and sparsity-inducing penalties.
result FIRE outperforms state-of-the-art rule ensemble algorithms.
DC3 uses deep learning to solve hard-constrained optimization problems efficiently.
problem Hard constraints in optimization problems make classical solvers slow and infeasible.
method DC3 employs a differentiable procedure to enforce feasibility and unrolls corrections for inequality constraints.
result DC3 achieves near-optimal solutions while maintaining feasibility in both synthetic and real-world tasks.
Understanding the global optimality in deep learning (DL) has been attracting more and more attention recently. Conventional DL solvers, however, have not been developed intentionally to seek for such global optimality. In this paper we propose a novel approximation algorithm, BPGrad, towards optimizing deep models glo…
We consider the problem of configuring general-purpose solvers to run efficiently on problem instances drawn from an unknown distribution. The goal of the configurator is to find a configuration that runs fast on average on most instances, and do so with the least amount of total work. It can run a chosen solver on a r…
GENOT matches cells across data modalities using neural OT solvers.
problem Scalability, privacy, and out-of-sample estimation issues in traditional OT solvers.
method Learn stochastic maps, parameterize OT maps, relax mass conservation, integrate quadratic solvers.
result Demonstrates significant potential for enhancing therapeutic strategies.
Efficient algorithms speed up adversarial training for linear models.
problem Adversarial training for linear models is computationally expensive.
method Tailored optimization algorithms for regression and classification.
result Significantly faster convergence rates for large-scale problems.
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
A new method uses heat diffusion to efficiently solve combinatorial optimization problems.
problem Challenges in combinatorial optimization due to discrete nature and limited search scope.
method Transforming the target function through heat diffusion to enable information flow and more efficient navigation.
result Superior performance across various combinatorial optimization problems.
A scalable method for training prediction models in predict-then-optimize
problem Training prediction models in the predict-then-optimize paradigm
method Decision-focused learning pipeline
result Decision quality competitive with state-of-the-art methods while reducing training time
Leveraging on the convexity of the Lasso problem , screening rules help in accelerating solvers by discarding irrelevant variables, during the optimization process. However, because they provide better theoretical guarantees in identifying relevant variables, several non-convex regularizers for the Lasso have been prop…
Recent work has shown how to embed differentiable optimization problems (that is, problems whose solutions can be backpropagated through) as layers within deep learning architectures. This method provides a useful inductive bias for certain problems, but existing software for differentiable optimization layers is rigid…
Financial portfolio optimization is a widely studied problem in mathematics, statistics, financial and computational literature. It adheres to determining an optimal combination of weights associated with financial assets held in a portfolio. In practice, it faces challenges by virtue of varying math. formulations, par…
Physarum-inspired solver speeds up LP solving in neural networks.
problem Efficiently solving linear programming problems within neural networks.
method Inspired by Physarum dynamics, a differentiable solver for LPs.
result Solver outperforms existing methods in video segmentation and few-shot learning.
Neural solver computes Wasserstein geodesics and velocity fields efficiently.
problem Computing Wasserstein geodesics and velocity fields efficiently.
method Sample-based neural network approach to solve the minimax problem.
result Directly samples from target distribution and estimates velocity field.
Develops a new solver for path-dependent PDEs using signature kernels.
problem Solving path-dependent PDEs (PPDEs) efficiently and accurately.
method Uses signature kernels to solve PPDEs by approximating the solution with minimal norm in a reproducing kernel Hilbert space.
result Proves the consistency of the numerical scheme, ensuring convergence to PPDE solutions as the number of collocation points increases.
DALTON improves ODE parameter estimation by learning from noisy data.
problem High sensitivity to parameters in ODEs produces unreliable parameter estimates.
method Data-adaptive probabilistic likelihood approximation for ODEs.
result DALTON produces more accurate parameter estimates than existing methods.
Continuous-time optimal stopping solved with deep reinforcement learning
problem Optimal stopping problems in continuous time
method CARLOS (Continuous-time Adaptive Reinforcement Learning for Optimal Stopping)
result Higher prices than existing Bermudan solvers, approaching American upper bound
We study \emph{TV regularization}, a widely used technique for eliciting structured sparsity. In particular, we propose efficient algorithms for computing prox-operators for ℓp-norm TV. The most important among these is ℓ1-norm TV, for whose prox-operator we present a new geometric analysis which unveils a …
Integer programming (IP) is a general optimization framework widely applicable to a variety of unstructured and structured problems arising in, e.g., scheduling, production planning, and graph optimization. As IP models many provably hard to solve problems, modern IP solvers rely on many heuristics. These heuristics ar…
Paper develops MMOT framework for financial applications with neural acceleration.
problem Financial optimization and calibration under multi-period martingale constraints.
method Theoretical analysis, incremental updates, adaptive sparse grids, hybrid neural-projection solver.
result Neural solver achieves 1597x speedup for real-time applications.
Paper optimizes neural network initialization using SMT solvers.
problem Improving neural network performance through better initialization.
method Reduces initialization to SMT problem solving.
result Proposed method achieves better performance than random initialization.
Sym-NCO leverages symmetricities to improve DRL-NCO performance.
problem Improving neural combinatorial optimization methods.
method Sym-NCO is a regularizer-based training scheme that exploits universal symmetricities in CO problems and solutions.
result Sym-NCO significantly improves DRL-NCO performance across various CO tasks.
A new approach RA improves stochastic optimization by executing multiple steps between subsample updates.
problem Improving the efficiency and effectiveness of stochastic optimization methods.
method Developed Retrospective Approximation (RA) which executes multiple steps between subsample updates using a deterministic solver.
result RA achieves almost sure and L1 consistency under weak conditions and optimizes iteration and oracle complexity.