A scalable deep learning framework accelerates training of large neural networks for solving 3D Poisson equations.
problem Training large-scale neural networks for solving complex PDEs efficiently.
method Combines multigrid techniques with distributed deep learning to accelerate training.
result Solves 3D Poisson equations up to 512x512x512 resolution efficiently.
Graph neural networks improve AMG convergence for sparse systems.
problem Efficiently constructing algebraic multigrid prolongation operators for sparse linear systems.
method Train a graph neural network to learn prolongation operators from matrix classes, using an unsupervised loss function.
result Improved convergence rates compared to classical AMG methods.
New framework for probabilistic linear solvers reduces manual effort.
problem Manual implementation of probabilistic iterative methods is laborious.
method Affine Tracing: Automatically constructs PIMs from standard implementations.
result Any realistic affine PIM is calibrated, motivating their adoption.
The support vector machine is a flexible optimization-based technique widely used for classification problems. In practice, its training part becomes computationally expensive on large-scale data sets because of such reasons as the complexity and number of iterations in parameter fitting methods, underlying optimizatio…
Speeds up deep neural networks training by 10x using GPU concurrency.
problem Training deep residual neural networks efficiently.
method Layer-wise parallel training with GPU concurrency and Nonlinear Multigrid.
result 10.2x speedup over traditional techniques.
New multigrid approach reduces CNN parameters by focusing on structured convolutions.
problem Redundancy in standard CNNs leads to high parameter count.
method Replace standard convolutions with structured multilevel convolutions.
result Linearly proportional number of parameters to network width, no loss in accuracy.
Algorithm solves American options with regime-switching using multigrid and compact finite difference.
problem Pricing American put options with regime-switching.
method Multigrid iterative algorithm based on compact finite difference schemes and Hermite interpolation.
result The algorithm provides a fast and efficient tool for pricing American put options with regime-switching.
Solving systems of linear equations is a problem occuring frequently in water engineering applications. Usually the size of the problem is too large to be solved via direct factorization. One can resort to iterative approaches, in particular the conjugate gradients method if the matrix is symmetric positive definite. P…
Kernel Multigrid accelerates Back-fitting for additive Gaussian Processes.
problem Slow convergence of Back-fitting in training additive Gaussian Processes.
method Kernel Packets (KP) and Sparse Gaussian Process Regression (GPR) to enhance Back-fitting.
result Kernel Multigrid reduces the required iterations to O(logn). This paper investigates multilevel initialization strategies for training very deep neural networks with a layer-parallel multigrid solver. The scheme is based on the continuous interpretation of the training problem as a problem of optimal control, in which neural networks are represented as discretizations of time-de…
PEARL uses reinforcement learning to improve matrix preconditioners.
problem Learning effective preconditioners for iterative solvers is challenging.
method PEARL employs an actor-critic reinforcement learning framework to learn preconditioners dynamically.
result PEARL outperforms traditional and neural preconditioners in flexibility and solving speed.
A multilevel optimization method for constrained problems.
problem Regularized constrained linear inverse problems with box constraints.
method Geometric multilevel optimization with varying discretization levels.
result Preserves feasibility of updates while speeding up computations.
We introduce a near-linear complexity (geometric and meshless/algebraic) multigrid/multiresolution method for PDEs with rough (L∞) coefficients with rigorous a-priori accuracy and performance estimates. The method is discovered through a decision/game theory formulation of the problems of (1) identifying restri…
A new autoencoder architecture captures multiscale data.
problem Multiscale spatio-temporal data representation.
method Integrates multigrid methods, convolutional autoencoders, and transfer learning.
result Adaptive, hierarchical architecture captures different scaled features dynamically.
The Poisson equation is commonly encountered in engineering, for instance in computational fluid dynamics (CFD) where it is needed to compute corrections to the pressure field to ensure the incompressibility of the velocity field. In the present work, we propose a novel fully convolutional neural network (CNN) architec…
Accelerates MCMC sampling for large-scale problems using machine learning.
problem Efficiently sampling large-scale Bayesian inference problems with high computational cost.
method Integrates low-fidelity machine learning models into a multilevel MCMC framework.
result Significantly accelerates multilevel sampling by a factor of two with similar accuracy.
Exact solver speeds up Weston-Watkins SVM subproblem significantly.
problem Improving performance of Weston-Watkins multiclass SVM.
method Novel reparametrization for exact subproblem solving.
result Significant speed-up over state-of-the-art solvers for large number of classes.
Optimizes neural networks with blackbox solvers using Time-cost Regularization.
problem Improving neural network performance by integrating efficient solvers for complex problems.
method Optimizes both the primary loss function and the performance of the blackbox solver using Time-cost Regularization. Introduces a hyper-blackbox concept to learn blackbox parameters.
result Significant improvement in neural network performance through optimization of blackbox solvers.
Study analyzes 3,171 stocks to pick efficient portfolios using quantum and classical solvers.
problem Creating efficient stock portfolios from a large dataset.
method Used classical and quantum solvers to optimize portfolios of 3,171 US stocks.
result Demonstrated the effectiveness of quantum and classical solvers in portfolio optimization.
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
New solver MPLP++ outperforms existing solvers for dense graph models.
problem Efficiently solving dense, discrete Graphical Models with pairwise potentials.
method Dual Block-Coordinate Ascent with MPLP++ modification.
result MPLP++ significantly outperforms existing solvers, including TRWS.
CRA improves UL-based CO solvers by dynamically smoothing and enforcing discreteness.
problem Local optima and artificial rounding issues in UL-based CO solvers.
method Continuous Relaxation Annealing (CRA) strategy that dynamically shifts from continuous to discrete solutions.
result Significantly enhances UL-based CO solver performance and eliminates artificial rounding.
Study compares 5 ODE solvers on 3 case studies, finding varying accuracy.
problem Comparing estimation accuracy of 5 ODE solvers on 3 case studies.
method Used 5 different numerical ODE solvers (Euler's, Heun's, Midpoint, Runge-Kutta 4th order, ODE45) on 3 case studies and compared their results.
result Different solvers have varying accuracy depending on the case study.
A new method combines classical and machine learning PDE solvers efficiently.
problem Combining classical and machine learning PDE solvers to reduce computational cost and improve accuracy.
method Proposes an approximate greedy router to select solvers at each iteration, mimicking a greedy approach.
result Consistently reduces final error and AUC of the error trajectory compared to single-solver baselines and hybrid approaches.
Higher-order ODE solvers improve deep learning performance.
problem Improving deep learning performance using higher-order ODE solvers.
method Evaluation and improvement of Runge-Kutta (RK) methods for deep learning.
result Higher-order RK solvers can improve deep learning performance by incorporating key ingredients of optimizers.
SA-Solver improves stochastic sampling from DPMs.
problem Efficient sampling from Diffusion Probabilistic Models (DPMs) is time-consuming.
method Proposes SA-Solver, an improved stochastic Adams method for solving diffusion SDE.
result SA-Solver achieves improved or comparable performance compared to SOTA methods for few-step sampling.
Accelerates data generation in score-based models.
problem Slow generation of realistic data by score-based models.
method Developed an adaptive step size SDE solver.
result Generates data 2-10 times faster with high quality.
Although optimization is the longstanding algorithmic backbone of machine learning, new models still require the time-consuming implementation of new solvers. As a result, there are thousands of implementations of optimization algorithms for machine learning problems. A natural question is, if it is always necessary to…
New taxonomy and improved solvers for discrete energy minimization.
problem Maximum-a-posteriori inference in discrete graphical models.
method Dual block-coordinate ascent rule, theoretical analysis, new solver variants.
result Improved state-of-the-art solver outperforming existing methods on all test instances.
End-to-end trainable graph matching using improved combinatorial solvers.
problem Graph matching in deep learning.
method Combining deep learning with optimized combinatorial solvers.
result Advances state-of-the-art on deep graph matching benchmarks.
MIP-GNN uses graph neural networks to predict variable biases for MIP solvers.
problem Improving combinatorial optimization through data-driven insights.
method Encoding MILP interactions as graphs, training a graph neural network to predict variable biases, and guiding the MIP solver with these predictions.
result Significant improvements in solving binary MILPs compared to default settings of state-of-the-art solvers.
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
Improved reinforcement learning with deep learning.
problem Extending MultiGrid Reinforcement Learning to work with deep learning.
method Combining potential-based reward shaping with a learned potential function from interaction, and adapting it for deep learning algorithms.
result DQN augmented with the approach performs significantly better on continuous control tasks.
The paper improves ODE solvers by integrating diverse information types.
problem Improving accuracy and physical meaningfulness of ODE solutions.
method Leveraging probabilistic solvers to include second-order information and physical conservation laws.
result Solutions become more accurate and physically meaningful with additional information.
DPM-Solver speeds up DPM sampling to 10-20 function evaluations.
problem Slow sampling from Diffusion Probabilistic Models (DPMs).
method Exact formulation of diffusion ODE solutions, using change-of-variable and exponentially weighted integral.
result Generates high-quality samples in 10-20 function evaluations.
We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
ML4CO uses machine learning to improve combinatorial optimization solvers.
problem Solving combinatorial problems in practice often involves related data distributions.
method Replacing heuristic components with machine learning approaches.
result Improved state-of-the-art combinatorial optimization solvers.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-craft…
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i.e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two…
GENIE accelerates DDM synthesis with higher-order solvers.
problem Efficiently solving the differential equation for high-quality generation.
method Higher-order Taylor methods, utilizing Jacobian-vector products.
result GENIE significantly accelerates synthesis compared to previous solvers.
New solver avoids memory issues for long differential equations.
problem Memory constraints in adaptive probabilistic ODE solvers.
method Fixed memory demands adaptive probabilistic solver using robust state estimation.
result Eliminates memory issues for long time series simulations.
skscope simplifies sparsity-constrained optimization in Python.
problem Tedious mathematical deduction and programming for sparsity-constrained optimization.
method Introduces skscope, a Python library that allows users to solve sparsity-constrained optimization problems by just programming the objective function.
result skscope enables state-of-the-art solvers to quickly attain sparse solutions in high-dimensional spaces, achieving up to 80x speedup.
Novel probabilistic solver speeds up solving related linear systems.
problem Efficiently solving multiple related linear systems.
method Probabilistic linear solver over the parameter space, leveraging solved systems.
result Faster and more efficient solution of related linear systems.
This work introduces a new loss function to improve the efficiency of optimization-based PDE solvers.
problem Optimization-based PDE solvers converge slowly and are inefficient compared to classical iterative solvers.
method Proposes a novel Stabilized Gradient Residual (SGR) loss function to modulate the condition number.
result The SGR loss achieves orders-of-magnitude faster convergence than the MSE loss in both ODIL and PINNs frameworks.
Machine learning speeds up GPR simulations.
problem Computational demands of simulating practical GPR problems.
method Automatic ML-based forward solver framework using gprMax.
result Near-real-time GPR simulations achieved.
ThiopheneIV is a new solver for implied volatility with proven monotonicity.
problem Efficiently solving implied volatility in financial models.
method Monotone core with Euler-Chebyshev and Halley steps, exact arithmetic proof, practical boundary handling.
result ThiopheneIV agrees closely with multiprecision Black reference prices at low latency.