New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Rex solves the inverse problem for ODE/SDE solvers, improving precision and stability.
problem Inversion of ODE/SDE solvers is inaccurate and impractical for precision applications.
method Rex uses Lawson methods to convert explicit Runge-Kutta schemes into algebraically reversible ones.
result Rex achieves near-machine-precision reconstruction and improves generative models.
Accelerates data generation in score-based models.
problem Slow generation of realistic data by score-based models.
method Developed an adaptive step size SDE solver.
result Generates data 2-10 times faster with high quality.
ACA method improves gradient estimation for neural ODEs, reducing error and training time.
problem Inaccurate gradient estimation methods hinder the performance of neural ODEs on benchmark tasks.
method Adaptive Checkpoint Adjoint (ACA) method that applies trajectory checkpointing, deletes redundant components, and supports adaptive solvers.
result ACA reduces error rate by half and training time by half compared to adjoint and naive methods on image classification tasks.
We developed efficient methods to compute gradients for Neural SDEs, improving training speed and accuracy.
problem Training Neural SDEs requires accurate and efficient computation of gradients, which is challenging due to the complexity of SDEs.
method We introduced a reversible Heun method for solving backwards-in-time SDEs and a Brownian Interval for sampling and reconstructing Brownian motion.
result Our methods significantly improve training speed and accuracy for Neural SDEs, outperforming state-of-the-art techniques.
Improved diffusion models solve inverse problems more accurately by correcting sample paths off the data manifold.
problem Current diffusion models for inverse problems often produce suboptimal results due to sample paths deviating from the data manifold.
method Proposed an additional correction term inspired by manifold constraints to make iterations closer to the data manifold.
result The proposed method boosts performance by a large margin, producing promising results in various applications.
Improved sampling for Diffusion Models by accounting for covariance.
problem Sampling quality degradation in few-step Diffusion Models.
method Covariance-aware sampler using Tweedie's formula and Fourier-space decomposition.
result Consistently superior samples compared to state-of-the-art samplers.
EVODiff optimizes DM inference by reducing conditional entropy, improving image generation.
problem Slow and inaccurate inference in diffusion models.
method Entropy-aware variance optimization for efficient inference.
result Significant improvement in image generation quality and efficiency.
New method solves blind inverse problems by optimizing both operator and image parameters.
problem Solving blind inverse problems with known forward operator.
method Parallel reverse diffusion guided by gradients from intermediate stages.
result State-of-the-art performance on blind deblurring and imaging through turbulence.
Paper introduces STSL, a second-order Tweedie sampler for efficient posterior sampling in inverse problems.
problem Computational challenges in sampling from posterior distributions using latent diffusion models.
method Introduces STSL, a novel second-order Tweedie sampler with tractable reverse process.
result STSL achieves 4X and 8X reduction in neural function evaluations compared to state-of-the-art solvers.
A JAX toolbox solves optimal transport problems for point clouds and histograms.
problem Optimal transport problems between point clouds and histograms.
method Automatic and custom reverse mode differentiation, vectorization, just-in-time compilation, and accelerators support.
result Solves a wide range of optimal transport problems including regularized OT, barycenters, Gromov-Wasserstein, and low-rank solvers.
A fast regime-split Black-Scholes implied volatility solver
problem Fast computation of implied volatility
method Analytical and numerical expansions
result Achieves near-machine precision with minimal iterations
Efficiently samples complex distributions using tensor train format.
problem Sampling from high-dimensional complex probability densities efficiently.
method Integrates tensor train format with backward stochastic differential equations (BSDEs) for fast, robust, and accurate sampling.
result Improved efficiency in sampling from challenging target distributions.
A new first-order sampler improves diffusion probabilistic model sampling quality.
problem The belief that first-order methods are inherently slower for diffusion probabilistic model sampling.
method A novel training-free, first-order sampler that approximates the forward-value evaluation via a one-step lookahead predictor.
result The proposed sampler provably approximates the ideal forward-value trajectory while retaining first-order convergence and can improve sample quality under the same NFE budget.
New method tackles video inverse problems using image diffusion models.
problem Spatio-temporal degradation in video inverse problems.
method Leverages image diffusion models to treat time dimension as batch dimension, introduces batch-consistent diffusion sampling.
result Achieves state-of-the-art reconstructions for various spatio-temporal degradations.
Generative model uses SDEs to transform data distributions.
problem Creating data from complex distributions.
method Stochastic differential equations (SDEs) for data transformation.
result Achieved record-breaking performance in image generation.
Exact solver speeds up Weston-Watkins SVM subproblem significantly.
problem Improving performance of Weston-Watkins multiclass SVM.
method Novel reparametrization for exact subproblem solving.
result Significant speed-up over state-of-the-art solvers for large number of classes.
Optimizes neural networks with blackbox solvers using Time-cost Regularization.
problem Improving neural network performance by integrating efficient solvers for complex problems.
method Optimizes both the primary loss function and the performance of the blackbox solver using Time-cost Regularization. Introduces a hyper-blackbox concept to learn blackbox parameters.
result Significant improvement in neural network performance through optimization of blackbox solvers.
Study analyzes 3,171 stocks to pick efficient portfolios using quantum and classical solvers.
problem Creating efficient stock portfolios from a large dataset.
method Used classical and quantum solvers to optimize portfolios of 3,171 US stocks.
result Demonstrated the effectiveness of quantum and classical solvers in portfolio optimization.
Stochastic networks are a plausible representation of the relational information among entities in dynamic systems such as living cells or social communities. While there is a rich literature in estimating a static or temporally invariant network from observation data, little has been done toward estimating time-varyin…
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
New solver MPLP++ outperforms existing solvers for dense graph models.
problem Efficiently solving dense, discrete Graphical Models with pairwise potentials.
method Dual Block-Coordinate Ascent with MPLP++ modification.
result MPLP++ significantly outperforms existing solvers, including TRWS.
CRA improves UL-based CO solvers by dynamically smoothing and enforcing discreteness.
problem Local optima and artificial rounding issues in UL-based CO solvers.
method Continuous Relaxation Annealing (CRA) strategy that dynamically shifts from continuous to discrete solutions.
result Significantly enhances UL-based CO solver performance and eliminates artificial rounding.
Study compares 5 ODE solvers on 3 case studies, finding varying accuracy.
problem Comparing estimation accuracy of 5 ODE solvers on 3 case studies.
method Used 5 different numerical ODE solvers (Euler's, Heun's, Midpoint, Runge-Kutta 4th order, ODE45) on 3 case studies and compared their results.
result Different solvers have varying accuracy depending on the case study.
A new method combines classical and machine learning PDE solvers efficiently.
problem Combining classical and machine learning PDE solvers to reduce computational cost and improve accuracy.
method Proposes an approximate greedy router to select solvers at each iteration, mimicking a greedy approach.
result Consistently reduces final error and AUC of the error trajectory compared to single-solver baselines and hybrid approaches.
Higher-order ODE solvers improve deep learning performance.
problem Improving deep learning performance using higher-order ODE solvers.
method Evaluation and improvement of Runge-Kutta (RK) methods for deep learning.
result Higher-order RK solvers can improve deep learning performance by incorporating key ingredients of optimizers.
SA-Solver improves stochastic sampling from DPMs.
problem Efficient sampling from Diffusion Probabilistic Models (DPMs) is time-consuming.
method Proposes SA-Solver, an improved stochastic Adams method for solving diffusion SDE.
result SA-Solver achieves improved or comparable performance compared to SOTA methods for few-step sampling.
Efficiently trains forward processes to minimize generative trajectories curvature.
problem High curvature of generative trajectories slows down sampling speed.
method Trains forward process to minimize curvature without ODE/SDE simulation.
result Lower curvature than previous models, decreased sampling costs.
Although optimization is the longstanding algorithmic backbone of machine learning, new models still require the time-consuming implementation of new solvers. As a result, there are thousands of implementations of optimization algorithms for machine learning problems. A natural question is, if it is always necessary to…
New taxonomy and improved solvers for discrete energy minimization.
problem Maximum-a-posteriori inference in discrete graphical models.
method Dual block-coordinate ascent rule, theoretical analysis, new solver variants.
result Improved state-of-the-art solver outperforming existing methods on all test instances.
End-to-end trainable graph matching using improved combinatorial solvers.
problem Graph matching in deep learning.
method Combining deep learning with optimized combinatorial solvers.
result Advances state-of-the-art on deep graph matching benchmarks.
MIP-GNN uses graph neural networks to predict variable biases for MIP solvers.
problem Improving combinatorial optimization through data-driven insights.
method Encoding MILP interactions as graphs, training a graph neural network to predict variable biases, and guiding the MIP solver with these predictions.
result Significant improvements in solving binary MILPs compared to default settings of state-of-the-art solvers.
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
The paper improves ODE solvers by integrating diverse information types.
problem Improving accuracy and physical meaningfulness of ODE solutions.
method Leveraging probabilistic solvers to include second-order information and physical conservation laws.
result Solutions become more accurate and physically meaningful with additional information.
DPM-Solver speeds up DPM sampling to 10-20 function evaluations.
problem Slow sampling from Diffusion Probabilistic Models (DPMs).
method Exact formulation of diffusion ODE solutions, using change-of-variable and exponentially weighted integral.
result Generates high-quality samples in 10-20 function evaluations.
We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
Classifies reversible and strongly reversible elements in quaternionic groups.
problem Classifying reversible and strongly reversible elements in quaternionic groups.
method Proves elements are reversible if and only if they are products of skew-involutions (resp. involutions).
result Proves elements are reversible if and only if they are products of skew-involutions (resp. involutions).
The paper classifies reversible and strongly reversible elements in Hermitian isometry groups.
problem Classifying reversible and strongly reversible elements in Hermitian isometry groups.
method Classification through group theory and algebraic manipulation.
result New classification of strongly reversible elements in Sp(n).
ML4CO uses machine learning to improve combinatorial optimization solvers.
problem Solving combinatorial problems in practice often involves related data distributions.
method Replacing heuristic components with machine learning approaches.
result Improved state-of-the-art combinatorial optimization solvers.
Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-craft…
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i.e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two…
New method reconstructs Black-Scholes option prices from current profiles.
problem Reconstructing Black-Scholes prices from current profiles, dealing with ill-posedness.
method Price-dimensional reduction using Legendre polynomials, Tikhonov regularization.
result Reconstructs Black-Scholes prices from noisy initial data, stabilizing the solution.
Improved sampling efficiency for inverse problems using variance-reduced diffusion methods.
problem Efficiently estimating noisy scores in inverse problems.
method Developed a nonparametric self-normalized importance sampling estimator and a state-dependent blending rule.
result Improved sample quality for fixed simulation budgets in synthetic targets and PDE-governed inverse problems.
GENIE accelerates DDM synthesis with higher-order solvers.
problem Efficiently solving the differential equation for high-quality generation.
method Higher-order Taylor methods, utilizing Jacobian-vector products.
result GENIE significantly accelerates synthesis compared to previous solvers.
New solver avoids memory issues for long differential equations.
problem Memory constraints in adaptive probabilistic ODE solvers.
method Fixed memory demands adaptive probabilistic solver using robust state estimation.
result Eliminates memory issues for long time series simulations.
skscope simplifies sparsity-constrained optimization in Python.
problem Tedious mathematical deduction and programming for sparsity-constrained optimization.
method Introduces skscope, a Python library that allows users to solve sparsity-constrained optimization problems by just programming the objective function.
result skscope enables state-of-the-art solvers to quickly attain sparse solutions in high-dimensional spaces, achieving up to 80x speedup.
Novel probabilistic solver speeds up solving related linear systems.
problem Efficiently solving multiple related linear systems.
method Probabilistic linear solver over the parameter space, leveraging solved systems.
result Faster and more efficient solution of related linear systems.