The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
Novel probabilistic solver speeds up solving related linear systems.
problem Efficiently solving multiple related linear systems.
method Probabilistic linear solver over the parameter space, leveraging solved systems.
result Faster and more efficient solution of related linear systems.
A learning algorithm optimizes SOR solver parameters for a sequence of linear systems efficiently.
problem Optimizing solver parameters for a sequence of related linear systems without extra computations.
method Bandit and contextual bandit algorithms for online learning of optimal parameters.
result The overall cost approaches the best fixed parameter as the sequence length increases.
New framework for probabilistic linear solvers reduces manual effort.
problem Manual implementation of probabilistic iterative methods is laborious.
method Affine Tracing: Automatically constructs PIMs from standard implementations.
result Any realistic affine PIM is calibrated, motivating their adoption.
Probabilistic solvers improve stability for stiff systems.
problem Performance penalties for small steps in stiff systems.
method Probabilistic exponential integrators that include fast linear dynamics in the prior.
result Proven L-stability and probabilistic error accounting.
AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.
problem Slow convergence of Krylov subspace solvers for ill-conditioned matrices.
method Graph neural surrogate and Bayesian optimization for AI-tuned MCMC parameters.
result 50% reduction in iterations to convergence on unseen system.
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
Graph neural networks improve AMG convergence for sparse systems.
problem Efficiently constructing algebraic multigrid prolongation operators for sparse linear systems.
method Train a graph neural network to learn prolongation operators from matrix classes, using an unsupervised loss function.
result Improved convergence rates compared to classical AMG methods.
New method calibrates probabilistic linear solver for online coverage guarantees.
problem Uncertainty in probabilistic linear solver solutions without coverage guarantees.
method Online conformal prediction-PLS (OCP-PLS) method to calibrate uncertainty thresholds.
result Validates online calibration of uncertainty thresholds via online conformal prediction.
In applications of Gaussian processes where quantification of uncertainty is of primary interest, it is necessary to accurately characterize the posterior distribution over covariance parameters. This paper proposes an adaptation of the Stochastic Gradient Langevin Dynamics algorithm to draw samples from the posterior …
New method solves constrained optimization problems efficiently.
problem Equality-constrained nonlinear, nonconvex optimization problems.
method Adaptive inexact Newton method with randomized iterative sketching.
result Global almost sure convergence and local linear/superlinear convergence.
OKRidge solves sparse ridge regression problems for nonlinear systems.
problem Identifying sparse governing equations for nonlinear dynamical systems.
method OKRidge algorithm using saddle point formulation and ADMM-based approach with efficient proximal operators.
result OKRidge achieves provable optimality with significantly faster run times than Gurobi.
This paper speeds up iterative GP inference with warm starting.
problem Improving scalability of Gaussian process inference.
method Warm starting sequential posteriors using known solutions.
result Significant speed-ups and improved Bayesian optimisation performance.
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i.e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two…
Exact solver speeds up Weston-Watkins SVM subproblem significantly.
problem Improving performance of Weston-Watkins multiclass SVM.
method Novel reparametrization for exact subproblem solving.
result Significant speed-up over state-of-the-art solvers for large number of classes.
Warm starts improve Gaussian process regression by up to 16x.
problem Optimizing hyperparameters for Gaussian processes.
method Iterative Gaussian processes with warm start optimization.
result Warm starts achieve the same results as conventional methods but significantly speed up computations.
RL accelerates portfolio optimization and option pricing by dynamically adjusting preconditioner sizes.
problem Large linear systems in portfolio optimization and option pricing lead to slow convergence.
method Reinforcement Learning (RL) dynamically adjusts block-preconditioner sizes to accelerate convergence.
result RL-driven solver significantly reduces computational cost and accelerates convergence.
New method uses Gaussian processes for solving linear PDEs with boundary conditions.
problem Solving linear PDEs with boundary conditions.
method Boundary Ehrenpreis--Palamodov Gaussian Processes (B-EPGPs).
result Significant accuracy and resource improvements over existing methods.
A new ML method speeds up PDE simulations without needing classical training.
problem Accelerating transient PDE simulations using machine learning.
method Online-learned preconditioners using a bandit algorithm.
result One-shot acceleration of PDE simulations.
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
New solver MPLP++ outperforms existing solvers for dense graph models.
problem Efficiently solving dense, discrete Graphical Models with pairwise potentials.
method Dual Block-Coordinate Ascent with MPLP++ modification.
result MPLP++ significantly outperforms existing solvers, including TRWS.
DiffOPF solves multi-valued OPF problems by sampling from system history.
problem Multi-valued and non-convex OPF problems due to system parameter variability.
method DiffOPF treats OPF as a conditional sampling problem, learning from historical data.
result DiffOPF enables statistically credible warm starts with favorable cost and constraint satisfaction trade-offs.
COSMIC identifies LTV systems from large data sets efficiently.
problem Identification of discrete-time linear time-variant systems from large-scale data.
method Formulates as regularized least squares problem, develops closed-form algorithm with linear complexity.
result Achieves optimal results even with large data volumes, significantly faster than general solvers.
Improved neural PDEs trained on augmented data enhance model accuracy and efficiency.
problem Training neural PDEs on limited data to accurately represent complex systems.
method Space-filling sampling of local states to generate augmented training data.
result Data-augmented neural PDEs outperform traditional emulators in accuracy and stability.
This paper improves linear system solving by optimizing matrix diagonal scaling.
problem Improving the condition number of a matrix for faster iterative methods.
method Left or right diagonal rescaling of the matrix A, with new bounds and algorithms.
result Jacobi preconditioning reduces A's condition number to within a quadratic factor of the best possible scaling.
Unified framework for solving linear systems with improved convergence rates.
problem Efficiently solving linear systems with randomized batch-sampling methods.
method Developed a unified randomized batch-sampling Kaczmarz framework with concentration inequalities for analysis.
result Derived new expected linear convergence rate bounds that are tighter and more reflective of empirical behavior.
New method combines ODE filters and numerical quadrature to propagate model uncertainty.
problem Propagation of model uncertainty in ODE solutions with uncertain parameters.
method Combining ODE filters with numerical quadrature.
result Effective propagation of both numerical and parametric uncertainty.
Study compares 5 ODE solvers on 3 case studies, finding varying accuracy.
problem Comparing estimation accuracy of 5 ODE solvers on 3 case studies.
method Used 5 different numerical ODE solvers (Euler's, Heun's, Midpoint, Runge-Kutta 4th order, ODE45) on 3 case studies and compared their results.
result Different solvers have varying accuracy depending on the case study.
Kaczmarz++ accelerates convergence for ill-conditioned systems.
problem Solving ill-conditioned linear systems efficiently.
method Adaptive momentum acceleration, Tikhonov-regularized projections, and memoization.
result Kaczmarz++ converges faster than Krylov methods on ill-conditioned systems.
Paper introduces LODE-GPs for modeling data following linear ODEs.
problem Modeling data from systems of linear ODEs.
method Symbolic construction of LODE-GPs using Smith normal form algorithms.
result Improves GP modeling of data from systems of linear ODEs.
This paper proposes a novel uncertainty quantification framework for computationally demanding systems characterized by a large vector of non-Gaussian uncertainties. It combines state-of-the-art techniques in advanced Monte Carlo sampling with Bayesian formulations. The key departure from existing works is the use of i…
New CAGP-GS framework improves Gaussian process uncertainty quantification.
problem Scaling issue in Gaussian processes for large datasets.
method Calibrated probabilistic linear solvers for reduced complexity.
result CAGP-GS framework provides more realistic uncertainty quantification.
LR-EDNN reduces PDE solver complexity by limiting network weights to low-rank subspace.
problem Efficiently solving time-dependent PDEs with deep neural networks.
method Low-rank constraint on network weights using SVD for efficient parameter updates.
result LR-EDNN achieves comparable accuracy to full EDNN with fewer parameters and lower cost.
Integrating logical reasoning within deep learning architectures has been a major goal of modern AI systems. In this paper, we propose a new direction toward this goal by introducing a differentiable (smoothed) maximum satisfiability (MAXSAT) solver that can be integrated into the loop of larger deep learning systems. …
Efficient algorithms speed up adversarial training for linear models.
problem Adversarial training for linear models is computationally expensive.
method Tailored optimization algorithms for regression and classification.
result Significantly faster convergence rates for large-scale problems.
We formulate probabilistic numerical approximations to solutions of ordinary differential equations (ODEs) as problems in Gaussian process (GP) regression with non-linear measurement functions. This is achieved by defining the measurement sequence to consist of the observations of the difference between the derivative …
We show how the discovery of robust scalable numerical solvers for arbitrary bounded linear operators can be automated as a Game Theory problem by reformulating the process of computing with partial information and limited resources as that of playing underlying hierarchies of adversarial information games. When the so…
New method uses Gaussian processes to improve PDE solver accuracy.
problem Uncertainty in PDE solver parameters and measurements.
method Physics-informed Gaussian process regression.
result Strictly generalizes weighted residual methods.
Scalable solver reduces PDE uncertainty with active learning.
problem High computational cost in solving PDEs.
method Stochastic dual descent and clustering-based active learning.
result Solver scales to large number of collocation points.
Physarum-inspired solver speeds up LP solving in neural networks.
problem Efficiently solving linear programming problems within neural networks.
method Inspired by Physarum dynamics, a differentiable solver for LPs.
result Solver outperforms existing methods in video segmentation and few-shot learning.
Contemporary power grids are being challenged by rapid voltage fluctuations that are caused by large-scale deployment of renewable generation, electric vehicles, and demand response programs. In this context, monitoring the grid's operating conditions in real time becomes increasingly critical. With the emergent large …
This work extends VQR to non-linear cases and provides scalable solvers.
problem Limitations of VQR in handling non-linear relationships and scalability.
method Extension to non-linear VQR, vector monotone rearrangement, fast solvers.
result Substantial improvement over linear VQR and scalable solvers.
SA-Solver improves stochastic sampling from DPMs.
problem Efficient sampling from Diffusion Probabilistic Models (DPMs) is time-consuming.
method Proposes SA-Solver, an improved stochastic Adams method for solving diffusion SDE.
result SA-Solver achieves improved or comparable performance compared to SOTA methods for few-step sampling.
New study shows faster convergence of SGD and Kaczmarz methods.
problem Improving convergence rates of iterative linear system solvers.
method Last-iterate convergence analysis of SGD with greedy step size over smooth quadratics.
result The t-th iterate attains an O(1/t3/4) convergence rate. During the development of autonomous systems such as driverless cars, it is important to characterize the scenarios that are most likely to result in failure. Adaptive Stress Testing (AST) provides a way to search for the most-likely failure scenario as a Markov decision process (MDP). Our previous work used a deep rei…
Holistic GLMs add constraints for better model quality.
problem Improving classical linear regression models.
method Sparsity-inducing, sign-coherence, and linear constraints.
result Holistic GLMs reliably solve GLMs for various responses.
New methods improve solving linear systems and preconditioning with reduced complexity.
problem Efficiently solving linear systems and preconditioning matrices.
method Developed structured semidefinite programming algorithms.
result Improved runtimes for preconditioning and solving linear systems.
skscope simplifies sparsity-constrained optimization in Python.
problem Tedious mathematical deduction and programming for sparsity-constrained optimization.
method Introduces skscope, a Python library that allows users to solve sparsity-constrained optimization problems by just programming the objective function.
result skscope enables state-of-the-art solvers to quickly attain sparse solutions in high-dimensional spaces, achieving up to 80x speedup.