A new diffusion sampling method combines Krylov subspace and diffusion models for faster and more efficient inverse problems.
problem Efficiently solving large-scale inverse problems in high-performance computing.
method Proposes a novel diffusion sampling strategy that integrates Krylov subspace methods with diffusion models.
result Demonstrates significant speedup (80x faster inference time) and improved reconstruction quality on real-world medical imaging problems.
New Krylov subspace methods speed up mixed-effects models with crossed random effects.
problem Slow computations for high-dimensional crossed random effects in mixed-effects models.
method Krylov subspace-based methods for generalized mixed-effects models with cross effects.
result Speedups by factors of up to 10,000 in computations for mixed-effects models.
Develops a Krylov subspace method for estimating nonlinear systems with random noise.
problem Estimating nonlinear dynamical systems with random noise.
method Lifted representation of nonlinear dynamical systems using transfer operators, extended Arnoldi method, and shift-invert Arnoldi method.
result Empirical validation of methods on synthetic and real-world healthcare data.
A new method for faster optimization in high dimensions.
problem Slow convergence in high-dimensional optimization problems.
method Subspace cubic regularized Newton method within Krylov subspace.
result Achieves a dimension-independent convergence rate of O(1/mk + 1/k^2).
A new method tackles bilevel optimization using Lanczos process for efficient hyper-gradient computation.
problem Efficiently solving large-scale bilevel optimization problems with gradient-based methods.
method Constructing low-dimensional approximate Krylov subspaces with the Lanczos process to approximate the Hessian inverse vector product.
result Demonstrates a O(ε−1) convergence rate and efficiency in synthetic and deep learning tasks. A new method for spectral positional encodings in directed graphs using Hermitian block Krylov subspaces.
problem Challenges in spectral positional encodings for directed graphs, including computational complexity and gauge invariance issues.
method Learnable spectral positional encodings of the form hθ(Aq)R, computed in a Hermitian block Krylov subspace from sparse matrix-vector products. result The method is gauge-invariant and converges to the exact eigendecomposition oracle as the depth grows.
Kaczmarz++ accelerates convergence for ill-conditioned systems.
problem Solving ill-conditioned linear systems efficiently.
method Adaptive momentum acceleration, Tikhonov-regularized projections, and memoization.
result Kaczmarz++ converges faster than Krylov methods on ill-conditioned systems.
In this paper, we propose a second order optimization method to learn models where both the dimensionality of the parameter space and the number of training samples is high. In our method, we construct on each iteration a Krylov subspace formed by the gradient and an approximation to the Hessian matrix, and then use a …
Hessian-free training has become a popular parallel second or- der optimization technique for Deep Neural Network training. This study aims at speeding up Hessian-free training, both by means of decreasing the amount of data used for training, as well as through reduction of the number of Krylov subspace solver iterati…
New method for directed graphs using learnable spectral positional encodings.
problem Challenges in magnetic Laplacians and unitary gauge invariance for directed graphs.
method Learnable spectral PEs of the form hθ(Aq)R, computed in Hermitian block Krylov subspace.
result Gauge-invariant and computationally efficient solution for directed graphs.
Proposes a new regularizer for semi-supervised learning on multilayer graphs.
problem Semi-supervised learning on multilayer graphs with labeled and unlabeled data.
method Generalized matrix mean regularizer and matrix-free numerical scheme.
result The regularizer outperforms state-of-the-art methods numerically.
Unified framework for multi-view learning with orthogonal projections.
problem Learning individual orthogonal projections for multiple views.
method Successive approximations via eigenvectors, iterative Krylov subspace method.
result Consistently competitive and often better than existing methods.
This paper tackles unpaired data in multi-view learning, proposing a new framework and models.
problem Handling unpaired data in multi-view learning, which is more common than paired data.
method Generalized uncorrelated multi-view subspace learning framework with successive alternating approximation (SAA) method.
result Proposed models perform competitively or better than baselines in multi-view feature extraction and multi-modality classification.
Efficiently computes matrix square roots and their inverses for large matrices.
problem Computing matrix square roots and inverses for large matrices efficiently.
method Combines Krylov subspace methods with rational approximation for quadratic-time computation.
result Achieves 4 decimal places of accuracy with fewer than 100 matrix-vector multiplications.
New method improves subspace iteration for eigenvectors in machine learning.
problem Computing eigenvectors for large-scale problems in machine learning.
method Subspace iteration with ℓ2o∞ norm convergence analysis. result Deterministic bounds and practical stopping criterion for improved performance.
New method speeds up kernel-based machine learning for force field reconstruction.
problem Scalability issues in kernel-based machine learning for force field reconstruction.
method Nyström-type methods to construct preconditioners based on low-rank approximations of the kernel matrix.
result Effective preconditioners lead to super-linear convergence in kernel-based machine learning.
Efficiently maps indoor magnetic fields with SKI and D-SKI.
problem Computing large-scale magnetic field maps in indoor environments.
method Structured kernel interpolation (SKI) with derivatives (D-SKI) for Gaussian process regression.
result Achieves better accuracy and faster computation than state-of-the-art methods.
AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.
problem Slow convergence of Krylov subspace solvers for ill-conditioned matrices.
method Graph neural surrogate and Bayesian optimization for AI-tuned MCMC parameters.
result 50% reduction in iterations to convergence on unseen system.
Local Neural Operators enable efficient system-level analysis of complex PDEs.
problem System-level analysis of large-scale dynamical systems using neural operators.
method Integrating local Neural Operators with Krylov subspace iterative methods for stability and bifurcation analysis.
result Demonstrated effectiveness of local Neural Operators in fixed-point, stability, and bifurcation analysis of nonlinear PDEs.
A flag is a sequence of nested subspaces. Flags are ubiquitous in numerical analysis, arising in finite elements, multigrid, spectral, and pseudospectral methods for numerical PDE; they arise in the form of Krylov subspaces in matrix computations, and as multiresolution analysis in wavelets constructions. They are comm…
Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…
We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared…
ISOKANN learns collective variables and effective dynamics for metastable transitions.
problem Understanding metastable transitions in complex molecular systems.
method Integrates Koopman operators with neural networks to extract CVs and effective dynamics.
result Reconstructs coarse-grained kinetics and reproduces transition times across barriers.
KBB algorithm reduces sample complexity for policy evaluation in general state spaces.
problem Policy evaluation in large state spaces with high sample complexity.
method Alternates between fitting Bellman residual and estimating value function via adaptive feature set growth.
result Super-linear convergence rates demonstrated, with reductions in sample complexity.
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
This study examines the relationship between PLS and OLS regression using eigenvalue distributions.
problem Analyzing the difference between PLS and OLS regression in terms of eigenvalue distributions.
method Examined the distance between PLS and OLS regression coefficients using the Mahalanobis distance and eigenvalue distributions of the regressor covariance matrix.
result Provided a bound on the distance between PLS and OLS regression coefficients that depends only on the eigenvalue distribution of the regressor covariance matrix.
Graph-Laplacians and their spectral embeddings play an important role in multiple areas of machine learning. This paper is focused on graph-Laplacian dimension reduction for the spectral clustering of data as a primary application. Spectral embedding provides a low-dimensional parametrization of the data manifold which…
We establish Evans-Krylov estimates for certain nonconvex fully nonlinear elliptic and parabolic equations by exploiting partial Legendre transformations. The equations under consideration arise in part from the study of the "pluriclosed flow" introduced by the first author and Tian
A new model uses Toeplitz matrices to analyze time-series data transitions.
problem Analyzing transitions in time-series data from nonautonomous systems.
method Deep Koopman-layered models with learnable Toeplitz matrices, leveraging Toeplitz matrices' universal property.
result The model demonstrates universality and generalization, outperforming existing methods.
Deep neural networks are usually trained with stochastic gradient descent (SGD), which minimizes objective function using very rough approximations of gradient, only averaging to the real gradient. Standard approaches like momentum or ADAM only consider a single direction, and do not try to model distance from extremum…
Diagonal Frog: High-order positivity-preserving FD schemes for anisotropic Fokker-Planck equations
problem Positivity-preserving discretizations for anisotropic Fokker-Planck equations
method Diagonal Frog discretization
result Second-order accuracy and mass conservation
Recently, neural network based approaches have achieved significant improvement for solving large, complex, graph-structured problems. However, their bottlenecks still need to be addressed, and the advantages of multi-scale information and deep architectures have not been sufficiently exploited. In this paper, we theor…
We describe how to use the perturbation theory of Caffarelli to prove Evans-Krylov type C2,α estimates for solutions of nonlinear elliptic equations in complex geometry, assuming a bound on the Laplacian of the solution. Our results can be used to replace the various Evans-Krylov type arguments in the complex geom…
In this work, we present direction-of-arrival (DoA) estimation algorithms based on the Krylov subspace that effectively exploit prior knowledge of the signals that impinge on a sensor array. The proposed multi-step knowledge-aided iterative conjugate gradient (CG) (MS-KAI-CG) algorithms perform subtraction of the unwan…
Note on advancements in nonlinear elliptic equations' regularity theory.
problem Nonlinear elliptic equations and their regularity.
method De Giorgi-Nash-Moser theory, Krylov-Safonov theory, Evans-Safonov theory.
result Contributions to Hilbert's 19th problem and fully nonlinear equations.
Enhances gradient estimates for Hermitian Monge-Ampère equations.
problem Improving estimates for Hermitian Monge-Ampère equations.
method Improves gradient estimates using Evans-Krylov and third derivatives estimates.
result Enhanced estimates for second and third order derivatives.
Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations xt+1=G(xt), e.g., gradient descent can be viewed as iteratively applying the operation G(x)≜x−α∇f(x). It is known that Anderson acceleration is quite efficient in practice and can be viewed…
Solving symmetric positive definite linear problems is a fundamental computational task in machine learning. The exact solution, famously, is cubicly expensive in the size of the matrix. To alleviate this problem, several linear-time approximations, such as spectral and inducing-point methods, have been suggested and a…
Paper proposes a new method for efficient second-order neural network training.
problem Infeasibility of Hessian calculation and noisy second-order information in deep learning.
method Adopting complex-step directional derivative (CSFD) for accurate Hessian computation and designing an effective Newton Krylov procedure.
result Our method outperforms existing methods and often converges one-order faster.
The class of non-rigid registration methods proposed in the framework of PDE-constrained Large Deformation Diffeomorphic Metric Mapping is a particularly interesting family of physically meaningful diffeomorphic registration methods. PDE-constrained LDDMM methods are formulated as constrained variational problems, wher…
The memory capacity of linear echo state networks is accurately calculated using new numerical methods.
problem Numerical evaluations of memory capacity in recurrent neural networks often contradict theoretical bounds.
method Developed robust numerical approaches exploiting MC neutrality with respect to the input mask matrix.
result Memory curves fully agree with theory when using the proposed methods.
The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…
The paper shows how to stabilize off-policy reinforcement learning using specific state representations.
problem Stability issues in reinforcement learning with function approximation and off-policy learning.
method Formal analysis of representation learning schemes based on the transition matrix of a policy.
result Schur and orthogonal bases of the Krylov subspace provide stable representations for TD learning.
This paper considers exponential utility indifference pricing for a multidimensional non-traded assets model subject to inter-temporal default risk, and provides a semigroup approximation for the utility indifference price. The key tool is the splitting method, whose convergence is proved based on the Barles-Souganidis…
Paper solves Hessian equations on Kähler manifolds.
problem Solving Hessian equations on Kähler manifolds.
method Combines elementary symmetric functions; provides sufficient and necessary condition.
result Generalizes results for Hessian and Hessian quotient equations.
In this paper we introduce a parameter dependent class of Krylov-based methods, namely CD, for the solution of symmetric linear systems. We give evidence that in our proposal we generate sequences of conjugate directions, extending some properties of the standard Conjugate Gradient (CG) method, in order to preserve the…
New lower bounds for sampling from log-concave distributions in higher dimensions.
problem Proving lower bounds for sampling from log-concave distributions in higher dimensions.
method Multiscale construction inspired by geometric measure theory and reduction to block Krylov algorithms.
result Query lower bounds for sampling from log-concave distributions in higher dimensions are established.
The paper studies curvature equations and their solvability.
problem Solving curvature type equations and their Dirichlet problems.
method General class of fully nonlinear curvature equations, Christoffel-Minkowski problem, degenerate equations.
result Solvability of curvature type equations and Dirichlet problems.