arXiv research

LightOn OPUs accelerate randomized numerical linear algebra, reducing computational costs.

CoLA automates efficient numerical linear algebra for complex matrix structures.

This chapter is based on lectures on Randomized Numerical Linear Algebra from the 2016 Park City Mathematics Institute summer school on The Mathematics of Data.

RandNLA uses randomness for matrix problems in machine learning.

Improves numerical solution of ill-conditioned linear systems for machine learning.

Efficient kernel methods for large datasets using GPU acceleration.

By using the viewpoint of modern computational algebraic geometry, we explore properties of the optimization landscapes of the deep linear neural network models. After clarifying on the various definitions of "flat" minima, we show that the geometrically flat minima, which are merely artifacts of residual continuous sy…

Methods from learning theory are used in the state space of linear dynamical and control systems in order to estimate the system matrices. An application to stabilization via algebraic Riccati equations is included. The approach is illustrated via a series of numerical examples.

This paper speeds up K-FAC for deep learning by focusing on only a few eigen-modes.

NGRC shows numerical instabilities with short lags and high-degree polynomials.

Graph neural networks improve AMG convergence for sparse systems.

We derive a numerical algorithm for evaluating the Riemannian logarithm on the Stiefel manifold with respect to the canonical metric. In contrast to the existing optimization-based approach, we work from a purely matrix-algebraic perspective. Moreover, we prove that the algorithm converges locally and exhibits a linear…

This paper offers a new algebraic perspective of GCCA using subspace intersection.

Linear algebra algorithms are used widely in a variety of domains, e.g machine learning, numerical physics and video games graphics. For all these applications, loop-level parallelism is required to achieve high performance. However, finding the optimal way to schedule the workload between threads is a non-trivial prob…

The paper finds formulas for flat models of certain Lie algebras.

Simple matrix formulas for Grassmannian curvatures.

New algorithms learn polytree structures from data.

SALSA efficiently approximates leverage scores for big data, improving ARMA model fitting.

We revisit the theory of Discrete Exterior Calculus (DEC) in 2D for general triangulations, relying only on Vector Calculus and Matrix Algebra. We present DEC numerical solutions of the Poisson equation and compare them against those found using the Finite Element Method with linear elements (FEML).

Neural network factorization speeds up Vlasov equation simulations.

Geometric AD framework simplifies derivative computation in JAX.

A new method for generating time-dependent densities efficiently.

We propose a tensor neural network ($t$-NN) framework that offers an exciting new paradigm for designing neural networks with multidimensional (tensor) data. Our network architecture is based on the $t$-product (Kilmer and Martin, 2011), an algebraic formulation to multiply tensors via circulant convolution. In this $t…

The affine Grassmannian is a noncompact smooth manifold that parameterizes all affine subspaces of a fixed dimension. It is a natural generalization of Euclidean space, points being zero-dimensional affine subspaces. We will realize the affine Grassmannian as a matrix manifold and extend Riemannian optimization algorit…

Solving symmetric positive definite linear problems is a fundamental computational task in machine learning. The exact solution, famously, is cubicly expensive in the size of the matrix. To alleviate this problem, several linear-time approximations, such as spectral and inducing-point methods, have been suggested and a…

Let G be a simple linear algebraic group defined over the complex numbers. Fix a proper parabolic subgroup P of G and a nontrivial antidominant character χof P. We prove that a holomorphic principal G-bundle E over a connected complex projective manifold M is semistable and the second Chern class of its adjoint bundle …

The Grassmannian of affine subspaces is a natural generalization of both the Euclidean space, points being zero-dimensional affine subspaces, and the usual Grassmannian, linear subspaces being special cases of affine subspaces. We show that, like the Grassmannian, the affine Grassmannian has rich geometrical and topolo…

Study Poisson algebras for Hamiltonian systems linearization.

This chapter introduces quaternion machine learning for 3D rotations.

Investigates linearity of group amalgams and new examples of non-linear groups.

Develops methods for spectral estimation and rare-event prediction in complex systems.

We prove a general theorem providing smoothed analysis estimates for conic condition numbers of problems of numerical analysis. Our probability estimates depend only on geometric invariants of the corresponding sets of ill-posed inputs. Several applications to linear and polynomial equation solving show that the estima…

Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields. Recently, the loss surfaces of deep neural networks have been explored to gain insight into outstanding questions in optimization, generalization, and network architecture design. However, the degree to wh…

With the advent of massive data sets much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity and dimensionality of the problems. In particular, covariance mat…

We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation …

Matrix approximation method for Bachelier option pricing and Greeks under stochastic volatility models

A group of matrices $G$ with entries in a number field $K$ is defined to be numerical if $G$ has a finite index subgroup of matrices whose entries are algebraic integers. It is shown that an irreducible or completely reducible subgroup of $GL(n,K)\subset GL(n,\mathbb{C})$ is numerical if and only if the traces of its e…

The statistical analysis of Randomized Numerical Linear Algebra (RandNLA) algorithms within the past few years has mostly focused on their performance as point estimators. However, this is insufficient for conducting statistical inference, e.g., constructing confidence intervals and hypothesis testing, since the distri…

Unified determinants via a single equation.

Motivated by numerical integration on manifolds, we relate the algebraic properties of invariant connections to their geometric properties. Using this perspective, we generalize some classical results of Cartan and Nomizu to invariant connections on algebroids. This has fundamental consequences for the theory of numeri…

FCM efficiently approximates committor function with interpretable kernel model.

Investigates integrable systems with linear periodic integral for e(3) Lie algebra.

New method reduces density estimation variance for multivariate data.

We apply state-of-the-art tools in modern high-dimensional numerical linear algebra to approximate efficiently the spectrum of the Hessian of modern deepnets, with tens of millions of parameters, trained on real data. Our results corroborate previous findings, based on small-scale networks, that the Hessian exhibits "s…

We show that formal isomorphism of intransitive linear Lie equations along transversal to the orbits can be extended to neighborhoods of these transversal. In analytic cases, the word formal is dropped from theorems. Also, we associate an intransitive Lie algebra with each intransitive linear Lie equation, and from the…

New basis and Schur-Weyl duality for loop Hecke algebra defined.

The geometric evolution equations provide new ways to address a variety of non-linear problems in Riemannian geometry, and, at the same time, they enjoy numerous physical applications, most notably within the renormalization group analysis of non-linear sigma models and in general relativity. They are divided into clas…

Efficiently solves inverse PDE problems with Gaussian processes.