arXiv research

Efficiently differentiate functions of large matrices using new adjoint systems.

New recommendations improve Gaussian process accuracy and stability.

Botnet, a group of coordinated bots, is becoming the main platform of malicious Internet activities like DDOS, click fraud, web scraping, spam/rumor distribution, etc. This paper focuses on design and experiment of a new approach for botnet detection from streaming web server logs, motivated by its wide applicability, …

Principal component analysis (PCA) is one of the most powerful tools in machine learning. The simplest method for PCA, the power iteration, requires $\mathcal O(1/Δ)$ full-data passes to recover the principal component of a matrix with eigen-gap $Δ$. Lanczos, a significantly more complex method, achieves an accelerated…

We propose the Lanczos network (LanczosNet), which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution. Relying on the tridiagonal decomposition of the Lanczos algorithm, we not only efficiently exploit multi-scale information via fast approximated computation of…

A new method tackles bilevel optimization using Lanczos process for efficient hyper-gradient computation.

The study examines algebraic structures of specific tensor forms in four-dimensional spacetimes.

One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions. Recent advances in inducing point methods have sped up GP marginal likelihood and posterior mean computations, leaving posterior covariance estimation and sampling as the remaining …

The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…

For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an $n \times n$ positive definite matrix, and its derivatives - leading to prohibitive $\mathcal{O}(n^3)$ computatio…

The paper analyzes contraction rates for GP regression approximations.

The purpose if this master's thesis is to study and develop a new algorithmic framework for Collaborative Filtering to produce recommendations in the top-N recommendation problem. Thus, we propose Lanczos Latent Factor Recommender (LLFR); a novel "big data friendly" collaborative filtering algorithm for top-N recommend…

New Krylov subspace methods speed up mixed-effects models with crossed random effects.

We develop and analyze efficient "coordinate-wise" methods for finding the leading eigenvector, where each step involves only a vector-vector product. We establish global convergence with overall runtime guarantees that are at least as good as Lanczos's method and dominate it for slowly decaying spectrum. Our methods a…

Numerous methods for computing conformal mesh paramterizations has been developed due to the vast applications in the field of geometry processing. Spectral conformal parameterization (SCP) is one of these methods to computing a quality conformal parameterization based on the spectral technique. SCP focus on a generali…

In all dimensions and arbitrary signature, we demonstrate the existence of a new local potential -- a double (2,3)-form -- for the Weyl curvature tensor, and more generally for all tensors with the symmetry properties of the Weyl curvature tensor. The classical four-dimensional Lanczos potential for a Weyl tensor -- a …

We study the algorithmic problem of estimating the mean of heavy-tailed random vector in $\mathbb{R}^d$, given $n$ i.i.d. samples. The goal is to design an efficient estimator that attains the optimal sub-gaussian error bound, only assuming that the random vector has bounded mean and covariance. Polynomial-time solutio…

This paper deals with finding an $n$-dimensional solution $x$ to a system of quadratic equations of the form $y_i=|\langle{a}_i,x\rangle|^2$ for $1\le i \le m$, which is also known as phase retrieval and is NP-hard in general. We put forth a novel procedure for minimizing the amplitude-based least-squares empirical los…

We exploit four-dimensional tensor identities to give a very simple proof of the existence of a Lanczos potential for a Weyl tensor in four dimensions with any signature, and to show that the potential satisfies a simple linear second order differential equation, e.g., a wave equation in Lorentz signature. Furthermore,…

The graph Laplacian is a standard tool in data science, machine learning, and image processing. The corresponding matrix inherits the complex structure of the underlying network and is in certain applications densely populated. This makes computations, in particular matrix-vector products, with the graph Laplacian a ha…

We analyze the Hessian spectra of large models up to 100B parameters.

The $(2k)$-th Gauss-Bonnet curvature is a generalization to higher dimensions of the $(2k)$-dimensional Gauss-Bonnet integrand, it coincides with the usual scalar curvature for $k=1$. The Gauss-Bonnet curvatures are used in theoretical physics to describe gravity in higher dimensional space times where they are known a…

We found in 2016 a few results on the mathematical structure of the conformal Killing differential sequence in arbitrary dimension $n$, in particular the rank and order changes of the successive differential operators for $n=3,n=4$ or $n\geq 5$. They were so striking that we did not dare to publish them before our form…

This paper proposes a novel profile likelihood method for estimating the covariance parameters in exploratory factor analysis of high-dimensional Gaussian datasets with fewer observations than number of variables. An implicitly restarted Lanczos algorithm and a limited-memory quasi-Newton method are implemented to deve…

Evaluating the log determinant of a positive definite matrix is ubiquitous in machine learning. Applications thereof range from Gaussian processes, minimum-volume ellipsoids, metric learning, kernel learning, Bayesian neural networks, Determinental Point Processes, Markov random fields to partition functions of discret…

Semidefinite programming (SDP) with diagonal constraints arise in many optimization problems, such as Max-Cut, community detection and group synchronization. Although SDPs can be solved to arbitrary precision in polynomial time, generic convex solvers do not scale well with the dimension of the problem. In order to add…

We present MLRG Deep Curvature suite, a PyTorch-based, open-source package for analysis and visualisation of neural network curvature and loss landscape. Despite of providing rich information into properties of neural network and useful for a various designed tasks, curvature information is still not made sufficient us…

Match van Stockum dust to vacuum metrics with a single parameter.

We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos …

We propose a new hybrid algorithm that allows incorporating both user and item side information within the standard collaborative filtering technique. One of its key features is that it naturally extends a simple PureSVD approach and inherits its unique advantages, such as highly efficient Lanczos-based optimization pr…

Study critical metrics on Riemannian manifolds, finding new minimizers and rigidity results.

We present a general framework for classification of sparse and irregularly-sampled time series. The properties of such time series can result in substantial uncertainty about the values of the underlying temporal processes, while making the data difficult to deal with using standard classification methods that assume …

HessFormer enables distributed Hessian computation for large models.

The main purpose of this paper is to revisit the well known potentials, called stress functions, needed in order to study the parametrizations of the stress equations, respectively provided by G.B. Airy (1863) for 2-dimensional elasticity, then by E. Beltrami (1892), J.C. Maxwell (1870) and G. Morera (1892) for 3-dimen…

We introduce a weighted de Rham operator which acts on arbitrary tensor fields by considering their structure as r-fold forms. We can thereby define associated superpotentials for all tensor fields in all dimensions and, from any of these superpotentials, we deduce in a straightforward and natural manner the existence …

Unified framework detects overfitting in crash classification models.

Correlated anomaly detection (CAD) from streaming data is a type of group anomaly detection and an essential task in useful real-time data mining applications like botnet detection, financial event detection, industrial process monitor, etc. The primary approach for this type of detection in previous researches is base…

The purpose of this paper is to revisit the Bianchi identities existing for the Riemann and Weyl tensors in the combined framework of the formal theory of systems of partial differential equations (Spencer cohomology, differential systems, formal integrability) and Algebraic Analysis (homological algebra, differential …

Efficient approximations reduce computation of matrix-based Renyi's entropy.

New insights into learning rates and batch sizes for neural networks using random matrix theory.

Paper analyzes iterates in high-dimensional linear models and proposes estimators for their generalization error.

Paper analyzes iterative learning for concept classes and learns half-spaces.

WaveFit uses fixed-point iteration to create high-quality neural vocoders.

Last iterate of Extragradient algorithm converges slower than averaged iterates in saddle point problems.

In this paper, we introduce the notions of an iterated planar Lefschetz fibration and an iterated planar open book decomposition and prove the Weinstein conjecture for contact manifolds supporting an open book that has iterated planar pages. For $n\geq 1$, we show that a $(2n+1)$-dimensional contact manifold $M$ suppor…

We prove that an iterated torus knot type fails the uniform thickness property (UTP) if and only if all of its iterations are positive cablings, which is precisely when an iterated torus knot type supports the standard contact structure. We also show that all iterated torus knots that fail the UTP support cabling knot …

Sharp analysis of power iteration for tensor PCA, improving convergence and stopping criteria.

We deduce from the work of Chen, that the restriction morphism from closed free iterated integrals to closed iterated integrals on loops is onto. We use this to show that the module of higher order invariants of smooth functions is generated by free closed iterated integrals.