arXiv research

Scalable Gaussian processes with latent Kronecker structure for large datasets.

Efficiently models learning curves using Gaussian processes with latent Kronecker structure.

In recent years, a class of dictionaries have been proposed for multidimensional (tensor) data representation that exploit the structure of tensor data by imposing a Kronecker structure on the dictionary underlying the data. In this work, a novel algorithm called "STARK" is provided to learn Kronecker structured dictio…

In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatio-temporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. To allow a sm…

We investigate the structure of 3-dimensional complete minimal hypersurfaces in the unit sphere with Gauss-Kronecker curvature identically zero.

We consider the problem of detecting whether a tensor signal having many missing entities lies within a given low dimensional Kronecker-Structured (KS) subspace. This is a matched subspace detection problem. Tensor matched subspace detection problem is more challenging because of the intertwined signal dimensions. We s…

In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor subspaces, i.e. subspaces with a Kronecker structure, for embedding the data into lower dimensions. …

EiGLasso speeds up sparse Kronecker-sum covariance estimation.

How can we model networks with a mathematically tractable model that allows for rigorous analysis of network properties? Networks exhibit a long list of surprising properties: heavy tails for the degree distribution; small diameters; and densification and shrinking diameters over time. Most present network models eithe…

MCCA extracts shared structure from multiple tensor datasets.

The aim of this paper is two-fold. First, a survey of the theory of Kronecker webs and their relations with bihamiltonian structures and PDEs is presented. Second, a partial solution to the problem of bisymplectic realization of a bihamiltonian structure is given. Both the goals are achieved by means of the notion of a…

Researchers solve the realization of Jordan-Kronecker invariants in Lie algebras.

This paper derives sufficient conditions for local recovery of coordinate dictionaries comprising a Kronecker-structured dictionary that is used for representing $K$th-order tensor data. Tensor observations are assumed to be generated from a Kronecker-structured dictionary multiplied by sparse coefficient tensors that …

SINGD improves KFAC for memory-efficiency and stability in low-precision training.

Kronecker product kernel provides the standard approach in the kernel methods literature for learning from graph data, where edges are labeled and both start and end vertices have their own feature representations. The methods allow generalization to such new edges, whose start and end vertices do not appear in the tra…

New methods use Kronecker-factored approximations for faster deep learning optimization.

Study on rotational hypersurfaces with constant Gauss-Kronecker curvature.

This paper studies iteration convergence of Kronecker graphical lasso (KGLasso) algorithms for estimating the covariance of an i.i.d. Gaussian random sample under a sparse Kronecker-product covariance model and MSE convergence rates. The KGlasso model, originally called the transposable regularized covariance model by …

Machine learning predicts Kronecker coefficients with high accuracy.

Nonsingular estimation of high dimensional covariance matrices is an important step in many statistical procedures like classification, clustering, variable selection an future extraction. After a review of the essential background material, this paper introduces a technique we call slicing for obtaining a nonsingular …

DKN adapts to medical imaging data with limited samples and interpretable models.

Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner t…

Discovering the underlying low dimensional structure of high dimensional data has attracted a significant amount of researches recently and has shown to have a wide range of applications. As an effective dimension reduction tool, singular value decomposition is often used to analyze high dimensional matrices, which are…

Bayesian method estimates Kronecker graphical models from autoregressive processes.

TensorSketch is an oblivious linear sketch introduced in Pagh'13 and later used in Pham, Pagh'13 in the context of SVMs for polynomial kernels. It was shown in Avron, Nguyen, Woodruff'14 that TensorSketch provides a subspace embedding, and therefore can be used for canonical correlation analysis, low rank approximation…

KTVGL models tensor time series data for interpretable dynamic network estimation.

A matrix completion problem is to recover the missing entries in a partially observed matrix. Most of the existing matrix completion methods assume a low rank structure of the underlying complete matrix. In this paper, we introduce an alternative and more general form of the underlying complete matrix, which assumes a …

Paper relaxes independence assumption for non-centered data.

Second-order optimization methods such as natural gradient descent have the potential to speed up training of neural networks by correcting for the curvature of the loss function. Unfortunately, the exact natural gradient is impractical to compute for large models, and most approximations either require an expensive it…

We investigate 3-dimensional complete minimal hypersurfaces in the hyperbolic space $\mathbb{H}^{4}$ with Gauss-Kronecker curvature identically zero. More precisely, we give a classification of complete minimal hypersurfaces with Gauss-Kronecker curvature identically zero, nowhere vanishing second fundamental form and …

Gaussian Conditional Random Fields (GCRF), as a structured regression model, is designed to achieve higher regression accuracy than unstructured predictors at the expense of execution time, taking into account the objects similarities and the outputs of unstructured predictors simultaneously. As most structural models,…

The present paper discusses that a prescribed Gauss-Kronecker curvature problem on the product of unit spheres.

We introduce a framework and early results for massively scalable Gaussian processes (MSGP), significantly extending the KISS-GP approach of Wilson and Nickisch (2015). The MSGP framework enables the use of Gaussian processes (GPs) on billions of datapoints, without requiring distributed inference, or severe assumption…

We consider the problem of matrix approximation and denoising induced by the Kronecker product decomposition. Specifically, we propose to approximate a given matrix by the sum of a few Kronecker products of matrices, which we refer to as the Kronecker product approximation (KoPA). Because the Kronecker product is an ex…

The study predicts Kronecker coefficients using interpretable machine learning models.

This paper uncovers the low-rank structure of neural network Hessians.

We introduce the Kronecker factored online Laplace approximation for overcoming catastrophic forgetting in neural networks. The method is grounded in a Bayesian online learning framework, where we recursively approximate the posterior after every task with a Gaussian, leading to a quadratic penalty on changes to the we…

We investigate complete minimal hypersurfaces in the Euclidean space $$, with Gauss-Kronecker curvature identically zero. We prove that, if $f:M^{3}\to {R}^{4}$ is a complete minimal hypersurface with Gauss-Kronecker curvature identically zero, nowhere vanishing second fundamental form and scalar curvature b…

New method for faster graph parameter inference from large random Kronecker graphs.

In this paper we present a local description for complete minimal hypersurfaces in $\mathbb{S}^5$ with zero Gauss-Kronecker curvature, zero $3$-mean curvature and nowhere zero second fundamental form.

Efficiently solves high-dimensional ODEs with probabilistic methods.

Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices. To achieve this goal, we introduce a novel network reparameterization based on the Kronecker-factored eigenbasis (KFE), and then apply Hessian-based structure…

A parallel algorithm learns efficient Kronecker product dictionaries.

Kronecker Products (KP) have been used to compress IoT RNN Applications by 15-38x compression factors, achieving better results than traditional compression methods. However when KP is applied to large Natural Language Processing tasks, it leads to significant accuracy loss (approx 26%). This paper proposes a way to re…

Shampoo optimizes preconditioners for faster convergence in machine learning.

Totally geodesic minimal hypersurfaces in $\mathbb H^5$ with specific curvature properties.

Study mapping class group action on character varieties, proving Kronecker's Theorem.

A new method for optimizing deep neural networks using TKFAC.