New method for faster graph parameter inference from large random Kronecker graphs.
problem Efficiently infer graph parameters from large random Kronecker graphs.
method Decompose adjacency matrix into signal and noise components, then use denoising and solving approach.
result Proposed method achieves comparable or better performance than existing methods at lower computational cost.
How can we model networks with a mathematically tractable model that allows for rigorous analysis of network properties? Networks exhibit a long list of surprising properties: heavy tails for the degree distribution; small diameters; and densification and shrinking diameters over time. Most present network models eithe…
Stochastic Kronecker graphs supply a parsimonious model for large sparse real world graphs. They can specify the distribution of a large random graph using only three or four parameters. Those parameters have however proved difficult to choose in specific applications. This article looks at method of moments estimators…
EiGLasso speeds up sparse Kronecker-sum covariance estimation.
problem Sparse Kronecker-sum inverse covariance estimation challenges in scalability and parameter identification.
method Newton's method combined with eigendecomposition of sample and feature graphs, approximating Hessian for speed.
result Two to three orders-of-magnitude speed-up on simulated and real-world data.
Kronecker product kernel provides the standard approach in the kernel methods literature for learning from graph data, where edges are labeled and both start and end vertices have their own feature representations. The methods allow generalization to such new edges, whose start and end vertices do not appear in the tra…
Improved GCRF model for multi-scale networks with reduced computation time.
problem Scaling issues in GCRF for large networks.
method Factor graph decomposition using Kronecker product, new Laplacian eigenvalue estimations.
result High prediction accuracy with improved computational complexity.
We describe the first sub-quadratic sampling algorithm for the Multiplicative Attribute Graph Model (MAGM) of Kim and Leskovec (2010). We exploit the close connection between MAGM and the Kronecker Product Graph Model (KPGM) of Leskovec et al. (2010), and show that to sample a graph from a MAGM it suffices to sample sm…
Proposes a novel graph learning framework for robust graph topology learning from graph signals.
problem Graph learning for revealing node relationships in data entities.
method Functional learning with smoothness-promoting graph learning, incorporating Kronecker product kernel.
result Improves robustness against missing and incomplete information in graph signals.
Researchers solve the realization of Jordan-Kronecker invariants in Lie algebras.
problem Identifying which Jordan-Kronecker invariants can be realized by Lie algebras.
method Analyzing the Kronecker and Jordan cases, proving impossibility for certain invariants, and describing realizability for others.
result Complete solution for Jordan and Kronecker cases, partial answers for others.
K-FAC speeds up training of modern neural networks with linear weight-sharing.
problem Efficiently training modern neural networks with linear weight-sharing layers.
method Kronecker-Factored Approximate Curvature (K-FAC) applied to linear weight-sharing layers.
result K-FAC-reduce is generally faster than K-FAC-expand for deep linear networks.
New method infers graph from dependent matrix data.
problem Inferring graph from dependent matrix data.
method Sparse-group lasso-based frequency-domain formulation with ADMM approach.
result Local convergence of inverse PSD estimators to true value.
Study on rotational hypersurfaces with constant Gauss-Kronecker curvature.
problem Exploring hypersurfaces with constant Gauss-Kronecker curvature.
method Solving ODE for generating curves and analyzing geometric properties.
result Discovery of non-compact rotational hypersurfaces with negative Gauss-Kronecker curvature and finite volume.
Machine learning predicts Kronecker coefficients with high accuracy.
problem Predicting Kronecker coefficients from tensor products of symmetric group representations.
method Training machine learning models (NN, CNN, GBDT) to classify Kronecker coefficients as zero or non-zero.
result Trained models achieve high accuracy (≈0.98) in classifying Kronecker coefficients. Bayesian method estimates Kronecker graphical models from autoregressive processes.
problem Estimating Kronecker graphical models from autoregressive Gaussian processes.
method Bayesian approach to estimate Kronecker graphical models.
result Effectiveness demonstrated through numerical experiments and real-world data application.
Graphs from van der Corput sequence embed into Chamanara surface.
problem Embedding graphs from van der Corput sequence into surfaces.
method Constructed 4-regular graphs from van der Corput sequence and Kronecker sequence, embedded into torus and Chamanara surface. result Graphs from van der Corput sequence embed into Chamanara surface with one edge removal.
TensorSketch is an oblivious linear sketch introduced in Pagh'13 and later used in Pham, Pagh'13 in the context of SVMs for polynomial kernels. It was shown in Avron, Nguyen, Woodruff'14 that TensorSketch provides a subspace embedding, and therefore can be used for canonical correlation analysis, low rank approximation…
KoPA approximates matrices using Kronecker products for better flexibility.
problem Matrix approximation and denoising with Kronecker product decomposition.
method Approximate a matrix as a sum of Kronecker products of smaller matrices using extended information criteria for configuration selection.
result KoPA selects the true configuration with high probability under suitable conditions.
Scalable Gaussian processes with latent Kronecker structure for large datasets.
problem Limited scalability of Gaussian processes for large datasets.
method Leveraging latent Kronecker structure, projecting kernel matrix onto latent Kronecker product, using iterative linear system solvers and pathwise conditioning.
result Outperforms state-of-the-art sparse and variational GPs on real-world datasets with up to five million examples.
In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatio-temporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. To allow a sm…
We investigate 3-dimensional complete minimal hypersurfaces in the hyperbolic space H4 with Gauss-Kronecker curvature identically zero. More precisely, we give a classification of complete minimal hypersurfaces with Gauss-Kronecker curvature identically zero, nowhere vanishing second fundamental form and …
New method for scalable stochastic neural networks using Kronecker Flow.
problem Scaling stochastic neural networks to high dimensions.
method Kronecker Flow, a scalable parameterization of noise generation.
result Competitive performance on various tasks compared to existing methods.
The present paper discusses that a prescribed Gauss-Kronecker curvature problem on the product of unit spheres.
SyGlasso models tensor data dependencies using Sylvester equations.
problem Capturing multiway dependencies in tensor-valued data.
method Based on Sylvester equation, uses nodewise regression for estimation.
result Demonstrates simultaneous estimation of brain connectivity and temporal dependencies.
Efficiently models learning curves using Gaussian processes with latent Kronecker structure.
problem Joint modeling of machine learning model performance across hyper-parameters and training progress.
method Imposes latent Kronecker structure to leverage efficient product kernels and handle missing values.
result Matches the performance of a Transformer on a learning curve prediction task.
We investigate the structure of 3-dimensional complete minimal hypersurfaces in the unit sphere with Gauss-Kronecker curvature identically zero.
Paper proposes a new method for approximating high-dimensional matrices using Kronecker products.
problem Discovering low-dimensional structure in high-dimensional data.
method Hybrid Kronecker Product Approximation (hKoPA) and estimation procedures.
result The proposed methods provide flexible and effective dimension reduction.
Expands small recommendation datasets to industrial scale.
problem Disconnection between academic and industrial data scales.
method Randomized fractal expansions using Kronecker Graph Theory.
result Generated synthetic data sets with 1.2B ratings, 2.2M users, and 855K items.
The study predicts Kronecker coefficients using interpretable machine learning models.
problem Predicting Kronecker coefficients of the symmetric group.
method Employed interpretable machine learning models with input features of triples of partitions and b-loadings.
result Achieved an accuracy of approximately 83% and over 99% with transformer-based models.
We investigate complete minimal hypersurfaces in the Euclidean space , with Gauss-Kronecker curvature identically zero. We prove that, if f:M3→R4 is a complete minimal hypersurface with Gauss-Kronecker curvature identically zero, nowhere vanishing second fundamental form and scalar curvature b…
This paper studies iteration convergence of Kronecker graphical lasso (KGLasso) algorithms for estimating the covariance of an i.i.d. Gaussian random sample under a sparse Kronecker-product covariance model and MSE convergence rates. The KGlasso model, originally called the transposable regularized covariance model by …
We consider the problem of detecting whether a tensor signal having many missing entities lies within a given low dimensional Kronecker-Structured (KS) subspace. This is a matched subspace detection problem. Tensor matched subspace detection problem is more challenging because of the intertwined signal dimensions. We s…
In recent years, a class of dictionaries have been proposed for multidimensional (tensor) data representation that exploit the structure of tensor data by imposing a Kronecker structure on the dictionary underlying the data. In this work, a novel algorithm called "STARK" is provided to learn Kronecker structured dictio…
In this paper we present a local description for complete minimal hypersurfaces in S5 with zero Gauss-Kronecker curvature, zero 3-mean curvature and nowhere zero second fundamental form.
New method for matrix completion using Kronecker product approximation.
problem Matrix completion with low Kronecker rank structure.
method Alternative matrix representation using Kronecker product, identification through mean squared error and modified cross-validation.
result Consistency of the method under suitable signal-to-noise ratio conditions.
A parallel algorithm learns efficient Kronecker product dictionaries.
problem Sparse representation of 2D signals like images and hyperspectral data.
method Highly parallelizable algorithm for learning separable dictionaries.
result Competitive sparse representations at lower computational cost.
Shampoo optimizes preconditioners for faster convergence in machine learning.
problem Improving convergence speed in machine learning optimization.
method Explicit connection between Shampoo's Kronecker product approximation and optimal matrix approximations.
result The square of Shampoo's approximation is equivalent to a single power iteration step for optimal Kronecker product approximation.
Totally geodesic minimal hypersurfaces in H5 with specific curvature properties.
problem Characterizing minimal hypersurfaces in hyperbolic space with certain curvature conditions.
method Analyzing properties of minimal hypersurfaces in H5 with constant scalar curvature and zero Gauss-Kronecker curvature. result Any complete minimal hypersurface in H5 with constant scalar curvature and zero Gauss-Kronecker curvature is totally geodesic. Study mapping class group action on character varieties, proving Kronecker's Theorem.
problem Topological-dynamical action of mapping class group on character varieties.
method Analyzes Tn-character variety and dense orbit conditions. result Provides a dynamical proof of Kronecker's Theorem.
A new method for optimizing deep neural networks using TKFAC.
problem Optimizing deep neural networks with second-order methods.
method Proposes Trace-restricted Kronecker-factored Approximate Curvature (TKFAC) for Fisher information matrix approximation.
result TKFAC improves performance on deep network architectures compared to state-of-the-art algorithms.
We give a partial local description of minimal hypersurfaces M3 with identically zero Gauß-Kronecker curvature function in the unit 4-sphere S4(1), without assumption on the compactness of M3.
MCCA extracts shared structure from multiple tensor datasets.
problem Extracting shared structure from multiple tensor datasets.
method Multilinear common component analysis (MCCA) using Kronecker products of mode-wise covariance matrices.
result MCCA constructs a common basis that retains information from multiple tensor datasets.
The aim of this paper is to complete the local classification of minimal hypersurfaces with vanishing Gauss-Kronecker curvature in a 4-dimensional space form. Moreover, we give a classification of complete minimal hypersurfaces with vanishing Gauss-Kronecker curvature and scalar curvature bounded from below.
In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor subspaces, i.e. subspaces with a Kronecker structure, for embedding the data into lower dimensions. …
This paper improves compression of large NLP models using doped Kronecker Products.
problem Accuracy loss when compressing large NLP tasks with Kronecker Products.
method Doping Kronecker Products with an overlay matrix to recover accuracy, and a new regularization scheme called co matrix dropout regularization (CMR).
result Compression of a large language model with LSTM layers of size 25 MB by 25x with 1.4% loss in perplexity score.
In this paper we present two new approaches to efficiently solve large-scale compressed sensing problems. These two ideas are independent of each other and can therefore be used either separately or together. We consider all possibilities. For the first approach, we note that the zero vector can be taken as the initial…
EigenDamage reduces neural network size and FLOPs with structured pruning in the Kronecker-Factored Eigenbasis.
problem Reducing neural network size and FLOPs while maintaining accuracy for resource-constrained devices.
method Kronecker-Factored Eigenbasis reparameterization and Hessian-based structured pruning.
result Empirically validated improvements in model size and FLOPs with negligible accuracy loss.
Kronecker trend filtering improves lattice data smoothing.
problem Estimating smooth functions on lattice data.
method Penalized least squares with Kronecker products of univariate trend filtering penalties.
result Kronecker trend filtering outperforms linear smoothers in high dimensions.
A new optimization method reduces memory and compute requirements for deep learning.
problem Memory and compute constraints in second-order stochastic optimizers for deep learning.
method Proposes KrAD, a novel factorization to approximate inverse Fisher matrix without inversion, leading to KrADagrad.
result Improves performance over Shampoo for 32-bit precision and comparable/generalization on real datasets.