EiGLasso speeds up sparse Kronecker-sum covariance estimation.
problem Sparse Kronecker-sum inverse covariance estimation challenges in scalability and parameter identification.
method Newton's method combined with eigendecomposition of sample and feature graphs, approximating Hessian for speed.
result Two to three orders-of-magnitude speed-up on simulated and real-world data.
GmGM models multi-axis data for faster analysis.
problem Efficiently modeling multi-axis data across multiple tensors.
method Generalizes Gaussian Graphical Model to learn sparse graph representations across shared axes.
result Achieves significant speedup (order of magnitude) for large multi-modal datasets.
Python package for manifold learning of millions of points.
problem Scalability of manifold learning algorithms for high-dimensional data.
method Modular, scalable implementation with fast approximate neighbors and sparse eigendecompositions.
result Embeds millions of data points in minutes, including a large dataset of galaxy spectra.
A new method for scalable spectral clustering using random binning features.
problem Scalability issues in spectral clustering for large-scale problems.
method Random Binning features to accelerate similarity graph construction and eigendecomposition.
result Achieves similar accuracy to standard spectral clustering but with linear computational cost.
A stable approach to eigendecomposition for deep learning networks.
problem Numerical instability in backpropagation of eigendecomposition results.
method A numerically stable and differentiable approach to eigendecomposition.
result Better robustness of the new approach over standard methods for ZCA whitening and PCA denoising.
EigenGame improves eigendecomposition by offering unbiased updates for larger datasets.
problem Minibatch bias in EigenGame limits convergence and parallelism.
method Proposed unbiased stochastic update for EigenGame.
result Asymptotic equivalence to EigenGame, greater parallelism, and improved performance.
GWNN uses graph wavelets for efficient graph CNNs.
problem Spectral graph CNNs' high computational cost and lack of interpretability.
method Graph wavelet transform for efficient graph convolution.
result GWNN significantly outperforms spectral graph CNNs.
Optimizes spectral density estimation for stationary and nonstationary processes.
problem Estimating spectral density of time series with complex structure.
method Optimally adaptive Bayesian spectral density estimation using smoothing spline covariance structure.
result Optimal eigendecomposition provides superior performance compared to alternative covariance functions.
New method for directed graphs using learnable spectral positional encodings.
problem Challenges in magnetic Laplacians and unitary gauge invariance for directed graphs.
method Learnable spectral PEs of the form hθ(Aq)R, computed in Hermitian block Krylov subspace.
result Gauge-invariant and computationally efficient solution for directed graphs.
A new method for spectral positional encodings in directed graphs using Hermitian block Krylov subspaces.
problem Challenges in spectral positional encodings for directed graphs, including computational complexity and gauge invariance issues.
method Learnable spectral positional encodings of the form hθ(Aq)R, computed in a Hermitian block Krylov subspace from sparse matrix-vector products. result The method is gauge-invariant and converges to the exact eigendecomposition oracle as the depth grows.
Study on negative eigenvalues in deep neural networks' loss landscapes.
problem Understanding the non-convex nature of deep neural networks' loss functions.
method Examined the Hessian matrix's eigendecompositions to analyze negative eigenvalues.
result Negative eigenvalues are crucial for understanding the loss landscape of deep networks.
Paper speeds up and extends Subclass Discriminant Analysis methods.
problem Improving efficiency and handling multi-view data in Subclass Discriminant Analysis.
method Developed a speed-up approach based on graph embedding and spectral regression, and a novel multi-view solution.
result Proposed methods achieve competitive performance and significantly decrease training time.
We propose a general matrix-valued multiple kernel learning framework for high-dimensional nonlinear multivariate regression problems. This framework allows a broad class of mixed norm regularizers, including those that induce sparsity, to be imposed on a dictionary of vector-valued Reproducing Kernel Hilbert Spaces. W…
We propose a general matrix-valued multiple kernel learning framework for high-dimensional nonlinear multivariate regression problems. This framework allows a broad class of mixed norm regularizers, including those that induce sparsity, to be imposed on a dictionary of vector-valued Reproducing Kernel Hilbert Spaces. W…
Two proxy methods for causal identification are compared.
problem Identifying causal effects in the presence of unmeasured variables.
method Bridge equation methods vs. array decomposition methods.
result Model restrictions and implications of assumptions differ between methods.
Modified training direction reduces generalization error in neural networks.
problem Reducing generalization error in neural networks.
method Theoretical analysis of modified natural gradient descent in function space.
result Modifying training direction in function space reduces total generalization error.
Second-order optimizers retain residual information after data deletion, affecting machine unlearning.
problem Residual information in second-order optimizers after data deletion.
method Comparison of first-order and second-order learners, eigendecomposition analysis.
result Second-order optimizers retain residual information, not detectable by first-order analysis.
Neumann eigenmaps improve landmark-based diffusion map embeddings.
problem Landmark-based diffusion map embeddings can be computationally inefficient and unstable.
method NeuMaps use a renormalized Neumann Laplacian for eigendecomposition, incorporating landmarks as a subgraph.
result NeuMaps offer a computationally efficient and stable embedding method.
We propose a fast algorithm for spectral embedding using stochastic gradient descent.
problem Scalability issue in spectral embedding due to eigendecomposition bottleneck.
method Reformulate spectral embedding as a stochastic optimization problem, replacing orthogonality constraint with an orthogonalization matrix.
result Efficient algorithm based on mini-batch gradient descent that outperforms existing techniques in execution speed.
New metric measures dynamical richness without relying on accuracy.
problem Lack of a reliable metric for measuring dynamical richness.
method Developed a computationally efficient, performance-independent metric based on low-rank bias.
result Metric recovers neural collapse as a special case and captures known transitions without accuracy.
A new method removes whitening for better non-Gaussian component analysis.
problem Data covariance matrix ill-conditioning hinders LSNGCA performance.
method Developed a whitening-free least-squares NGCA method.
result Demonstrated superior performance compared to whitened LSNGCA.
New scalable methods for learning with indefinite kernels.
problem Learning with indefinite kernels, especially for structured data.
method Derivation of Nyström method, efficient eigendecomposition, scalable learning methods.
result Principled and theoretically well-founded means for large-scale learning problems.
Ginger efficiently approximates curvature with linear complexity for neural networks.
problem Quadratic memory and cubic time complexity for computing curvature matrices in deep learning.
method Ginger uses eigendecomposition to maintain the inverse of the generalized Gauss-Newton matrix, achieving linear memory and time complexity.
result Ginger provides an effective and efficient curvature approximation for non-convex objectives.
CovNet models covariance for multidimensional functional data efficiently.
problem Estimating covariance for functional data over multidimensional domains.
method Covariance Networks (CovNet) for efficient modeling and estimation.
result CovNet can approximate any covariance up to desired precision efficiently.
Generative model improves word embeddings with latent factors.
problem Incorporating latent factors into word embeddings.
method Generative model with low rank positive semidefinite approximation.
result Generative model outperforms existing methods on benchmark datasets.
A new method samples DPPs without eigendecomposition, making it faster.
problem Sampling DPPs efficiently when dealing with many points.
method Uses Cholesky decompositions and a thinning procedure.
result The method avoids costly eigenvalue/eigenvector computations.
Graph pooling method uses GNN to cluster graphs efficiently.
problem Efficiently cluster graphs for graph pooling.
method Formulates a continuous relaxation of minCUT, trains GNN to minimize it.
result GNN-based clustering learns fast and performs well.
Bilinear MLPs offer a new way to interpret deep learning models without complex nonlinearities.
problem Lack of mechanistic understanding in how MLPs compute.
method Introduced bilinear MLPs without element-wise nonlinearities, analyzed their weights using tensor and eigendecomposition.
result Bilinear MLPs provide interpretable weight structures and enable adversarial attacks and overfitting analysis.
New algorithms for sampling DPPs reduce computational cost.
problem Sampling DPPs efficiently with reduced computational cost.
method Three-phase algorithm with eigendecomposition, eigenvector sampling, and orthogonalisation. Dual variant for memory constraints.
result Algorithm reduces computational complexity from O(Nμ3) to O(Nμ2). TL-PCA uses transfer learning to improve PCA performance with limited target data.
problem PCA performance is limited with scarce target data.
method Transfer learning approach to PCA (TL-PCA) that combines source task knowledge with target task data.
result Improved PCA representation for dimensionality reduction with limited target data.
New method for testing directed graphs using surrogate data.
problem No established method for statistical testing on directed graphs.
method Define directed graph wide-sense stationary signals, generate surrogates preserving covariance, construct null distributions.
result Feasibility and superiority of new approach over existing methods.
New algorithm updates eigenvectors of evolving graphs efficiently.
problem Updating eigenvectors of dynamic graphs.
method Subspace projection based on Rayleigh-Ritz projections.
result Strong performance in eigenvector approximation and downstream tasks.
Improved statistical computation through efficient matrix sampling.
problem Reducing computational cost in large-scale statistical methods.
method Accumulative sub-sampling method to improve statistical efficiency.
result Effective matrix size control improves computational efficiency.
A new algorithm for PCA reduces computation time while maintaining good generalization performance.
problem Efficiently performing Principal Component Analysis (PCA) on matrix data.
method Follow the Perturbed Leader paradigm, using Gaussian noise to approximate eigendecompositions.
result Achieves a small factor of O(n1/4) off the optimum regret with O(kn2) per trial. This paper proposes a novel kernel approach to linear dimension reduction for supervised learning. The purpose of the dimension reduction is to find directions in the input space to explain the output as effectively as possible. The proposed method uses an estimator for the gradient of regression function, based on the…
LASE learns graph embeddings by unrolling GD iterations into a neural network.
problem Learning graph embeddings from graph inputs.
method Truncating and interpreting GD iterations as layers in a GNN.
result LASE outperforms eigendecomposition routines and GNNs in link prediction and node classification.
Semi-supervised clustering aims to introduce prior knowledge in the decision process of a clustering algorithm. In this paper, we propose a novel semi-supervised clustering algorithm based on the information-maximization principle. The proposed method is an extension of a previous unsupervised information-maximization …
The paper investigates the convergence of Vendi scores under finite samples and introduces a truncated version for better performance.
problem The Vendi score's convergence is hindered by computational limitations when using large sample sizes.
method The authors introduce the t-truncated Vendi score to address this issue by truncating the eigenspectrum of the kernel matrix.
result The t-truncated Vendi score converges to its asymptotic limit with a smaller number of samples, improving upon the standard Vendi score.
In a Bayesian learning setting, the posterior distribution of a predictive model arises from a trade-off between its prior distribution and the conditional likelihood of observed data. Such distribution functions usually rely on additional hyperparameters which need to be tuned in order to achieve optimum predictive pe…
Unified framework for differentiable graph partitioning with probabilistic cuts.
problem Lack of general guarantees and principled gradients in prior probabilistic relaxations of graph cuts.
method Unified probabilistic framework covering a wide class of cuts, including Normalized Cut, with tight analytic upper bounds.
result Rigorous, numerically stable foundation for scalable, differentiable graph partitioning.
Solves kernel dimension reduction while making features interpretable.
problem Making kernel dimension reduction methods interpretable.
method Projects onto a subspace before kernel feature mapping, using ISM for optimization.
result Extends ISM's theoretical guarantees to a family of kernels, enabling broader applicability.
Determinantal point processes (DPPs) are well-suited for modeling repulsion and have proven useful in many applications where diversity is desired. While DPPs have many appealing properties, such as efficient sampling, learning the parameters of a DPP is still considered a difficult problem due to the non-convex nature…
A new algorithm for faster incremental kernel PCA and Nyström approximation.
problem Efficiency in handling streaming data and large datasets.
method Incremental kernel PCA and Nyström approximation using rank one updates.
result More computationally and memory efficient than existing methods.
Scalability of statistical estimators is of increasing importance in modern applications and dimension reduction is often used to extract relevant information from data. A variety of popular dimension reduction approaches can be framed as symmetric generalized eigendecomposition problems. In this paper we outline how t…
The paper proves a non-asymptotic test error approximation for KRR.
problem Understanding the test error of Kernel Ridge Regression.
method Established a non-asymptotic deterministic approximation for test error of KRR.
result The test error of KRR can be approximated by a closed-form estimate derived from the spectrum of the kernel operator.
This paper speeds up mean curvature computation for high-dimensional data.
problem Efficiently computing mean curvature in high-dimensional datasets.
method Two contributions: algebraic identity and truncated SVD approximation.
result Mean curvature computation reduced from O(m4) to O(k2m+kmp2). Deep kernel learning for clustering improves on spectral methods.
problem Discovering effective kernels for clustering.
method Neural network producing embeddings motivated by spectral clustering, optimized via gradient adaptations on the Stiefel manifold.
result Trained embeddings outperform state-of-the-art deep clustering methods and traditional approaches.
New algorithms solve k-GenEV and k-CCA problems faster.
problem Finding top k generalized eigenvectors and canonical-correlation vectors.
method Proposed LazyEV and LazyCCA algorithms with linear dependence on input size and k. result First doubly-accelerated algorithms with running times independent of matrix condition number and eigengap.