Accelerates optimal transport computation by 10x with spectral insights.
problem Exponential slow-down of convergence in Entropic Optimal Transport as regularization weakens.
method Spectral insights and spectral warm-start strategy to mitigate convergence issues.
result Faster convergence compared to the reference method Sinkhorn algorithm.
The paper explores the Rumin complex and spectral sequence on Carnot groups.
problem Understanding the relationship between Rumin complex and spectral sequence on Carnot groups.
method Investigates the Rumin complex and spectral sequence on Carnot groups, focusing on the filtration by homogeneous weights.
result Provides a detailed insight into the relationship between the Rumin complex and the spectral sequence on Carnot groups.
Rewiring GNNs to optimize community and feature alignment improves their performance.
problem Improving GNNs' performance by addressing over-squashing and generalization issues.
method Three rewiring strategies: ComMa, FeaSt, and ComFy, targeting community structure, node labels, and their alignment.
result Rewiring strategies enhance GNNs' performance by optimizing label-community alignment.
Word2vec analysis reveals spectral underpinnings.
problem Lack of theoretical justification for word2vec.
method Rigorous spectral analysis of word2vec's nonlinear functional.
result Word2vec may be primarily driven by spectral method.
Graph spectral analysis can yield meaningful embeddings of graphs by providing insight into distributed features not directly accessible in nodal domain. Recent efforts in graph signal processing have proposed new decompositions-e.g., based on wavelets and Slepians-that can be applied to filter signals defined on the g…
Recently, there is increasing interest and research on the interpretability of machine learning models, for example how they transform and internally represent EEG signals in Brain-Computer Interface (BCI) applications. This can help to understand the limits of the model and how it may be improved, in addition to possi…
Clustering is fundamental for gaining insights from complex networks, and spectral clustering (SC) is a popular approach. Conventional SC focuses on second-order structures (e.g., edges connecting two nodes) without direct consideration of higher-order structures (e.g., triangles and cliques). This has motivated SC ext…
Paper explores SNN for learning spectral geometric info from data.
problem Challenges in applying traditional eigensolvers to big data.
method Introduces Spectral Neural Networks (SNN) as an alternative.
result Investigates tradeoffs and optimization landscape of SNN.
Spectral Method is a commonly used scheme to cluster data points lying close to Union of Subspaces by first constructing a Random Geometry Graph, called Subspace Clustering. This paper establishes a theory to analyze this method. Based on this theory, we demonstrate the efficiency of Subspace Clustering in fairly broad…
Muon optimizer simplifies matrix optimization with spectral orthogonalization.
problem Matrix optimization challenges, especially with large condition numbers.
method Simplified Muon optimizer using spectral orthogonalization of gradients.
result Simplified Muon converges linearly with independent scalar sequences, outperforming gradient descent and Adam.
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
Improved neural network predicts spectral functions more accurately than traditional methods.
problem Reconstructing real-time spectral functions from imaginary-time Green's functions is ill-posed and challenging.
method Feature Learning Network (FL-net) for enhanced prediction accuracy.
result FL-net achieves at least 20% improvement over traditional methods like MEM.
A framework uses free probability to analyze Transformer models.
problem Understanding the dynamics and complexity of Transformer-based language models.
method Formal operator-theoretic analysis using free probability theory.
result Entropy-based generalization bounds derived under freeness assumptions.
Interpolates mean shift and spectral clustering on graphs.
problem Data clustering algorithms.
method Fokker-Planck equations on data graphs.
result New theoretical insights on diffusion maps and mean shift dynamics.
Graph pruning improves neural network performance by addressing squashing and smoothing issues.
problem Over-squashing and over-smoothing in Graph Neural Networks.
method Proposes edge deletions to simultaneously address over-squashing and over-smoothing, optimizing spectral gap.
result Edge deletions improve generalization and distinguishability of nodes of different classes.
We analyze DMs using spectral methods to design effective noise schedules.
problem Lack of theoretical foundation for synthesis process decisions in DMs.
method Introduced a frequency response perspective based on Gaussianity assumption.
result Proposed a spectral transfer function to understand DM inference process.
Proposes a deep network for multi-class classification using spectral training and Gaussian kernel.
problem Multi-class classification with deep networks.
method Spectral training with linear weights and Gaussian kernel activation, constrained on Stiefel Manifold.
result Theoretical guarantee of global optimum and insight into network generalization.
This paper re-visits the spectral method for learning latent variable models defined in terms of observable operators. We give a new perspective on the method, showing that operators can be recovered by minimizing a loss defined on a finite subset of the domain. A non-convex optimization similar to the spectral method …
Study clusters Indian stocks using polyspectral means for nuanced market insights.
problem Analyzing temporal patterns and financial relationships in Indian stock market.
method k-means clustering algorithm applied to polyspectral means of stock data.
result Identified five distinctive clusters of stocks with varying ownership structures.
New insights into spectral clustering reveal strong connections within eigenvectors.
problem Clustering on graphs when there are two underlying clusters.
method Analyzes the eigenvector corresponding to the second largest eigenvalue of the adjacency matrix.
result Vertices with extreme values in the eigenvector are more reliably classified.
Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.
problem Stability and sample quality issues in GAN training.
method Spectral normalization controls gradient explosion and vanishing, improving GAN training stability and sample quality.
result Bidirectional Scaled Spectral Normalization (BSSN) outperforms standard spectral normalization in sample quality and training stability.
Magnitude of geometric shapes studied for smooth manifolds, revealing spectral geometry insights.
problem Understanding the geometric significance of Leinster's magnitude for smooth manifolds.
method Investigation of magnitude function for various distance functions, including submanifolds and Riemannian manifolds, with asymptotic analysis in the limit.
result Magnitude function is well-defined and meromorphically continued for large distances, revealing volume, surface area, and curvature integrals.
Spectral feature learning improves IV regression for causal effect estimation.
problem Estimating causal effects in the presence of hidden confounders.
method Two-stage least squares estimator based on spectral features.
result Performance of the method depends on strong spectral alignment and slow eigenvalue decay.
Improved spectral clustering via Gromov-Wasserstein Learning.
problem Optimizing graph partitioning performance.
method Bridge spectral clustering and GWL, using heat kernel for stable node correspondences.
result Improved graph partitioning results without compromising theoretical guarantees.
The paper proves criticality criteria and spectral splitting theorems for manifolds with Ricci bounds.
problem Understanding criticality and splitting theorems for manifolds with spectral Ricci bounds.
method Proving criticality criteria and spectral splitting theorems for manifolds with more than one end and spectral Ricci bounds.
result New insights into Li-Wang's theory and applications to stable and δ-stable minimal hypersurfaces.
Biological and social systems consist of myriad interacting units. The interactions can be represented in the form of a graph or network. Measurements of these graphs can reveal the underlying structure of these interactions, which provides insight into the systems that generated the graphs. Moreover, in applications s…
The problem of Hybrid Linear Modeling (HLM) is to model and segment data using a mixture of affine subspaces. Different strategies have been proposed to solve this problem, however, rigorous analysis justifying their performance is missing. This paper suggests the Theoretical Spectral Curvature Clustering (TSCC) algori…
The inverse covariance matrix provides considerable insight for understanding statistical models in the multivariate setting. In particular, when the distribution over variables is assumed to be multivariate normal, the sparsity pattern in the inverse covariance matrix, commonly referred to as the precision matrix, cor…
Improves few-shot learning using multi-task representation learning theory.
problem Few-shot learning with limited data.
method Multi-task representation learning theory and spectral-based regularization.
result Improved performance of meta-learning methods through new spectral regularization.
Study reveals class disparities in balanced datasets through spectral imbalance.
problem Class disparities in balanced datasets are overlooked despite model performance gaps.
method Developed a theoretical framework and studied 11 encoders to diagnose spectral imbalance.
result Identified spectral imbalance as a source of class disparities in balanced datasets.
With inspiration from Random Forests (RF) in the context of classification, a new clustering ensemble method---Cluster Forests (CF) is proposed. Geometrically, CF randomly probes a high-dimensional data cloud to obtain "good local clusterings" and then aggregates via spectral clustering to obtain cluster assignments fo…
This paper proposes a new approach to construct high quality space-filling sample designs. First, we propose a novel technique to quantify the space-filling property and optimally trade-off uniformity and randomness in sample designs in arbitrary dimensions. Second, we connect the proposed metric (defined in the spatia…
We introduce the convolutional spectral kernel (CSK), a novel family of non-stationary, nonparametric covariance kernels for Gaussian process (GP) models, derived from the convolution between two imaginary radial basis functions. We present a principled framework to interpret CSK, as well as other deep probabilistic mo…
Networks or graphs can easily represent a diverse set of data sources that are characterized by interacting units or actors. Social networks, representing people who communicate with each other, are one example. Communities or clusters of highly connected actors form an essential feature in the structure of several emp…
FreDN separates trends and periodicities in non-stationary time series forecasts.
problem Spectral entanglement and computational burden in frequency-domain methods for non-stationary time series.
method FreDN introduces a learnable Frequency Disentangler module to separate trend and periodic components directly in the frequency domain, and uses a ReIm Block to reduce complexity.
result FreDN outperforms state-of-the-art methods by up to 10% on long-term forecasting benchmarks.
Paper provides unbiased spectral moment estimates from finite data.
problem Challenges in estimating spectral moments from limited data.
method Dynamic programming approach to estimate spectral moments of kernel integral operator.
result Demonstrates consistency with theoretical spectra and practical utility in neural networks.
Study of meromorphic connections and their spectral duals in gl3(C).
problem Exploring ℏ-deformed meromorphic connections and their spectral duals. method Using apparent singularities and their dual partners as Darboux coordinates, the Hamiltonian evolutions and reductions are derived.
result Spectral duality extends to Hamiltonian evolutions, tau-functions, and Hermitian matrix models on both sides.
Study identifies cancer genes through graph anomaly analysis of protein interactions.
problem Insufficient modeling of biological information in protein interaction networks for cancer gene identification.
method Proposes HIerarchical-Perspective Graph Neural Network (HIPGNN) to detect weight heterogeneity and spectral flattening in cancer gene nodes.
result HIPGNN detects weight heterogeneity and spectral flattening, leading to improved cancer gene identification.
This paper focuses on obtaining clustering information about a distribution from its i.i.d. samples. We develop theoretical results to understand and use clustering information contained in the eigenvectors of data adjacency matrices based on a radial kernel function with a sufficiently fast tail decay. In particular, …
In this paper, we provide a novel construction of the linear-sized spectral sparsifiers of Batson, Spielman and Srivastava [BSS14]. While previous constructions required Ω(n4) running time [BSS14, Zou12], our sparsification routine can be implemented in almost-quadratic running time O(n2+ε). The funda…
We present an algorithm for extraction of a probabilistic deterministic finite automaton (PDFA) from a given black-box language model, such as a recurrent neural network (RNN). The algorithm is a variant of the exact-learning algorithm L*, adapted to a probabilistic setting with noise. The key insight is the use of con…
ULES embeds dynamic networks with stability guarantees.
problem Stability of time-varying node embeddings in evolving networks.
method Unfolded Laplacian Spectral Embedding (ULSE) using normalized Laplacian operators.
result ULES satisfies cross-sectional and longitudinal stability under dynamic stochastic block model.
Spectro-Riemannian Graph Neural Networks integrate spectral and curvature signals for better graph representation learning.
problem Enhance graph representation learning by leveraging spectral and curvature signals.
method Proposes Spectro-Riemannian Graph Neural Networks (CUSP) that combines spectral and curvature insights.
result Empirical evaluation shows CUSP outperforms state-of-the-art models by up to 5.3%.
Improves regression models' performance on covariate shift.
problem Out-of-distribution generalization for regression.
method Spectrally adapting the weights of a pre-trained neural regression model.
result Spectral adaptation improves out-of-distribution performance.
Two spectral clustering methods for multi-layer networks are analyzed and compared.
problem Community detection in multi-layer networks.
method Sum and debiased sum of squared adjacency matrices for spectral clustering.
result Debiased sum of squared adjacency matrices outperforms sum of adjacency matrices.
SEDA improves RLDA for high-dimensional data.
problem Inconsistent performance of RLDA in high-dimensional scenarios.
method Developed a non-asymptotic approximation of misclassification rate, derived new theoretical results on eigenvectors, and proposed SEDA algorithm.
result SEDA achieves higher classification accuracy and dimensionality reduction compared to existing LDA methods.
Graph clustering is a basic technique in machine learning, and has widespread applications in different domains. While spectral techniques have been successfully applied for clustering undirected graphs, the performance of spectral clustering algorithms for directed graphs (digraphs) is not in general satisfactory: the…
Are neural networks biased toward simple functions? Does depth always help learn more complex features? Is training the last layer of a network as good as training all layers? How to set the range for learning rate tuning? These questions seem unrelated at face value, but in this work we give all of them a common treat…