The paper bridges spectral and spatial graph convolutions, improving model capacity and transferability.
problem Improving graph neural networks by bridging spectral and spatial design.
method Theoretical demonstration and general framework for spectral analysis, new spectral convolutions, and depthwise separable convolutions.
result General framework allows spectral analysis of ConvGNNs, showing their performance and limits, and proposing new spectral convolutions.
A new method speeds up spectral normalization for neural nets.
problem Efficiently controlling the spectral norm of convolutional layers.
method Depthwise separable convolutions with spectral normalization.
result Significant reduction in computational and memory costs.
Improved singular value approximation for convolutional layers.
problem Improving accuracy of singular value approximation for linear convolutional layers.
method Developed a new spectral density matrix method for singular value approximation with improved accuracy and reduced computational complexity.
result Obtained moderate improvement in singular value distribution compared to circular approximation.
Previous research has shown that computation of convolution in the frequency domain provides a significant speedup versus traditional convolution network implementations. However, this performance increase comes at the expense of repeatedly computing the transform and its inverse in order to apply other network operati…
Graph convolutional networks fail to use eigenvectors beyond the first, unlike spectral embedding.
problem Understanding when graph convolutional networks fail compared to spectral embedding.
method Presented a simple generative model to illustrate failure.
result Graph convolutional networks fail to use eigenvectors beyond the first in certain graphs.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.
New bounds for CNNs show better generalization than previous models.
problem Improving understanding of CNNs' generalization ability.
method Proposed tighter generalization bounds for CNNs by exploiting the sparse and permutation structure of weight matrices and spectral norms of convolution operations.
result Theoretical and experimental results show tighter bounds for CNNs than existing bounds.
New bounds improve deep learning performance efficiently.
problem Improving generalization and robustness of deep learning models.
method Deriving four provable upper bounds on spectral norm of convolution layers, differentiable and efficient.
result Minimum of four bounds is a tight, differentiable and efficient upper bound on spectral norm.
Multi-output Gaussian processes (MOGPs) are an extension of Gaussian Processes (GPs) for predicting multiple output variables (also called channels, tasks) simultaneously. In this paper we use the convolution theorem to design a new kernel for MOGPs, by modeling cross channel dependencies through cross convolution of t…
Improved spectral-based GCN for directed graphs.
problem Cannot directly work on directed graphs.
method Redefined Laplacians to improve propagation model.
result Outperforms state-of-the-art methods on directed graph datasets.
Analyzes how diffusion models learn, revealing a spectral bias in structure mastery.
problem Understanding the learning dynamics and bias in diffusion models.
method Developed an analytical framework using a Gaussian-equivalence principle to solve gradient-flow dynamics and integrate probability-flow ODEs.
result Exposes a universal inverse-variance spectral law: high-variance structure is mastered faster than low-variance detail.
Spectral graph convolutional neural networks (CNNs) require approximation to the convolution to alleviate the computational complexity, resulting in performance loss. This paper proposes the topology adaptive graph convolutional network (TAGCN), a novel graph convolutional network defined in the vertex domain. We provi…
Graph diffusion convolution improves graph learning by leveraging generalized graph diffusion.
problem Noisy and arbitrarily defined edges in real graphs.
method Graph diffusion convolution (GDC) using generalized graph diffusion like heat kernel and personalized PageRank.
result Replacing message passing with graph diffusion convolution leads to significant performance improvements.
Discrete Fourier transforms provide a significant speedup in the computation of convolutions in deep learning. In this work, we demonstrate that, beyond its advantages for efficient computation, the spectral domain also provides a powerful representation in which to model and train convolutional neural networks (CNNs).…
Novel Haar-Laplacian for directed graphs enhances spectral graph applications.
problem Lack of suitable Laplacian for directed graphs in spectral graph theory.
method Inspired by Haar-like transformation, introduces a Hermitian matrix preserving direction and weight.
result HaarNet outperforms in weight prediction and denoising on directed graphs.
Enhances GCNs to resist graph attacks.
problem Vulnerability of GCNs to topological attacks.
method Introduces a new robust convolution operator and training paradigm.
result Improves adversarial robustness and performance.
Novel CSK kernel improves GP model generalization for non-stationary patterns.
problem Improving generalization of Gaussian process models for non-stationary data.
method Introduced convolutional spectral kernel (CSK) derived from convolution of imaginary radial basis functions, using Fourier transform for interpretation.
result CSK improves GP model generalization on spatiotemporal datasets.
Paper revisits graph-CNNs using Laplace-Beltrami spectral filters and polynomials.
problem Improving spectral graph convolutional neural networks (graph-CNNs).
method Developed Laplace-Beltrami CNN (LB-CNN) by replacing graph Laplacian with LB operator and approximating spectral filters using Chebyshev, Laguerre, and Hermite polynomials.
result Classification accuracy of LB-CNN is not dependent on the type of polynomials or operators.
Developed a framework for designing filters in spectral GCNNs with improved performance.
problem Designing effective filters for spectral GCNNs with regularization properties.
method Exploring regularization properties of graph Laplacian and proposing a generalized framework for filter design.
result New filters derived from the framework outperform state-of-the-art techniques in semi-supervised node classification.
Improves DNN robustness to adversarial attacks.
problem Adversarial attacks degrade DNN robustness.
method Spectral normalization of weight matrices.
result Spectral normalization enhances DNN generalization.
BankGCN improves graph convolution networks by handling multi-channel signals with adaptive filter banks.
problem Handling multi-channel graph signals with limited architectures.
method BankGCN decomposes multi-channel signals into subspaces and uses adapted filters for each subspace.
result BankGCN achieves excellent performance in graph classification on benchmark datasets.
A new spectral pooling method reduces loss in CNNs for better performance.
problem Lossy downsampling in CNNs reduces discriminability.
method Hartley Spectral Pooling using Hartley Transform.
result Hartley Spectral Pooling preserves more structure features than max/average pooling.
Paper shows spectral filters can transfer between different graphs discretizing the same space.
problem Transferability of spectral filters between different graphs.
method Analysis of spectral filters on graphs discretizing the same space.
result Spectral filters have similar effects on graphs discretizing the same space.
New metric assesses image dataset complexity.
problem Assessing the complexity of image classification datasets.
method Cumulative Spectral Gradient (CSG) derived from spectral clustering.
result CSG correlates with CNN test accuracy and dataset separability.
Corrected graph convolutions improve node classification on graphs.
problem Oversmoothing in graph convolutions degrades performance.
method Theoretical analysis based on CSBM, spectral analysis for k rounds of corrected graph convolutions.
result Corrected graph convolutions can improve node classification performance exponentially.
JSCN improves cross-domain recommendation by learning domain-invariant user representations.
problem Cross-domain recommendation data sparsity and domain-incompatibility issues.
method JSCN uses multi-layer spectral convolutions on different graphs to learn domain-invariant user representations and domain adaptive user mappings.
result Significant improvement in cross-domain recommendation performance (9.2% recall, 36.4% MAP improvements).
Novel multigraph network improves chemical classification tasks.
problem Learning from variable graphs with multiple relationships.
method Proposed a multigraph network using Chebyshev GCNs to handle variable graphs and learned edges.
result Achieved competitive results on chemical classification benchmarks.
Improved model for non-smooth signals with complex spectra.
problem Current models struggle with non-smooth signals and complex spectral structures.
method CGPCM and RGPCM models with causality and Bayesian nonparametric interpretations, improved variational inference.
result Proposed models show better performance on synthetic and real-world data.
A framework uses free probability to analyze Transformer models.
problem Understanding the dynamics and complexity of Transformer-based language models.
method Formal operator-theoretic analysis using free probability theory.
result Entropy-based generalization bounds derived under freeness assumptions.
In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain connectomes or words' embedding, represented by graphs. We present a formulation o…
IGT learns graph representations without supervision.
problem Building deep unsupervised graph representations.
method Generic complex-valued spectral graph architecture from Fourier transform generalization, greedy concave objective for discriminative and invariant features.
result IGT learns both discriminative and invariant features from graph topology.
ARMA filters improve graph neural networks' flexibility and robustness.
problem Improving graph neural networks' performance and robustness.
method Proposes an ARMA-based graph convolutional layer for graph neural networks.
result ARMA filters outperform polynomial filters in various graph tasks.
New complexity measure shows similar generalization bounds for CNNs and non-CNNs.
problem Understanding why CNNs generalize well despite fitting random labels.
method Theoretical and empirical investigation of spectral complexity measure insensitivity to CNN invariances.
result Spectral complexity measure results in the same upper bound complexity estimates for CNNs and non-CNNs, contradicting common intuition.
SF-GCN improves semi-supervised classification by fusing multi-view data structures.
problem Semi-supervised classification challenges due to multi-view data diversity and complexity.
method Structure fusion based on graph convolutional networks (SF-GCN) that balances specificity and commonality.
result SF-GCN outperforms state-of-the-art methods on citation networks datasets.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
Recently, there is increasing interest and research on the interpretability of machine learning models, for example how they transform and internally represent EEG signals in Brain-Computer Interface (BCI) applications. This can help to understand the limits of the model and how it may be improved, in addition to possi…
Study shows consistency of shallow GCNNs on sampled point clouds under manifold assumption.
problem Consistency of shallow GCNNs on sampled point clouds under manifold assumption.
method Functional analysis perspective, weakly compact product of unit balls, Sobolev regularity, frequency cutoff.
result Proves Γ-convergence of regularized empirical risk minimization functionals and convergence of their global minimizers. New method controls linear systems with partial info and disturbances.
problem Controlling linear dynamical systems under partial observation and adversarial disturbances.
method Double Spectral Control (DSC) using two-level spectral approximation strategy.
result Matches best known regret guarantees with exponential runtime improvement.
New graph convolution captures local features on non-Euclidean grids.
problem Capturing local features on irregular, coarse non-Euclidean grids.
method Low-rank learnable local filters in graph convolutions.
result Proves more expressive than previous spectral graph convolution methods.
Paper proves spectral filters can be transferred between graphs.
problem Proving spectral filters can be transferred between graphs.
method Introducing the Cayley smoothness space and proving filters in this space are linearly stable.
result Graph spectral filters are transferable if they are in the Cayley smoothness space.
This paper explains GNNs using graph signal denoising.
problem Understanding how GNNs work for node representation learning.
method Spectral graph convolutional networks and graph attention networks are analyzed from the perspective of graph signal denoising.
result GNNs implicitly solve graph signal denoising problems.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
Study optimal spectral estimator for semi-supervised node classification.
problem Semi-supervised node classification on CSBM with limited labels.
method Spectral estimator inspired by PCA, graph ridge regression, GCN.
result Achieves information-theoretical threshold for exact recovery.
We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions.…
An orbifold is a Morita equivalence class of a proper {\' e}tale Lie groupoid. A unitary equivalence class of spectral triples over the algebra of smooth invariant functions are associated with any compact spin orbifold. In the case of an effective spin orbifold we construct a collection of spectral triples over the sm…
NASirt automates CNN architecture design for spectral data.
problem Designing optimal neural architectures for complex data.
method Neural Architecture Search (NAS) with Item Response Theory (IRT) for instance-level complexity.
result NASirt achieves high accuracy (97.40%) on spectral datasets.
GWNN uses graph wavelets for efficient graph CNNs.
problem Spectral graph CNNs' high computational cost and lack of interpretability.
method Graph wavelet transform for efficient graph convolution.
result GWNN significantly outperforms spectral graph CNNs.
Graph neural networks can be adapted to new graphs with a limit object called graphon NNs.
problem Transferability of graph neural networks across different graphs.
method Introduced graphon NNs as limit objects of GNNs and proved a bound on the difference between GNN and graphon-NN outputs.
result The bound on the difference between GNN and graphon-NN outputs vanishes with growing number of nodes if the graph convolutional filters are bandlimited.