Paper tackles over-smoothing in deep GCNs, proposing DropEdge to improve performance.
problem Over-smoothing reduces expressivity in deep GCNs, especially affecting node classification.
method Theoretical analysis of GCN behavior with depth, proposing DropEdge to alleviate over-smoothing.
result DropEdge improves performance on various GCNs, shallow and deep.
Graph Neural Networks (GNNs) have achieved promising performance on a wide range of graph-based tasks. Despite their success, one severe limitation of GNNs is the over-smoothing issue (indistinguishable representations of nodes in different classes). In this work, we present a systematic and quantitative study on the o…
KuramotoGNN uses Kuramoto model to prevent over-smoothing in graph neural networks.
problem Over-smoothing in graph neural networks where node features become indistinguishable.
method Integrates Kuramoto model to prevent phase synchronization and instead achieve frequency synchronization.
result KuramotoGNN reduces over-smoothing on various graph deep learning tasks.
Graphs can be smoothed or squashed too, study finds.
problem Graph Neural Networks struggle with over-smoothing and over-squashing issues.
method Unified framework using Ollivier-Ricci curvature to address both issues.
result Over-smoothing and over-squashing linked to positive and negative graph curvature respectively.
Graph neural networks over-smooth when layers increase, reducing discriminative power.
problem Over-smoothing in graph neural networks reduces model performance as the number of layers increases.
method Analyzed over-smoothing in general graph neural network architecture using Dirichlet energy.
result The Dirichlet energy of embeddings converges to zero, leading to loss of discriminative power.
Advocates against over-smoothing and over-squashing in GNNs, suggesting they are less critical than previously thought.
problem Over-smoothing and over-squashing in Graph Neural Networks (GNNs).
method Challenged the prevailing focus on these phenomena, proposing that performance decreases are due to uninformative receptive fields and localised information distribution.
result Performance decreases are mostly uncorrelated with over-smoothing and over-squashing, and optimal model depths remain small.
Graph pruning improves neural network performance by addressing squashing and smoothing issues.
problem Over-squashing and over-smoothing in Graph Neural Networks.
method Proposes edge deletions to simultaneously address over-squashing and over-smoothing, optimizing spectral gap.
result Edge deletions improve generalization and distinguishability of nodes of different classes.
Paper analyzes and improves graph convolutional networks for node classification.
problem Over-smoothing in GCNs causes poor performance in node classification tasks.
method Interpreted GCNs from an optimization perspective, introduced metrics to measure over-smoothing, derived a new kernel GCN+.
result GCN+ reduces over-smoothing and improves node classification performance.
Deep GCNII tackles over-smoothing problem in graph convolutional networks.
problem Over-smoothing problem in shallow graph convolutional networks.
method Proposes GCNII with initial residual and identity mapping techniques.
result Deep GCNII outperforms state-of-the-art methods on various tasks.
A new technique normalizes nodes within groups to improve GNN performance.
problem Over-smoothing in deeper GNNs reduces node distinguishability.
method Differentiable group normalization (DGN) to separate node distributions among groups.
result DGN makes GNN models more robust to over-smoothing and achieves better performance with deeper GNNs.
New method uses curvature to improve graph neural networks.
problem Graph Neural Networks struggle with over-smoothing and over-squashing.
method Augmented Forman-Ricci curvature (AFRC) for scalable rewiring.
result AFRC effectively mitigates over-smoothing and over-squashing.
GHNet improves graph learning by balancing homogeneity and heterogeneity.
problem Over-smoothing in GCN leads to similar node representations.
method GHNet uses gating units to balance homogeneity and heterogeneity in feature propagation.
result GHNet achieves larger receptive fields without over-smoothing.
The paper investigates how Global Self-attention improves GCNs.
problem Improving expressive power and addressing overfitting in GCNs.
method Applying Global Self-attention mechanism over graph features.
result GSA mechanism enhances expressive power and mitigates overfitting and over-smoothing in GCNs.
\emph{Over-fitting} and \emph{over-smoothing} are two main obstacles of developing deep Graph Convolutional Networks (GCNs) for node classification. In particular, over-fitting weakens the generalization ability on small dataset, while over-smoothing impedes model training by isolating output representations from the i…
By Federer and Fleming there exist at least one mass-minimizing normal current in every real-valued homology class of a Riemannian manifold. However the regularity of the mass-minimizing currents and their distributions may generally be quite complicated. In this paper we shall study how to construct nice metrics so th…
A new method boosts graph neural networks by preventing over-smoothing and over-squashing.
problem Graph Neural Networks struggle with long-range signals and over-smoothing/over-squashing.
method Proposes PowerEmbed, a layer-wise normalization technique inspired by spectral graph embedding.
result PowerEmbed prevents over-smoothing and avoids over-squashing, improving performance on heterophilous graphs.
Boosting theory explains why multi-scale GNNs work.
problem Over-smoothing in graph neural networks.
method Gradient boosting and transductive learning analysis.
result Test error bound decreases with more node aggregations.
Graph Random Neural Network improves semi-supervised learning on graphs.
problem Over-smoothing, non-robustness, and weak-generalization in GNNs with few labeled nodes.
method Random propagation strategy and consistency regularization.
result Significantly outperforms state-of-the-art GNN baselines on semi-supervised node classification.
Unified framework for adaptive connection sampling in GNNs improves performance and robustness.
problem Over-smoothing and over-fitting in deep GNNs.
method Adaptive connection sampling trained jointly with GNN model parameters.
result Adaptive connection sampling mathematically equivalent to Bayesian GNNs approximation.
Improves GCNNs with node transition probabilities and DropNode regularization.
problem Over-fitting and over-smoothing issues in GCNNs.
method Message passing based on node transition probabilities and DropNode regularization.
result Improved GCNNs with better node representations and reduced over-fitting and over-smoothing.
SStaGCN improves GCN by stacking and aggregation for better node feature extraction.
problem Mitigating over-smoothing in GCN for heterogeneous graph data.
method SStaGCN combines stacking and aggregation to improve GCN performance.
result SStaGCN effectively mitigates over-smoothing and enhances node feature extraction.
Infer-AVAE infers missing user attributes from incomplete data using a novel adversarial approach.
problem Incomplete user attributes in social networks.
method Infer-AVAE combines MLP and GNNs with adversarial training to infer missing attributes.
result Infer-AVAE outperforms baselines by 7.0% in accuracy on real-world datasets.
We study differential invariants of linear differential operators and use them to find conditions for equivalence of differential operators acting in line bundles over smooth manifolds with respect to groups of authomorphisms.
STAG injects noise into graph neural networks to improve performance.
problem Graph neural networks suffer from over-smoothing and limited discrimination.
method Introduces a stochastic aggregation framework (STAG) with adaptive noise injection.
result STAG models correct both over-smoothing and discrimination issues.
New view: Deep GCNs learn to anti-oversmooth during training.
problem Performance drop in deep GCNs due to oversmoothing.
method Interpreted GCN as MLP + graph regularization, analyzed training process.
result Deep GCNs learn to anti-oversmooth during training, not over-smooth.
New approach to deeper graph neural networks to avoid performance degradation.
problem Performance degradation of graph neural networks when going deeper.
method Decoupling representation transformation and propagation in graph convolution operations.
result Deeper graph neural networks can be used to learn graph node representations from larger receptive fields.
Graph convolutions can enhance high frequencies, leading to over-sharpening.
problem Graph convolutions suffer from over-smoothing and poor performance on heterophilic graphs.
method Rigorously prove that linear graph convolutions minimize a generalized Dirichlet energy, showing that weight matrices induce edge-wise attraction or repulsion.
result Graph convolutions can enhance high frequencies, leading to over-sharpening instead of over-smoothing.
Graph attention is not always beneficial; conditions for perfect node classification are identified.
problem Understanding when graph attention mechanisms improve node classification performance.
method Theoretical analysis using Contextual Stochastic Block Models (CSBMs).
result Graph attention mechanisms are more effective when structure noise exceeds feature noise, and simpler graph convolution operations are better when feature noise predominates.
Researchers compute monodromy groups of surface families over quartic curves.
problem Computing monodromy groups of surface families over smooth quartic curves.
method Analyzing cyclic branched covers of P2 over smooth quartic curves, computing monodromy groups for del Pezzo and K3 surfaces. result Obtained monodromy groups for del Pezzo and K3 surfaces, including Weyl group $W\left(E_{7}
ight)$ and arithmetic lattice $U\left(h_{L_{-}}
ight)$.
The hermitian analog of Aleksandrov's area measures of convex bodies is investigated. A characterization of those area measures which arise as the first variation of unitarily invariant valuations is established. General smooth area measures are shown to form a module over smooth valuations and the module of unitarily …
Graph Attention Networks (GATs) are the state-of-the-art neural architecture for representation learning with graphs. GATs learn attention functions that assign weights to nodes so that different nodes have different influences in the feature aggregation steps. In practice, however, induced attention functions are pron…
DeeperGCN tackles deep GCNs by overcoming vanishing gradient and over-smoothing issues.
problem Vanishing gradient, over-smoothing, and over-fitting issues in deep GCNs.
method DeeperGCN uses differentiable generalized aggregation functions and a novel normalization layer (MsgNorm) to train deep GCNs.
result DeeperGCN significantly boosts performance on large-scale graph learning tasks.
Most density-based clustering methods largely rely on how well the underlying density is estimated. However, density estimation itself is also a challenging problem, especially the determination of the kernel bandwidth. A large bandwidth could lead to the over-smoothed density estimation in which the number of density …
New SKT manifolds created using toric geometry.
problem Creating SKT manifolds.
method Using toric geometry and J-construction. result Infinite families of SKT manifolds produced.
Novel GNN model tackles few-shot learning with improved performance.
problem Few-shot learning with GNN suffers from over-fitting and over-smoothing.
method Proposes Attentive GNN with triple-attention mechanism.
result Improves GNN performance for few-shot learning tasks.
Constructs non-Kähler Calabi-Yau manifolds with large Betti numbers.
problem Finding non-Kähler Calabi-Yau manifolds with high Betti numbers.
method Smoothing normal crossing varieties to create K3 fibrations over smooth projective varieties.
result Examples of non-Kähler Calabi-Yau manifolds with arbitrarily large 2nd Betti numbers.
This paper develops a local analogue of the ADHM construction, which characterises ASD instantons defined over smooth bounded domains inside Euclidean R4 diffeomorphic to the 4-ball, in terms of infinite dimensional Hilbert spaces and bounded Hermitian linear operators satisfying an analogue of the ADHM equ…
Graph Convolutional Networks (GCNs) are state-of-the-art graph based representation learning models by iteratively stacking multiple layers of convolution aggregation operations and non-linear activation operations. Recently, in Collaborative Filtering (CF) based Recommender Systems (RS), by treating the user-item inte…
We propose a learning-based filter that allows us to directly modify a synthetic speech waveform into a natural speech waveform. Speech-processing systems using a vocoder framework such as statistical parametric speech synthesis and voice conversion are convenient especially for a limited number of data because it is p…
Sheaf Neural Networks improve graph learning with geometric insights.
problem Graph heterophily and over-smoothing issues.
method Inspired by Riemannian geometry, computes sheaves using orthogonal maps.
result Achieves promising results with reduced computational overhead.
Introduces data augmentation for graph convolutional networks, proposing Monte Carlo Graph Learning.
problem Lack of transparency in graph convolutional networks.
method Data augmentation through graph structure, training traditional classifiers on expanded training set.
result MCGL shows better tolerance to graph structure noise than GCN on noisy graphs.
We propose a parallel-data-free voice-conversion (VC) method that can learn a mapping from source to target speech without relying on parallel data. The proposed method is general purpose, high quality, and parallel-data free and works without any extra data, modules, or alignment procedure. It also avoids over-smoothi…
Study reveals decurve flows in graph propagation models.
problem Limitations of traditional graph analysis and propagation mechanisms.
method Introduces Generalized Propagation Neural Networks (GPNNs) and Continuous Unified Ricci Curvature (CURC).
result Observation of decurve flow during training of graph neural networks, revealing propagation dynamics.
The Seiberg-Witten equations are defined on certain complex line bundles over smooth oriented four manifolds. When the base manifold is a complex Kahler surface, the Seiberg-Witten equations are essentially the Abelian vortex equations. Using known non-abelian generalizations of the vortex equations as a guide, we expl…
This is a sequel of our paper [arXiv:1809.08425] on the Quot-scheme limit and variational properties of Donaldson's functional, which established its coercivity for slope stable holomorphic vector bundles over smooth projective varieties. Assuming that the coercivity is uniform in a certain sense, we provide a new proo…
Study Lie algebroid connections on principal bundles over complex projective varieties.
problem Existence and properties of Lie algebroid connections on principal bundles.
method Definition and study of Lie algebroid valued connections on holomorphic principal G-bundles, investigation of existence criteria.
result Investigation of criteria for existence of Lie algebroid connections on principal G-bundles over smooth complex projective curves.
We consider natural algebraic differential operations acting on geometric quantities over smooth manifolds. We introduce a method of study and classification of such operations, called IT-reduction. It reduces the study of natural operations to the study of polynomial maps between (vector) spaces of jets which are equi…
We provide new bounds on a flux integral over the portion of the boundary of one regular domain contained inside a second regular domain, based on properties of the second domain rather than the first one. This bound is amenable to numerical computation of a flux through the boundary of a domain, for example, when ther…