arXiv research

Vanishing nodes cause hidden nodes to behave similarly, complicating deep neural network training.

Sigmoid-type networks avoid vanishing gradients with regularization and rescaling.

We consider the problem of decentralized consensus optimization, where the sum of $n$ smooth and strongly convex functions are minimized over $n$ distributed agents that form a connected network. In particular, we consider the case that the communicated local decision variables among nodes are quantized in order to all…

Given a graph embedded in an orientable surface, a process consisting of random excitations and random node and face balancing is constructed and analyzed. It is shown that given a priori bounds g' on the genus and n' on the number of nodes, one can determine the genus of the surface from local observations of the proc…

An analytic approach and description are presented for the moduli cotangent sheaf for suitable stable curve families including noded fibers. For sections of the square of the relative dualizing sheaf, the residue map at a node gives rise to an exact sequence. The residue kernel defines the vanishing residue subsheaf. F…

We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized optimization schemes face two major system bottlenecks: stragglers' delay and communication overhead. In this paper, we tackle these bottlene…

Graph neural networks can be adapted to new graphs with a limit object called graphon NNs.

MPNNs over-squash distant node information, study shows.

In this paper, we consider the problem of learning an unknown graph via queries on groups of nodes, with the result indicating whether or not at least one edge is present among those nodes. While learning arbitrary graphs with $n$ nodes and $k$ edges is known to be hard in the sense of requiring $Ω( \min\{ k^2 \log n, …

Study bandwidth-limited training and inference of language models.

Ghost mechanism explains abrupt learning in RNNs, revealing constraints on optimization landscapes.

We consider community detection in Degree-Corrected Stochastic Block Models (DC-SBM). We propose a spectral clustering algorithm based on a suitably normalized adjacency matrix. We show that this algorithm consistently recovers the block-membership of all but a vanishing fraction of nodes, in the regime where the lowes…

In this paper, we investigate community detection in networks in the presence of node covariates. In many instances, covariates and networks individually only give a partial view of the cluster structure. One needs to jointly infer the full cluster structure by considering both. In statistics, an emerging body of work …

DeeperGCN tackles deep GCNs by overcoming vanishing gradient and over-smoothing issues.

This paper proposes an improved design of the perceptron unit to mitigate the vanishing gradient problem. This nuisance appears when training deep multilayer perceptron networks with bounded activation functions. The new neuron design, named auto-rotating perceptron (ARP), has a mechanism to ensure that the node always…

The paper explores statistical limits for detecting correlation in tree structures.

New non-Kähler 3-folds constructed via log conifold transitions.

Clustering is one of the most important unsupervised problems in machine learning and statistics. Among many existing algorithms, kernel k-means has drawn much research attention due to its ability to find non-linear cluster boundaries and its inherent simplicity. There are two main approaches for kernel k-means: SVD o…

Paper improves robust spectral clustering for noisy data.

Spectral clustering is a celebrated algorithm that partitions objects based on pairwise similarity information. While this approach has been successfully applied to a variety of domains, it comes with limitations. The reason is that there are many other applications in which only \emph{multi}-way similarity measures ar…

Graph semi-supervised learning classifies points on manifold using variational autoencoders and GNN.

Analyzes Saito vanishing theorem using $L^{2}$ methods.

LC-GNN improves GNNs for node classification by incorporating label consistency.

SENSE enhances node sequences in graphs using vector embeddings.

Network node embedding is an active research subfield of complex network analysis. This paper contributes a novel approach to learning network node embeddings and direct node classification using a node ranking scheme coupled with an autoencoder-based neural network architecture. The main advantages of the proposed Dee…

Network embedding has proved extremely useful in a variety of network analysis tasks such as node classification, link prediction, and network visualization. Almost all the existing network embedding methods learn to map the node IDs to their corresponding node embeddings. This design principle, however, hinders the ex…

The paper uses non-abelian Hodge theory to generalize Kodaira vanishing theorems.

Study on a knot invariant's vanishing order.

Proposes GIL for semi-supervised graph classification.

Embedding graph nodes into a vector space can allow the use of machine learning to e.g. predict node classes, but the study of node embedding algorithms is immature compared to the natural language processing field because of a diverse nature of graphs. We examine the performance of node embedding algorithms with respe…

Proves a vanishing property for symplectic manifold cohomology.

CADE learns dual node representations for better generalization.

Graph structured data provide two-fold information: graph structures and node attributes. Numerous graph-based algorithms rely on both information to achieve success in supervised tasks, such as node classification and link prediction. However, node attributes could be missing or incomplete, which significantly deterio…

SANNE model generates embeddings for unseen nodes in graph networks.

Study on Nijenhuis tensor forms and vanishing properties.

Study finds significant instability in node embeddings due to randomness.

New vanishing theorems for genera derived under almost nonnegative Ricci curvature.

Proposes ML-GCN for multi-label network node representation learning.

CatGCN improves GCNs by modeling feature interactions for categorical node features.

HighwayGraph models long-distance node relations in GNNs with improved performance.

This paper studies node embeddings of networks, revealing their geometric properties.

A symplectic form has a primitive with nowhere vanishing $β$.

A new method improves node classification in graphs with limited labels.

Paper proves impossibility of three desirable properties in node embedding.

Learning node embeddings that capture a node's position within the broader graph structure is crucial for many prediction tasks on graphs. However, existing Graph Neural Network (GNN) architectures have limited power in capturing the position/location of a given node with respect to all other nodes of the graph. Here w…

GraphCL learns node representations by maximizing similarity between perturbed node features.

ARGEW improves node embeddings for weighted homophilous graphs by emphasizing strong edge weights.

SPARC tackles cold-start nodes in graphs by using spectral embeddings.