Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

6.3%12.5%18.8%25.0% · Mar 199319922001200920172026
48 results for vanishing nodes

Vanishing nodes cause hidden nodes to behave similarly, complicating deep neural network training.

problem Complicating deep neural network training due to hidden nodes behaving similarly.
method Introduced vanishing node indicator (VNI) to quantify node redundancy and correlated behavior.
result The effective number of nodes vanishes to one as VNI increases, indicating a challenging training scenario.

We consider the problem of decentralized consensus optimization, where the sum of nn smooth and strongly convex functions are minimized over nn distributed agents that form a connected network. In particular, we consider the case that the communicated local decision variables among nodes are quantized in order to all…

2018-06-29abs ↗pdf ↗

An analytic approach and description are presented for the moduli cotangent sheaf for suitable stable curve families including noded fibers. For sections of the square of the relative dualizing sheaf, the residue map at a node gives rise to an exact sequence. The residue kernel defines the vanishing residue subsheaf. F…

2012-04-17abs ↗pdf ↗

We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized optimization schemes face two major system bottlenecks: stragglers' delay and communication overhead. In this paper, we tackle these bottlene…

2019-07-24abs ↗pdf ↗

Graph neural networks can be adapted to new graphs with a limit object called graphon NNs.

problem Transferability of graph neural networks across different graphs.
method Introduced graphon NNs as limit objects of GNNs and proved a bound on the difference between GNN and graphon-NN outputs.
result The bound on the difference between GNN and graphon-NN outputs vanishes with growing number of nodes if the graph convolutional filters are bandlimited.

MPNNs over-squash distant node information, study shows.

problem Over-squashing in MPNNs where node features ignore distant nodes.
method Theoretical analysis of MPNNs' over-squashing, focusing on width, depth, and graph topology.
result Width mitigates over-squashing but makes network more sensitive, depth doesn't help, graph topology is key.

In this paper, we consider the problem of learning an unknown graph via queries on groups of nodes, with the result indicating whether or not at least one edge is present among those nodes. While learning arbitrary graphs with nn nodes and kk edges is known to be hard in the sense of requiring $Ω( \min\{ k^2 \log n, …

2019-05-09abs ↗pdf ↗

Study bandwidth-limited training and inference of language models.

problem Training and inference of language models on scattered data with limited bandwidth.
method Analyzed two protocols: Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for inference.
result Explicit high-probability KL-consistency rate and distribution-free marginal-coverage bound for Federated Conformal RAG.

Ghost mechanism explains abrupt learning in RNNs, revealing constraints on optimization landscapes.

problem Understanding abrupt learning in recurrent neural networks (RNNs) trained on working memory tasks.
method Introducing the ghost mechanism, a process driven by saddle-node bifurcations, to analyze and model abrupt learning.
result A critical learning rate scales as an inverse power law with the timescale of computation, leading to vanishing and oscillatory gradients.

In this paper, we investigate community detection in networks in the presence of node covariates. In many instances, covariates and networks individually only give a partial view of the cluster structure. One needs to jointly infer the full cluster structure by considering both. In statistics, an emerging body of work …

2016-07-10abs ↗pdf ↗

DeeperGCN tackles deep GCNs by overcoming vanishing gradient and over-smoothing issues.

problem Vanishing gradient, over-smoothing, and over-fitting issues in deep GCNs.
method DeeperGCN uses differentiable generalized aggregation functions and a novel normalization layer (MsgNorm) to train deep GCNs.
result DeeperGCN significantly boosts performance on large-scale graph learning tasks.

The paper explores statistical limits for detecting correlation in tree structures.

problem Detecting correlation between two tree structures.
method Investigates conditions for existence of one-sided tests in the limit of large tree depth.
result Identifies a phase transition at correlation parameter s=αs = \sqrt{α}, where tests exist for s>αs > \sqrt{α}.

Clustering is one of the most important unsupervised problems in machine learning and statistics. Among many existing algorithms, kernel k-means has drawn much research attention due to its ability to find non-linear cluster boundaries and its inherent simplicity. There are two main approaches for kernel k-means: SVD o…

2016-06-06abs ↗pdf ↗

Spectral clustering is a celebrated algorithm that partitions objects based on pairwise similarity information. While this approach has been successfully applied to a variety of domains, it comes with limitations. The reason is that there are many other applications in which only \emph{multi}-way similarity measures ar…

2018-05-23abs ↗pdf ↗

Graph semi-supervised learning classifies points on manifold using variational autoencoders and GNN.

problem Classifying points on low-dimensional manifolds using limited labeled data.
method Model data as points on a manifold, approximate manifold with VAE, construct geometric graph, solve semi-supervised node classification with GNN.
result Generalization gap diminishes with graph size and training procedure, vanishing asymptotically.

LC-GNN improves GNNs for node classification by incorporating label consistency.

problem Limited performance of GNNs due to label consistency assumption not always holding.
method LC-GNN uses node pairs with the same label but unconnected to expand GNN's receptive field.
result LC-GNN outperforms traditional GNNs in semi-supervised node classification.

SENSE enhances node sequences in graphs using vector embeddings.

problem Efficiently capturing graph node sequences for applications.
method SENSE-S learns node embeddings and composes them for sequences, preserving node order.
result SENSE-S increases multi-label classification and link-prediction accuracy by up to 50% and 78% respectively.

Network embedding has proved extremely useful in a variety of network analysis tasks such as node classification, link prediction, and network visualization. Almost all the existing network embedding methods learn to map the node IDs to their corresponding node embeddings. This design principle, however, hinders the ex…

2019-04-17abs ↗pdf ↗

CADE learns dual node representations for better generalization.

problem Transductive graph embeddings cannot generalize to unseen nodes or across different graphs.
method CADE combines real-time neighborhoods with neighbor-attentioned representation, preserving known node memory.
result CADE outperforms state-of-the-art methods in generalization and context-awareness.

Graph structured data provide two-fold information: graph structures and node attributes. Numerous graph-based algorithms rely on both information to achieve success in supervised tasks, such as node classification and link prediction. However, node attributes could be missing or incomplete, which significantly deterio…

2019-07-23abs ↗pdf ↗

New vanishing theorems for genera derived under almost nonnegative Ricci curvature.

problem Vanishing theorems for genera under specific curvature conditions.
method Almost nonnegative Ricci curvature and infinite fundamental group.
result Vanishing theorems for Todd genus, A^\widehat{A}-genus, elliptic genera, Witten genus, and Euler characteristic number for Alexandrov spaces.

Proposes ML-GCN for multi-label network node representation learning.

problem Complex multi-label networks with correlated labels.
method Two Siamese GCNs model node-label and label-label interactions, integrated under a unified objective function.
result Effective node representation learning with preserved label interactions.

CatGCN improves GCNs by modeling feature interactions for categorical node features.

problem Suboptimal initial node representations in GCNs due to lack of feature interaction modeling.
method Integrates explicit interaction modeling (local and global) into initial node representation learning for categorical node features.
result CatGCN enhances initial node representations through feature interaction modeling, leading to improved model performance.

HighwayGraph models long-distance node relations in GNNs with improved performance.

problem Limited-layer information propagation in GNNs hinders long-distance node relation modeling.
method Proposes two solutions: implicit and explicit modeling of long-distance node relations using shallow GNN architectures and a self-training framework.
result HighwayGraph achieves consistent and significant improvements over four GNNs on three benchmark datasets.

This paper studies node embeddings of networks, revealing their geometric properties.

problem Understanding the geometric properties of node embeddings in random networks.
method Characterization of ergodic limits, generalization, and convex relaxations of random walk node embedding objectives.
result The optimal node embedding Grammians have rank 1 for a nuclear norm relaxation of the non-randomized objective.

A new method improves node classification in graphs with limited labels.

problem Semi-supervised multi-label node classification in attributed graphs.
method Collaborative Graph Walk (Multi-Label-Graph-Walk) using reinforcement learning.
result Significantly better multi-label classification performance compared to state-of-the-art methods.

Learning node embeddings that capture a node's position within the broader graph structure is crucial for many prediction tasks on graphs. However, existing Graph Neural Network (GNN) architectures have limited power in capturing the position/location of a given node with respect to all other nodes of the graph. Here w…

2019-06-11abs ↗pdf ↗

GraphCL learns node representations by maximizing similarity between perturbed node features.

problem Learning node representations in graph data without labeled data.
method Contrastive learning of node embeddings using graph neural networks and a loss function.
result Significantly outperforms state-of-the-art in unsupervised node classification benchmarks.

ARGEW improves node embeddings for weighted homophilous graphs by emphasizing strong edge weights.

problem Lack of accurate node embeddings for weighted homophilous graphs.
method ARGEW (Augmentation of Random walks by Graph Edge Weights) augments random walks by emphasizing nodes with larger edge weights.
result ARGEW produces embeddings where node pairs with strong edge weights have closer embeddings.