Graph data widely exist in many high-impact applications. Inspired by the success of deep learning in grid-structured data, graph neural network models have been proposed to learn powerful node-level or graph-level representation. However, most of the existing graph neural networks suffer from the following limitations…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “graph neural nets”
Graph Neural Nets (GNNs) have received increasing attentions, partially due to their superior performance in many node and graph classification tasks. However, there is a lack of understanding on what they are learning and how sophisticated the learned graph functions are. In this work, we propose a dissection of GNNs …
Paper proposes JDR to denoise graph features and rewire graphs for better node classification.
problem Jointly denoise noisy graph features and rewire graphs for improved node classification.
method Align leading spectral spaces of graph and feature matrices to solve non-convex optimization problem.
result JDR consistently outperforms existing methods on various node classification tasks.
HGNet improves GNNs' ability to handle long-range interactions in graphs.
problem Insufficiency of GNNs in capturing long-range interactions.
method Introduces hierarchical message passing models with multi-resolution graph representations.
result HGNet outperforms conventional GNNs in molecular property prediction.
New neural nets respect triangle inequality, improving graph and reinforcement learning performance.
problem Neural nets lack inductive bias for certain subadditive distances.
method Introduced novel architectures that universally approximate norm-induced metrics.
result Neural nets with triangle inequality inductive bias outperform existing approaches.
Proposes -Nets, polynomial neural networks, for improved representation power.
problem Improving representation power in deep learning models.
method Introduces -Nets, a new class of deep polynomial neural networks.
result Demonstrates -Nets outperform standard DCNNs and achieve state-of-the-art results.
Optimizing quantum graphs yields geodesic nets on surfaces.
problem Finding optimal quantum graphs for geodesic nets.
method Optimizing functionals from spectral theory to find geodesic nets.
result Critical metrics for eigenvalues give rise to geodesic nets.
GMT improves interpretability of XGNNs by approximating SubMT.
problem Limited understanding of existing interpretable subgraph learning methods.
method Formulated subgraph multilinear extension (SubMT) and designed GMT architecture.
result GMT outperforms state-of-the-art in both interpretability and generalizability.
Graph U-Netscs.LG
We consider the problem of representation learning for graph data. Convolutional neural networks can naturally operate on images, but have significant challenges in dealing with graph data. Given images are special cases of graphs with nodes lie on 2D lattices, graph embedding tasks have a natural correspondence with i…
Numerous pattern recognition applications can be formed as learning from graph-structured data, including social network, protein-interaction network, the world wide web data, knowledge graph, etc. While convolutional neural network (CNN) facilitates great advances in gridded image/video understanding tasks, very limit…
Neural network learns from higher-order connections in molecules.
problem Graph neural networks fail to account for local and hidden structures in graphs.
method Developed a neural network that can pass messages and aggregate information across higher-order paths.
result The model improves molecular property prediction.
Recently, graph neural networks have been adopted in a wide variety of applications ranging from relational representations to modeling irregular data domains such as point clouds and social graphs. However, the space of graph neural network architectures remains highly fragmented impeding the development of optimized …
Entangled embedded periodic nets and crystal frameworks are defined, along with their dimension type, homogeneity type, adjacency depth and periodic isotopy type. We obtain periodic isotopy classifications for various families of embedded nets with small quotient graphs. We enumerate the 25 periodic isotopy classes of …
We present GraphMix, a regularization method for Graph Neural Network based semi-supervised object classification, whereby we propose to train a fully-connected network jointly with the graph neural network via parameter sharing and interpolation-based regularization. Further, we provide a theoretical analysis of how G…
Proposes polynomial neural networks for improved function approximation in various tasks.
problem Improving function approximation in various tasks like image generation, face verification, and 3D mesh representation learning.
method Introduces polynomial neural networks (-Nets) and three tensor decompositions to reduce parameter count and enhance expressiveness.
result Demonstrates that -Nets can produce state-of-the-art results in challenging tasks without non-linear activation functions.
AdaCAD improves semi-supervised classification by focusing on intra-class nodes.
problem Improving semi-supervised classification by addressing inter-class connections in graphs.
method AdaCAD uses a class-attentive diffusion process to adaptively aggregate nodes based on their class similarity.
result AdaCAD significantly outperforms state-of-the-art methods in semi-supervised classification.
Geodesic nets on flat spheres are studied using Gauss-Bonnet theorem.
problem Existence and non-existence of specific geodesic nets on flat spheres.
method The theorem of Gauss-Bonnet is applied to demonstrate results.
result Existence and non-existence of geodesic nets on regular doubled polygons.
Develops method for learning signed graphs from smooth signals.
problem Learning signed graphs from observed data, especially in contexts with both positive and negative interactions.
method Uses net Laplacian as graph shift operator and minimizes total variation of observed signals with ADMM.
result Theoretical proofs of convergence and estimation error bound provided.
SLAM-net learns to navigate visually in challenging indoor environments.
problem Challenges in SLAM for visual robot navigation, especially in noisy conditions.
method Differentiable SLAM Network (SLAM-net) that encodes a particle filter SLAM algorithm in a differentiable graph and learns components through backpropagation.
result Significantly outperforms ORB-SLAM in noisy conditions and improves the Habitat Challenge 2020 PointNav task.
An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…
We give algorithms with provable guarantees that learn a class of deep nets in the generative model view popularized by Hinton and others. Our generative model is an node multilayer neural net that has degree at most for some and each edge has a random edge weight in . Our algorithm learns {\em …
Theorem shows generic metrics yield non-degenerate geodesic nets.
problem Characterizing geodesic nets on generic metrics.
method Proving all connected embedded nets are non-degenerate for Baire-generic metrics.
result All stationary geodesic nets are non-degenerate for generic metrics.
Despite the phenomenal success of deep learning in recent years, there remains a gap in understanding the fundamental mechanics of neural nets. More research is focussed on handcrafting complex and larger networks, and the design decisions are often ad-hoc and based on intuition. Some recent research has aimed to demys…
Stochastic blockmodels (SBM) and their variants, , mixed-membership and overlapping stochastic blockmodels, are latent variable based generative models for graphs. They have proven to be successful for various tasks, such as discovering the community structure and link prediction on graph-structured data. Recentl…
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …
Extends string-net theory to 3D TQFT via surface graphs and surgery.
problem Formulate 3D TQFT using string-net theory.
method Extend string-net construction to 3D TQFT using surface graphs and surgery.
result Alternative description of Turaev-Viro model using string-nets.
GIT-Net uses neural networks to approximate PDE operators efficiently.
problem Approximating PDE operators for complex geometries.
method Parametrizes adaptive generalized integral transforms with deep neural networks.
result GIT-Net outperforms existing neural network operators in multiple areas.
DGNN predicts financial margin calls under stress tests.
problem Forecasting margin calls in dynamic financial networks.
method Dynamic Graph Neural Network (DGNN) architecture.
result DGNN produces accurate forecasts up to 21 days.
Knot theory is the study of isotopy classes of embeddings of the circle into a 3-manifold, specifically . The Fáry-Milnor Theorem says that any curve in of total curvature less than is unknotted. More generally, a (finite) graph consists of a finite number of edges and vertices. Given a topologica…
Gradient descent struggles to learn equivariant neural networks, even with symmetries.
problem Learning equivariant neural networks via gradient descent is hard.
method Lower bounds for various equivariant neural network classes.
result Gradient descent struggles to learn equivariant neural networks, even with symmetries.
DP-Net uses dynamic programming for efficient deep neural network compression.
problem Efficiently compressing deep neural networks while maintaining accuracy.
method Dynamic Programming for optimal weight quantization and clustering-friendly training.
result Achieves up to 77X compression ratio on Wide ResNet with minimal accuracy loss.
Study measures impact of data and neural net similarity on transferability in restaurant sales data.
problem Identify indicators for successful transferability of neural nets across different data sets.
method Empirical study on sales data from six restaurants, calculating indicators based on data and neural net similarities.
result Negative correlations between transferability and indicators, allowing better model performance and fewer transfers.
Learning to Optimize is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinf…
PNA improves GNNs for graph data with multiple aggregators.
problem Capturing continuous features in graph neural networks.
method Combines multiple aggregators with degree-scalers.
result PNA outperforms existing models on graph theory and real-world tasks.
This paper considers the power of deep neural networks (deep nets for short) in realizing data features. Based on refined covering number estimates, we find that, to realize some complex data features, deep nets can improve the performances of shallow neural networks (shallow nets for short) without requiring additiona…
Paper tackles shape graph registration using neural networks.
problem Constrained registration of shape graphs with varying nodes and edges.
method Shape-Graph Matching Network (SGM-net) with an elastic shape metric loss function.
result State-of-the-art matching performance and reduced computational cost.
DNF-Net tackles tabular data challenges with neural architecture.
problem Handling tabular data efficiently using neural networks.
method DNF-Net uses a neural architecture with inductive bias corresponding to logical Boolean formulas in disjunctive normal form over affine soft-threshold decision terms.
result DNF-Net significantly outperforms fully connected networks on tabular data.
The vast majority of the neural network literature focuses on predicting point values for a given set of response variables, conditioned on a feature vector. In many cases we need to model the full joint conditional distribution over the response variables rather than simply making point predictions. In this paper, we …
Network data appears in very diverse applications, like biological, social, or sensor networks. Clustering of network nodes into categories or communities has thus become a very common task in machine learning and data mining. Network data comes with some information about the network edges. In some cases, this network…
SDE-Net quantifies uncertainty in deep nets using stochastic dynamics.
problem Uncertainty quantification in deep neural networks.
method Viewing DNN transformations as state evolution of a stochastic dynamical system, introducing a Brownian motion term for epistemic uncertainty.
result SDE-Net outperforms existing methods in uncertainty estimation across various tasks.
We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural nets and by using the technique of chaining mutual information introduced in Asadi…
fSDE-Net generates time series with long-term memory using neural networks.
problem Generating time series with long-term memory from irregularly sampled data.
method fSDE-Net: neural fractional Stochastic Differential Equation Network using fractional Brownian motion.
result fSDE-Net can replicate distributional properties of real time-series data.
New algorithm reduces neural net error in contextual bandits.
problem Neural contextual bandits with general activation functions.
method Proposed an efficient algorithm with sublinear regret bound.
result Demonstrated provably sublinear regret bound in finite regime.
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernelstat.ML
Recent works have shown that on sufficiently over-parametrized neural nets, gradient descent with relatively large initialization optimizes a prediction function in the RKHS of the Neural Tangent Kernel (NTK). This analysis leads to global convergence results but does not work when there is a standard regulari…
BCD Nets use variational inference to estimate DAGs with uncertainty.
problem Uncertainty in inferring causal graphs from limited data.
method Variational inference framework for Bayesian DAG estimation.
result BCD Nets outperform maximum-likelihood methods in low data regimes.
Study on how reparametrization affects neural nets' parameter spaces from a geometric perspective.
problem Inconsistencies in flatness measures, optimization, and probability densities under reparametrization.
method Riemannian geometry to study invariance of neural nets under reparametrization.
result Invariance of neural nets is an inherent property if the metric is explicitly represented and transformation rules are correct.
Unified framework for U-Net design and analysis.
problem Understudied design and architecture of U-Nets.
method Theoretical results, Multi-ResNets, function constraints encoding.
result Competitive and superior performance in various tasks.
Based on the tree architecture, the objective of this paper is to design deep neural networks with two or more hidden layers (called deep nets) for realization of radial functions so as to enable rotational invariance for near-optimal function approximation in an arbitrarily high dimensional Euclidian space. It is show…