Bayesian Optimization for graph node subset functions.
problem Optimizing functions over node subsets in graphs.
method Bayesian Optimization framework for combinatorial optimization on graphs.
result Effectiveness of the proposed BO framework on various graph types and tasks.
Azure (the cloud service provided by Microsoft) is composed of physical computing units which are called nodes. These nodes are controlled by a software component called Fabric Controller (FC), which can consider the nodes to be in one of many different states such as Ready, Unhealthy, Booting, etc. Some of these state…
Distributed optimization often consists of two updating phases: local optimization and inter-node communication. Conventional approaches require working nodes to communicate with the server every one or few iterations to guarantee convergence. In this paper, we establish a completely different conclusion that each node…
Optimal graph classification uses message-passing neural networks.
problem Node classification on sparse graphs with fixed feature dimensions.
method Asymptotic local Bayes optimality, message-passing graph neural networks.
result Optimal message-passing architecture interpolates between MLP and convolution.
Accelerates Bayesian optimization of function networks with partial evaluations.
problem Optimizing expensive-to-evaluate function networks with varying node costs.
method Proposes an accelerated algorithm that uses global Monte Carlo simulations to select node-specific candidate inputs.
result Achieves up to a 16x speedup over the original p-KGFN algorithm while maintaining competitive query efficiency.
Anytime MiniBatch speeds up online distributed optimization by handling slow nodes.
problem Mitigating the impact of slow nodes (stragglers) in distributed optimization.
method Proposes an online distributed optimization method that averages minibatch gradients via consensus rounds.
result Prevents stragglers from slowing progress without wasting work.
Paper analyzes and improves graph convolutional networks for node classification.
problem Over-smoothing in GCNs causes poor performance in node classification tasks.
method Interpreted GCNs from an optimization perspective, introduced metrics to measure over-smoothing, derived a new kernel GCN+.
result GCN+ reduces over-smoothing and improves node classification performance.
A new method reduces the complexity of decentralized optimization.
problem Decentralized stochastic non-convex optimization over a network.
method GT-HSGD, a hybrid variance-reduced method.
result Achieves an oracle complexity of O(n^(-1)ε^(-3)) for small ε.
Improves tree model performance by considering future node splits.
problem Improving tree model performance.
method Next-Depth Lookahead Tree (NDLT) model that evaluates future node splits.
result Enhanced tree model performance.
Study optimal spectral estimator for semi-supervised node classification.
problem Semi-supervised node classification on CSBM with limited labels.
method Spectral estimator inspired by PCA, graph ridge regression, GCN.
result Achieves information-theoretical threshold for exact recovery.
This paper studies node embeddings of networks, revealing their geometric properties.
problem Understanding the geometric properties of node embeddings in random networks.
method Characterization of ergodic limits, generalization, and convex relaxations of random walk node embedding objectives.
result The optimal node embedding Grammians have rank 1 for a nuclear norm relaxation of the non-randomized objective.
In this work, we present a method for node embedding in temporal graphs. We propose an algorithm that learns the evolution of a temporal graph's nodes and edges over time and incorporates this dynamics in a temporal node embedding framework for different graph prediction tasks. We present a joint loss function that cre…
New algorithms for collaborative learning in uncertain, decentralized environments.
problem Byzantine collaborative learning in uncertain, decentralized environments.
method Two asynchronous solutions to averaging agreement, each optimal according to some dimension.
result New algorithms achieve optimal Byzantine resilience and collaborative learning.
Graph InfoClust learns node representations by capturing cluster-level information, improving graph mining tasks.
problem Leveraging cluster-level node information for unsupervised graph representation learning.
method Graph InfoClust (GIC) uses a differentiable K-means method to compute clusters and jointly optimizes mutual information between nodes of the same cluster.
result GIC outperforms state-of-the-art methods in various downstream tasks with a 0.9% to 6.1% gain.
A lot of effort has been invested into characterizing the convergence rates of gradient based algorithms for non-linear convex optimization. Recently, motivated by large datasets and problems in machine learning, the interest has shifted towards distributed optimization. In this work we present a distributed algorithm …
Graph Denoising Policy Network learns robust representations from noisy graphs.
problem Noise sensitivity in graph representation learning.
method Reinforcement learning to select signal neighborhoods and aggregate features.
result Significantly outperforms state-of-the-art methods on node classification tasks.
A GPU framework speeds up BnB for discrete optimization problems.
problem Optimizing large-scale discrete problems with GPU limitations.
method Parallel BnB nodes in GPU batches, using padding and custom kernels.
result One to two orders of magnitude speedup and zero optimality gap.
Policy-GNN optimizes GNN aggregation for diverse node iterations.
problem Optimizing GNN performance by varying aggregation iterations for different nodes.
method Policy-GNN uses a meta-policy framework with deep reinforcement learning to adaptively determine the number of aggregations for each node.
result Policy-GNN significantly outperforms state-of-the-art alternatives on real-world datasets.
The recently developed bag-of-paths (BoP) framework consists in setting a Gibbs-Boltzmann distribution on all feasible paths of a graph. This probability distribution favors short paths over long ones, with a free parameter (the temperature T) controlling the entropic level of the distribution. This formalism enables…
We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum soluti…
Network embedding has proved extremely useful in a variety of network analysis tasks such as node classification, link prediction, and network visualization. Almost all the existing network embedding methods learn to map the node IDs to their corresponding node embeddings. This design principle, however, hinders the ex…
We analyze the convergence of gradient-based optimization algorithms that base their updates on delayed stochastic gradient information. The main application of our results is to the development of gradient-based distributed optimization algorithms where a master node performs parameter updates while worker nodes compu…
Neural Optimal Design of Experiments improves inverse problem solving efficiency.
problem Optimal experimental design in inverse problems.
method Jointly trains a reconstruction model and design variables in a single loop.
result Significantly reduces computational complexity and improves reconstruction accuracy.
The paper connects DNN generalization to node SNR using information theory.
problem Exploring the reasons behind DNN generalization performance.
method Using information theory, the paper derives SNR expressions for DNN nodes and uses them to quantify weight optimization.
result Good SNR performance in DNN nodes correlates with good generalization.
Privacy preserving networks can be modelled as decentralized networks (e.g., sensors, connected objects, smartphones), where communication between nodes of the network is not controlled by an all-knowing, central node. For this type of networks, the main issue is to gather/learn global information on the network (e.g.,…
Proposes GIL for semi-supervised graph classification.
problem Semi-supervised classification of graph data with limited labeled nodes.
method Graph Inference Learning framework that learns node label inference from graph topology.
result Significantly improves semi-supervised node classification performance.
Unified framework for graph coarsening using node features and graph matrices.
problem Dimensionality reduction of large graphs while preserving node features.
method Optimization-based framework that unifies graph learning and dimensionality reduction.
result The learned coarsened graph is ε-similar to the original graph, where ε is a small positive number.
Paper introduces a new method to identify brain hubs using both structural and functional connectivity.
problem Hub node identification in brain networks using only functional connectivity.
method Graph signal processing framework that models functional activity as graph signals on structural connectivity.
result The proposed GraFHub framework identifies hub nodes more accurately than conventional methods.
Optimizes energy efficiency in wireless sensor networks with limited information.
problem Maximizing energy efficiency in energy harvesting wireless sensor networks with limited channel state information.
method Modeling as a Multi-Armed Bandits problem and developing an Upper Confidence Bound algorithm.
result Significant gains in energy efficiency compared to benchmark schemes.
We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized optimization schemes face two major system bottlenecks: stragglers' delay and communication overhead. In this paper, we tackle these bottlene…
New model optimizes oil product distribution via pipelines.
problem Optimizing oil product distribution via pipelines.
method Discrete-time mixed integer linear programming model.
result Significant reductions in pipeline operational cost.
New bounds improve graph node classification using optimal transport.
problem Improving transductive generalization bounds for graph node classification.
method Representation-based generalization bounds via optimal transport, expressed in terms of Wasserstein distances.
result Strong correlation between derived bounds and empirical generalization in graph node classification.
Adaptive GPR-GNN optimizes node feature and topology learning.
problem Optimizing GNNs for both node features and graph topology, regardless of homophily or heterophily.
method Adaptive Universal Generalized PageRank (GPR) Graph Neural Network (GPR-GNN) that learns optimal GPR weights.
result Significant performance improvement on node classification tasks compared to state-of-the-art GNNs.
We investigate the problem of active learning on a given tree whose nodes are assigned binary labels in an adversarial way. Inspired by recent results by Guillory and Bilmes, we characterize (up to constant factors) the optimal placement of queries so to minimize the mistakes made on the non-queried nodes. Our query se…
Polynomial-time algorithm for near-optimal community detection in graphs.
problem Node-private community estimation in stochastic block models.
method Explicit Lipschitz surrogate and accept-reject algorithm for sampling community labels.
result Achieves minimax rates for exact recovery with polynomial-time runtime and logarithmic privacy parameter.
FONDUE identifies ambiguous nodes in networks for better analysis.
problem Ambiguous nodes in network data.
method Network embedding for node disambiguation.
result FONDUE outperforms existing methods in ambiguous node identification.
Modeling generative process of growing graphs has wide applications in social networks and recommendation systems, where cold start problem leads to new nodes isolated from existing graph. Despite the emerging literature in learning graph representation and graph generation, most of them can not handle isolated new nod…
DiSeNE generates interpretable node embeddings without supervision.
problem Lack of interpretability in unsupervised node embeddings.
method Disentangled representation learning with novel objective functions and metrics.
result DiSeNE produces interpretable node embeddings aligned with graph structure.
New BNN model proves optimal posterior concentration and enables practical inference.
problem Improving generalization and uncertainty quantification in deep neural networks.
method Proposes a new node-sparse BNN model with theoretical guarantees and a novel MCMC algorithm for inference.
result Proves near minimax optimal posterior concentration rate and adaptiveness to true model smoothness.
New algorithm finds causal parent nodes without graph learning.
problem Finding causal relationships without knowing the graph structure.
method Developed an efficient algorithm using atomic interventions.
result Algorithm optimally performs interventions with sublinear complexity.
Global pairwise network alignment (GPNA) aims to find a one-to-one node mapping between two networks that identifies conserved network regions. GPNA algorithms optimize node conservation (NC) and edge conservation (EC). NC quantifies topological similarity between nodes. Graphlet-based degree vectors (GDVs) are a state…
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
Paper proposes JDR to denoise graph features and rewire graphs for better node classification.
problem Jointly denoise noisy graph features and rewire graphs for improved node classification.
method Align leading spectral spaces of graph and feature matrices to solve non-convex optimization problem.
result JDR consistently outperforms existing methods on various node classification tasks.
Proposes a Bayesian approach for automatic node selection in sparse neural networks.
problem Reduces structural complexity and computational speedup in large-scale predictive models.
method Uses spike-and-slab Gaussian priors and variational Bayes approach for node selection.
result Establishes variational posterior consistency and optimal contraction rates for sparse networks.
TuneUp improves GNN training by focusing on hard-to-learn nodes.
problem Sub-optimal training of GNNs on all nodes equally.
method Two-stage training: base GNN + tail node improvement.
result Significant improvement in tail node prediction performance.
The analysis of scientific data of increasing size and complexity requires statistical machine learning methods that are both interpretable and predictive. Union of Intersections (UoI), a recently developed framework, is a two-step approach that separates model selection and model estimation. A linear regression algori…
Improved spectral clustering via Gromov-Wasserstein Learning.
problem Optimizing graph partitioning performance.
method Bridge spectral clustering and GWL, using heat kernel for stable node correspondences.
result Improved graph partitioning results without compromising theoretical guarantees.
Study on learning sparse fixed-structure Gaussian Bayesian networks with near-optimal sample complexity.
problem Learning a fixed-structure Gaussian Bayesian network up to a bounded error in total variation distance.
method Analysis of node-wise least squares regression and introduction of BatchAvgLeastSquares and CauchyEst algorithms.
result BatchAvgLeastSquares and CauchyEstTree have near-optimal sample complexity.