New method learns dynamic brain communication patterns across regions.
problem Current methods struggle with time-varying brain communications and scalability.
method Adaptive Delay Model (ADM) using Markovian Gaussian Processes.
result Captures dynamic neural communication patterns over time.
This paper explores advancements in neural network communication for distributed settings.
problem Challenges in training and evaluating neural networks in distributed, privacy-constrained environments.
method Development and evaluation of novel training and evaluation schemes for neural networks.
result Advantages of novel approaches over traditional cloud solutions, including privacy-preservation, increased security, and high training speed.
Community detection is a fundamental problem in machine learning. While deep learning has shown great promise in many graphrelated tasks, developing neural models for community detection has received surprisingly little attention. The few existing approaches focus on detecting disjoint communities, even though communit…
New algorithms reduce communication in GNN training.
problem Higher communication costs in GNNs due to sparse connectivity.
method Parallel algorithms for sparse-dense matrix multiplication.
result Asymptotic reduction in communication compared to previous methods.
A layered neural network is now one of the most common choices for the prediction of high-dimensional practical data sets, where the relationship between input and output data is complex and cannot be represented well by simple conventional models. Its effectiveness is shown in various tasks, however, the lack of inter…
Proposes a new model for community detection using neural priors.
problem Community detection in graphs with node attributes.
method Neural-prior stochastic block model with belief propagation and approximate message passing algorithm.
result Identifies phase transitions and algorithmically hard regions for community detection.
A novel framework for adaptive multi-agent communication in reinforcement learning.
problem Manual specification of communication structures in multi-agent reinforcement learning.
method Learning Structured Communication (LSC) framework using hierarchical graph neural networks.
result Adaptive hierarchical formations and efficient message propagation among agents.
GNNs robustness in community detection is studied with various perturbations.
problem Understanding GNNs robustness in community detection tasks.
method Systematic computational evaluation of six GNN architectures on synthetic and real-world networks.
result Supervised GNNs achieve higher baseline accuracy, while DMoN shows stronger resilience to perturbations.
Stochastic Gradient Descent (SGD) is the most popular algorithm for training deep neural networks (DNNs). As larger networks and datasets cause longer training times, training on distributed systems is common and distributed SGD variants, mainly asynchronous and synchronous SGD, are widely used. Asynchronous SGD is com…
Study neural communication systems with bandwidth-limited channels.
problem Reliable message transmission despite noisy channels.
method Jointly model compression and error correction with neural networks; introduce prior for missing information; use auxiliary latent variables.
result Joint neural communication systems outperform separate models under expected information loss.
I2C enables agents to learn efficient communication without redundancy.
problem Redundant broadcast communication in multi-agent cooperation.
method I2C learns a prior for agent-agent communication via causal inference and reinforcement learning.
result I2C reduces communication overhead and improves multi-agent cooperative performance.
Rewiring GNNs to optimize community and feature alignment improves their performance.
problem Improving GNNs' performance by addressing over-squashing and generalization issues.
method Three rewiring strategies: ComMa, FeaSt, and ComFy, targeting community structure, node labels, and their alignment.
result Rewiring strategies enhance GNNs' performance by optimizing label-community alignment.
Efficiently maximizes AUC with deep nets, reducing communication rounds.
problem Maximizing AUC with deep neural networks in a distributed setting.
method Communication-efficient distributed optimization algorithm for non-convex concave AUC maximization.
result Achieves linear speedup with significantly fewer communication rounds.
Improved community detection in graphs with probabilistic models.
problem Lack of probabilistic formulation and fixed number of communities in GNN-based methods.
method Combines GNNs with amortized clustering for variable numbers of clusters.
result Improved performance on synthetic and real datasets compared to previous methods.
Proposes a secure communication method independent of eavesdropper's decoder.
problem Lack of practical security constraints in existing methods.
method Dual MINE-based neural secure communications model.
result Security performance is not affected by eavesdropper's decoding means.
Novel neural framework for scalable community detection and link prediction.
problem Efficiently learning graph topology and community structure in large graphs.
method Proposes a joint SBM loss function for community detection and link prediction, leveraging neural networks.
result Framework efficiently handles large graphs with a single forward pass, improving scalability and performance.
A2SGD reduces distributed SGD communication to O(1) per worker.
problem Heavy communication costs in distributed SGD for large models.
method Two-level gradient averaging to consolidate gradients to two local averages.
result Achieves O(1) communication complexity per worker, significantly reducing traffic and training time.
IoT nodes compress measurements into DNN outputs for efficient communication.
problem Efficient communication of high-dimensional IoT data with limited bandwidth.
method Modeling IoT node measurements as DNN intermediate outputs and optimizing model parameters.
result Approximately 96% reduction in transmissions with only 2.5% loss in inference accuracy.
Optimal gradient quantization reduces communication costs in distributed deep learning.
problem High communication costs in distributed training of deep neural networks.
method Deduced optimal gradient quantization conditions for binary and multi-level quantization, developed novel schemes for dynamic quantization levels.
result Demonstrated superior performance of proposed quantization schemes on CIFAR and ImageNet datasets.
Paper proposes an adaptive gradient method for federated learning.
problem Communication efficiency in federated learning.
method Adaptive gradient method for federated learning.
result Guaranteed convergence and communication efficiency in federated learning.
This paper proposes a method to learn graph representations by partitioning edges into communities.
problem Graph neural networks ignore how edges are formed, leading to suboptimal representation learning.
method Introduces a generative model to partition edges into community-specific weighted edges, then uses these for GNN-based inference and classification.
result The method learns discriminative representations for both node-level and graph-level classification tasks.
Interneurons improve learning in neural networks by accelerating convergence.
problem Rapid adaptation to changing input statistics in neural networks.
method Two mathematically tractable recurrent linear neural networks were compared: one with direct recurrent connections and the other with interneurons that mediate recurrent communication.
result The network with interneurons converges more quickly than the network with direct recurrent connections, scaling logarithmically with initialization spectrum.
Two new coding schemes improve the efficient communication of noisy data.
problem Efficient communication of noisy data in machine learning.
method Ordered Random Coding (ORC) and Hybrid Coding Scheme.
result Improved coding schemes over existing approaches.
DP-REC combines privacy and communication efficiency in federated learning.
problem Combining privacy and communication efficiency in federated learning.
method DP-REC uses Relative Entropy Coding (REC) for compression and a minor modification for differential privacy.
result DP-REC reduces communication costs while maintaining privacy comparable to state-of-the-art methods.
A new algorithm reduces communication in deep learning training.
problem Slow convergence in deep neural networks due to exploding gradients.
method Communication-efficient distributed gradient clipping algorithm.
result Algorithm achieves linear speedup and reduced communication rounds.
Graph neural networks help AI agents learn more complex language.
problem Understanding how AI agents learn and use language.
method Developed graph referential games to compare different AI models.
result Graph neural networks enable AI to learn more complex, compositional language.
For high data rate wireless communication systems, developing an efficient channel estimation approach is extremely vital for channel detection and signal recovery. With the trend of high-mobility wireless communications between vehicles and vehicles-to-infrastructure (V2I), V2I communications pose additional challenge…
End-to-end learning of codes for secure BPSK communication in Gaussian wiretap channel.
problem Secure communication in the presence of an adversary for BPSK modulation.
method Learning finite-length codes using deep neural networks (DNNs) with mutual information estimation.
result Learned codes achieve near-optimal security performance, as demonstrated by numerical results.
RiskNet predicts penalties in unreliable communication networks using GNNs.
problem Predicting penalties in networks with unreliable resources.
method Graph Neural Network (GNN) based approach trained on random graphs.
result Precisely models penalties across various network topologies.
GRADE models evolving graph dynamics by learning node and community representations.
problem Lack of tools to study temporal community dynamics in evolving graphs.
method GRADE is a probabilistic model that learns evolving node and community representations via a random walk prior and variational inference.
result GRADE outperforms baselines in dynamic link prediction and dynamic community detection.
Shared workspace improves neural module coordination in deep learning.
problem Pairwise interactions in deep learning models lack global coordination.
method Introduce a shared global workspace with bandwidth limitations among neural modules.
result Capacity limitations encourage specialization and synchronization.
Federated learning is an emerging technique used to prevent the leakage of private information. Unlike centralized learning that needs to collect data from users and store them collectively on a cloud server, federated learning makes it possible to learn a global model while the data are distributed on the users' devic…
DBGDGM models dynamic brain graphs for better understanding brain function.
problem Previous brain graph models ignore temporal dynamics, limiting their usefulness.
method DBGDGM clusters brain regions into evolving communities and learns dynamic node embeddings.
result DBGDGM outperforms baselines in graph generation, dynamic link prediction, and graph classification.
Adversarial perturbations and RIS interaction vectors improve covert communication.
problem Covert communication in the presence of RISs.
method Designing RIS interaction vectors to balance receiver and eavesdropper detection, adding adversarial perturbations to signals.
result Adversarial perturbations and RIS interaction vectors can be jointly designed to boost covert communications.
A new algorithm reduces communication in graph-structured VFL.
problem Training GNNs in VFL with interrelated samples.
method Model splitting and lazy aggregation.
result GLASU trains GNNs with reduced communication, matching centralized performance.
Neural node embeddings have recently emerged as a powerful representation for supervised learning tasks involving graph-structured data. We leverage this recent advance to develop a novel algorithm for unsupervised community discovery in graphs. Through extensive experimental studies on simulated and real-world data, w…
Recent successes and advances in Deep Neural Networks (DNN) in machine vision and Natural Language Processing (NLP) have motivated their use in traditional signal processing and communications systems. In this paper, we present results of such applications to the problem of automatic modulation recognition. Variations …
Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes are becoming increasingly relevant. A major issue in distributed training is the limited communication bandwidth between contributing nodes o…
AdaQuantFL reduces communication in federated learning by adaptively quantizing model updates.
problem Efficient communication of model updates in federated learning with high-dimensional models and limited bandwidth.
method AdaQuantFL uses adaptive quantization to reduce the number of bits for model updates while maintaining low error floor.
result AdaQuantFL converges in fewer communicated bits compared to fixed quantization levels, with minimal impact on accuracy.
This letter tackles channel assignment in uplink wireless communication systems.
problem Maximizing the sum rate of all users in uplink wireless communication systems with integer channel assignment constraints.
method A convex optimization based algorithm is used to find the optimal channel assignment. Machine learning approaches, including CNNs, FNNs, random forest, and GRUs, are employed to reduce computation time.
result Machine learning methods largely reduce computation time with slightly compromised prediction accuracy.
FedLog reduces communication in federated learning by sharing data summaries.
problem Significant communication overhead in federated learning with large model parameters.
method Shares minimal sufficient statistics via Bayesian inference and differential privacy.
result High learning accuracy with low communication overhead.
New algorithm reduces communication traffic in decentralized learning.
problem Communication bottleneck in decentralized learning for low-bandwidth workers.
method Sparsification and adaptive peer selection to reduce communication traffic.
result Significant reduction in communication traffic compared to existing methods.
FedFisher improves one-shot FL by using Fisher information.
problem Reducing communication rounds in federated learning.
method Bayesian perspective, Fisher information matrices, diagonal Fisher, K-FAC approximation.
result FedFisher achieves vanishingly small error in two-layer neural networks.
Physics-based deep learning improves fiber-optic communication efficiency.
problem Improving signal propagation in fiber-optic communication systems.
method Parameterizing the split-step method of solving the nonlinear Schrödinger equation as a deep neural network.
result Filters can be pruned to as few as 3 taps/step without sacrificing performance.
Traditionally, community detection in graphs can be solved using spectral methods or posterior inference under probabilistic graphical models. Focusing on random graph families such as the stochastic block model, recent research has unified both approaches and identified both statistical and computational detection thr…
A new gradient quantization scheme improves communication efficiency in distributed training.
problem Efficiently compressing gradients for parallel training of large models.
method Proposes a new gradient quantization scheme with theoretical guarantees and empirical performance.
result The new scheme matches and exceeds the performance of existing methods.
Large-scale distributed training of neural networks is often limited by network bandwidth, wherein the communication time overwhelms the local computation time. Motivated by the success of sketching methods in sub-linear/streaming algorithms, we introduce Sketched SGD, an algorithm for carrying out distributed SGD by c…
The deep learning trend has recently impacted a variety of fields, including communication systems, where various approaches have explored the application of neural networks in place of traditional designs. Neural networks flexibly allow for data/simulation-driven optimization, but are often employed as black boxes det…