arXiv research

Research shows minimal communication limits adaptive function estimation rates.

A new federated learning method reduces communication costs and improves adaptivity.

FedLion improves Federated Learning by speeding up convergence and reducing communication costs.

Paper proposes an adaptive gradient method for federated learning.

AdaQuantFL reduces communication in federated learning by adaptively quantizing model updates.

New method reduces communication costs in distributed nonconvex optimization.

When scaling distributed training, the communication overhead is often the bottleneck. In this paper, we propose a novel SGD variant with reduced communication and adaptive learning rates. We prove the convergence of the proposed algorithm for smooth but non-convex problems. Empirical results show that the proposed alg…

Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.

Distributed sensors compress and send features to a fusion center for linear regression.

A new algorithm reduces communication in decentralized optimization.

We study distributed estimation methods under communication constraints in a distributed version of the nonparametric random design regression model. We derive minimax lower bounds and exhibit methods that attain those bounds. Moreover, we show that adaptive estimation is possible in this setting.

Study exact community recovery in noisy SBM with limited queries.

MACC learns communication protocols by adapting counterfactual reasoning.

We introduce the community exploration problem that has many real-world applications such as online advertising. In the problem, an explorer allocates limited budget to explore communities so as to maximize the number of members he could meet. We provide a systematic study of the community exploration problem, from off…

Paper closes the gap in MP-MAB problems with novel adaptive communication and exploration.

COMP-AMS optimizes distributed training with compressed gradients, achieving similar accuracy with less communication.

Multiplex networks, a special type of multilayer networks, are increasingly applied in many domains ranging from social media analytics to biology. A common task in these applications concerns the detection of community structures. Many existing algorithms for community detection in multiplexes attempt to detect commun…

To reduce the long training time of large deep neural network (DNN) models, distributed synchronous stochastic gradient descent (S-SGD) is commonly used on a cluster of workers. However, the speedup brought by multiple workers is limited by the communication overhead. Two approaches, namely pipelining and gradient spar…

MARINA-P improves non-smooth federated optimization with adaptive stepsizes.

Study examines Fed's pandemic communication strategies.

Flexible framework improves communication efficiency across various systems.

Adaptive batch sizes improve local gradient methods in distributed training.

This work proposes ACTC for adaptive distributed learning under communication constraints.

Adaptive quantization improves SGD accuracy in data-parallel settings.

This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation. Simple rules are designed to detect slowly-varying gradients and, therefore, trigger the reuse of outdated gradients. The resultant gra…

This paper targets solving distributed machine learning problems such as federated learning in a communication-efficient fashion. A class of new stochastic gradient descent (SGD) approaches have been developed, which can be viewed as the stochastic generalization to the recently developed lazily aggregated gradient (LA…

Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random communication delays. This work considers a distributed training framework where each worker node is allowed to perform local model updates a…

Community detection is a central problem of network data analysis. Given a network, the goal of community detection is to partition the network nodes into a small number of clusters, which could often help reveal interesting structures. The present paper studies community detection in Degree-Corrected Block Models (DCB…

New algorithms reduce communication costs in collaborative learning.

With the recent advances in complex networks theory, graph-based techniques for image segmentation has attracted great attention recently. In order to segment the image into meaningful connected components, this paper proposes an image segmentation general framework using complex networks based community detection algo…

Interneurons improve learning in neural networks by accelerating convergence.

This paper improves MARL for networked systems through new protocols and discount factors.

Paper identifies sparse structures and communities in heterogeneous graphical models.

The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…

Paper develops an efficient mean estimator for 1-bit communication constraints.

This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We summarize the general categories of topology for communication structures in MARL literature, which are often manually specified. Then we propose a novel framework termed as Learning Stru…

Graph convolutional networks (GCNs) are a family of neural network models that perform inference on graph data by interleaving vertex-wise operations and message-passing exchanges across nodes. Concerning the latter, two key questions arise: (i) how to design a differentiable exchange protocol (e.g., a 1-hop Laplacian …

Federated learning framework improves model generalization and privacy.

Adaptive personalized federated learning improves local model personalization.

We develop a secure aggregation protocol for federated learning that reduces communication and computation costs.

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

Few-shot domain adaptation improves autoencoder performance in changing wireless channels.

A new framework maximizes influence spread in social networks by accounting for inter-community diffusion.

Network agents solve adaptive regression problems with compressed signals.

Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the \emph{`local-SGD'} model in which the workers train their model i…

This paper presents a novel spectral algorithm with additive clustering designed to identify overlapping communities in networks. The algorithm is based on geometric properties of the spectrum of the expected adjacency matrix in a random graph model that we call stochastic blockmodel with overlap (SBMO). An adaptive ve…

New method learns dynamic brain communication patterns across regions.

Enhances FDA for robust, efficient federated learning.