arXiv research

Improved distributed learning with reduced communication costs.

New algorithm reduces individual regret and communication costs in cooperative bandits.

New method reduces regret and communication costs in federated Q-learning.

BICompFL tackles bi-directional compression challenges in stochastic FL, reducing communication costs by an order of magnitude.

Paper studies fundamental limits of communication in distributed learning.

New algorithms reduce communication costs in collaborative learning.

The high cost of communicating gradients is a major bottleneck for federated learning, as the bandwidth of the participating user devices is limited. Existing gradient compression algorithms are mainly designed for data centers with high-speed network and achieve $O(\sqrt{d} \log d)$ per-iteration communication cost at…

FedElasticNet reduces communication costs and handles client drift in FL.

We study the problem of regret minimization for distributed bandits learning, in which $M$ agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the…

ComEx protocol reduces communication costs in cooperative bandits.

CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.

A distributed algorithm reduces communication cost in linear bandits to near-optimal levels.

A new method reduces communication costs in decentralized optimization.

D2P-Fed improves privacy and communication in federated learning.

New RL algorithms reduce costs for single-agent and federated learning.

LC-FL uses generative models to reduce communication costs in federated learning.

FedLion improves Federated Learning by speeding up convergence and reducing communication costs.

The paper provides bounds on estimation error in a distributed online learning setting.

Learning sparse combinations is a frequent theme in machine learning. In this paper, we study its associated optimization problem in the distributed setting where the elements to be combined are not centrally located but spread over a network. We address the key challenges of balancing communication costs and optimizat…

In this paper, we propose a communication- and computation-efficient algorithm to solve a convex consensus optimization problem defined over a decentralized network. A remarkable existing algorithm to solve this problem is the alternating direction method of multipliers (ADMM), in which at every iteration every node up…

A novel algorithm reduces communication costs in federated best arm identification.

Federated Q-Learning achieves linear regret speedup with low communication cost.

Optimizes communication in federated learning using rate-distortion theory.

New method reduces communication costs in distributed nonconvex optimization.

Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…

New algorithm reduces communication costs in distributed deep learning.

Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.

Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.

The Gradient Boosted Tree (GBT) algorithm is one of the most popular machine learning algorithms used in production, for tasks that include Click-Through Rate (CTR) prediction and learning-to-rank. To deal with the massive datasets available today, many distributed GBT methods have been proposed. However, they all assu…

This paper optimizes AI inference on edge devices with reduced communication and computation costs.

Novel compression method preserves privacy while reducing communication costs.

Paper proposes efficient communication scheme for statistical learning.

New protocol reduces communication costs for heterogeneous bandits over complex networks.

We study distribution testing with communication and memory constraints in the following computational models: (1) The {\em one-pass streaming model} where the goal is to minimize the sample complexity of the protocol subject to a memory constraint, and (2) A {\em distributed model} where the data samples reside at mul…

CSE-FSL reduces communication and storage costs in federated learning.

ADPSGD optimizes SGD convergence by varying averaging periods based on model variance.

One-shot algorithm for feature-distributed kernel PCA reduces communication costs.

We develop a secure aggregation protocol for federated learning that reduces communication and computation costs.

Bayesian approach reduces FL communication cost by one-shot.

Large-scale distributed training of neural networks is often limited by network bandwidth, wherein the communication time overwhelms the local computation time. Motivated by the success of sketching methods in sub-linear/streaming algorithms, we introduce Sketched SGD, an algorithm for carrying out distributed SGD by c…

We consider the issue of multiple agents learning to communicate through reinforcement learning within partially observable environments, with a focus on information asymmetry in the second part of our work. We provide a review of the recent algorithms developed to improve the agents' policy by allowing the sharing of …

Efficient algorithm reduces communication costs in sparse regression.

New sparsification technique for SGD reduces communication costs.

Private anchors affect how information is communicated and can improve or distort transmission.

First-order optimization methods, such as stochastic gradient descent (SGD) and its variants, are widely used in machine learning applications due to their simplicity and low per-iteration costs. However, they often require larger numbers of iterations, with associated communication costs in distributed environments. I…

With the increase in the amount of data and the expansion of model scale, distributed parallel training becomes an important and successful technique to address the optimization challenges. Nevertheless, although distributed stochastic gradient descent (SGD) algorithms can achieve a linear iteration speedup, they are l…

New DCD and BDCD methods for K-SVM and K-RR reduce communication costs.

New batched Langevin Thompson Sampling reduces communication costs for sequential decision making.