DFedAvgM is a decentralized FedAvg with momentum for privacy and communication efficiency.
problem Efficiently train models with privacy and communication efficiency in federated learning.
method Decentralized Federated Averaging with Momentum (DFedAvgM) on clients connected by an undirected graph, using stochastic gradient descent with momentum and quantization.
result DFedAvgM converges under trivial assumptions and can be improved with the PŁ property, numerically verified.
Optimal client sampling reduces communication in federated learning.
problem Efficiently aggregate model updates from distributed clients in federated learning.
method Model weights as an Ornstein-Uhlenbeck process to estimate uncommunicated updates; optimal client sampling strategy.
result Significant reduction in communication with competitive or superior performance.
New method improves FL efficiency by shuffling data, balancing privacy and accuracy.
problem Balancing privacy, communication, and accuracy in federated learning.
method Developed communication-efficient schemes for private mean estimation, combining privacy amplification and shuffled data.
result Achieved same privacy, optimization performance with lower communication cost.
Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue, we introduce two novel strategies to reduce communication costs: (1) the use of lossy compression on the global model sent server-to-client;…
FedElasticNet reduces communication costs and handles client drift in FL.
problem Expensive communication costs and client drift issues in federated learning.
method Leverages elastic net regularizers to sparsify local updates and limit client drift.
result FedElasticNet effectively resolves communication cost and client drift problems.
A new FL algorithm reduces communication overhead by selectively updating model parameters.
problem Data heterogeneity and communication overhead in federated learning.
method Uses age of information metric to selectively update model parameters and group clients with similar data.
result Our method can expedite training and surpass other communication-efficient strategies in efficiency.
Efficiently recovers network community structure from clients' small subgraphs.
problem Recovering community structure in federated myopic learning settings.
method Developed an algorithm to compute consensus signed weighted graph from clients' evidence.
result Exact recovery of network structure is possible in polynomial time under certain conditions.
This paper analyzes and improves convergence in federated learning with biased client selection.
problem Analyzing convergence in federated learning with biased client selection.
method First convergence analysis of federated optimization for biased client selection strategies, proposing Power-of-Choice framework.
result Power-of-Choice strategies converge up to 3 times faster and give 10% higher test accuracy than random selection.
Distributed stochastic gradient descent is an important subroutine in distributed learning. A setting of particular interest is when the clients are mobile devices, where two important concerns are communication efficiency and the privacy of the clients. Several recent works have focused on reducing the communication c…
HeteroFL trains diverse clients with varying capabilities efficiently.
problem Training models on clients with different computational and communication capacities.
method Proposes HeteroFL framework to adaptively distribute subnetworks based on client capabilities.
result Adaptive subnetwork distribution leads to efficient computation and communication.
CSE-FSL reduces communication and storage costs in federated learning.
problem High communication and storage costs in federated learning.
method CSE-FSL uses an auxiliary network to locally update client models and sends only selected epochs' smashed data.
result Significant communication reduction with state-of-the-art convergence and model accuracy.
Federated learning supports exact support recovery with minimal communication.
problem Learning the exact support of sparse linear regression in federated learning.
method One-shot communication algorithm for exact support recovery without optimization.
result Polynomial sample complexity and logarithmic number of clients required.
MpFL models clients as strategic players to reach equilibrium with less communication.
problem Real-world clients act independently with individual objectives, not aligned with a shared global model.
method MpFL uses game-theoretic modeling and PEARL-SGD algorithm for local updates and communication.
result PEARL-SGD reaches an equilibrium with less communication than non-local updates in stochastic setup.
A novel algorithm reduces communication costs in federated best arm identification.
problem Identifying the best arm in a federated multi-armed bandit setup with minimal communication cost.
method Proposes a novel algorithm called FedElim that communicates only in exponential time steps.
result Demonstrates that communication is almost cost-free in FedElim, with a total cost at most 3 times the maximum under its variant.
FedLoRU improves FL efficiency by using low-rank updates.
problem Communication inefficiency and performance reduction in Federated Learning.
method Proposes FedLoRU, a low-rank update framework for FL, which reduces communication costs while maintaining performance.
result FedLoRU achieves convergence rates similar to FedAvg and is robust to heterogeneous and large numbers of clients.
We compare communication efficiencies of two compelling distributed machine learning approaches of split learning and federated learning. We show useful settings under which each method outperforms the other in terms of communication efficiency. We consider various practical scenarios of distributed learning setup and …
Federated learning obtains a central model on the server by aggregating models trained locally on clients. As a result, federated learning does not require clients to upload their data to the server, thereby preserving the data privacy of the clients. One challenge in federated learning is to reduce the client-server c…
Bayesian approach reduces FL communication cost by one-shot.
problem High communication cost in optimization-based FL for high-dimensional models.
method Bayesian pseudocoresets and function-space inference for one-shot FL.
result Achieves prediction performance competitive to state-of-the-art with up to 2 orders of magnitude reduction in communication cost.
Federated Learning allows multiple parties to jointly train a deep learning model on their combined data, without any of the participants having to reveal their local data to a centralized server. This form of privacy-preserving collaborative learning however comes at the cost of a significant communication overhead du…
A new approach corrects bias in federated learning due to varying communication links.
problem Bias in federated learning due to non-uniform and time-varying communication links.
method Proposes Federated Postponed Broadcast (FedPBC) to correct bias in Federated Average (FedAvg).
result FedPBC converges to a stationary point of the global objective, overcoming bias caused by varying communication links.
A federated minimax framework for heterogeneous clients.
problem Training with edge devices having different datasets and capabilities.
method Proposes a federated minimax optimization framework with normalized updates.
result Improves convergence and communication complexity for nonconvex functions.
New algorithm reduces federated learning rounds and improves privacy.
problem Inefficient synchronous federated learning causing scalability issues.
method Asynchronous federated learning with reduced communication and differential privacy via Gaussian noise.
result The algorithm reduces waiting times and network communication, making federated learning more scalable and private.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
problem Byzantine attacks and communication inefficiency in federated learning.
method Transformed robust aggregation for zero-order optimization under client heterogeneity.
result CyBeR-0 achieves stable performance with minimal communication costs and reduced memory usage.
FLowDUP trains personalized models with unlabeled clients in low dimensions.
problem Training personalized models on clients with only unlabeled data.
method FLowDUP uses a forward pass with unlabeled data and a transductive PAC-Bayesian bound to generate personalized models in a low-dimensional subspace.
result FLowDUP enables efficient communication and computation with personalized models generated from unlabeled data.
SCAFFLSA reduces communication complexity for federated learning with heterogeneous clients.
problem Quantifying and reducing communication complexity in federated learning with heterogeneous clients.
method Proposes SCAFFLSA, a variant of FedLSA using control variates to correct for client drift.
result SCAFFLSA achieves logarithmic communication complexity for statistically heterogeneous agents, scaling with the inverse of the desired accuracy.
New method reduces FL communication cost by leveraging server's side information.
problem High communication cost in federated learning.
method Exploits server's side information to compress client updates.
result Up to 82 times smaller bitrate with higher accuracy.
FastSecAgg improves federated learning security and efficiency.
problem Privacy leakage in federated learning due to model parameter sharing.
method Introduces FastSecAgg, a secure aggregation protocol with FFT-based multi-secret sharing (FastShare).
result Efficient in computation and communication, robust to client dropouts.
Byzantine-resilient federated learning with local iterations and robust mean estimation.
problem Byzantine clients disrupt federated learning with local iterations.
method Local SGD iterations, robust mean estimation, and matrix concentration result.
result Convergence analysis for strongly-convex and non-convex smooth objectives in heterogeneous data settings.
Optimal privacy and accuracy in distributed mean estimation with compression.
problem Achieving optimal accuracy under privacy and communication constraints.
method Compression to reduce communication while maintaining privacy and accuracy.
result Achieves optimal error with significantly reduced communication.
This paper addresses privacy in federated learning with wireless clients and base stations.
problem Privacy of clients' data in federated learning with hierarchical wireless architecture.
method Derives communication cost limits and introduces private aggregation schemes tailored for hierarchical wireless systems.
result Private aggregation schemes reduce communication costs by multiplicative factors compared to information-theoretic limits.
Collaborative learning allows participants to jointly train a model without data sharing. To update the model parameters, the central server broadcasts model parameters to the clients, and the clients send updating directions such as gradients to the server. While data do not leave a client device, the communicated gra…
Optimizes communication in federated learning using rate-distortion theory.
problem Reduces communication cost in federated learning while maintaining model accuracy.
method Applies rate-distortion theory to model updates, proposing distortion as a proxy for accuracy.
result Near-optimal communication reduction, outperforming other methods on a FL benchmark.
New algorithm tackles multi-agent bandits with heavy-tailed data.
problem Maximizing system performance in multi-agent settings with heavy-tailed data.
method Algorithm exploits hub-like structures and synchronization among clients.
result Regret bound of O(M1−α1logT) for homogeneous settings, O(MlogT) for heterogeneous. FedAMD framework improves federated learning with partial client participation.
problem Data heterogeneity and inactive client updates in partial client participation.
method Anchor sampling divides clients into anchor and miner groups, using large and small batches respectively.
result FedAMD achieves faster convergence and improved model performance compared to state-of-the-art methods.
FetchSGD reduces communication in federated learning with sketching.
problem Communication bottlenecks and convergence issues in federated learning.
method FetchSGD uses Count Sketch to compress and merge model updates efficiently.
result FetchSGD achieves high compression rates and good convergence without sparse client participation.
Federated learning framework improves model generalization and privacy.
problem Communication overhead and statistical heterogeneity in FL.
method Prototypes and lightweight adapters for local model refinement.
result Improves classification accuracy over baseline algorithms.
Quantum federated learning improves with non-IID data using one-shot communication.
problem Performance degradation in federated learning with non-IID data.
method Quantum algorithms and local density estimators for non-IID data.
result One-shot communication complexity for non-IID quantum federated learning.
Semi-Federated Learning clusters clients for efficient model training.
problem Efficiency and data distribution challenges in Federated Learning.
method Local clustering and in-cluster training with a sequential training manner.
result Semi-Federated Learning reduces communication costs and improves robustness to Non-IID data.
GradSkip reduces local training steps for better communication efficiency.
problem High communication costs in distributed optimization.
method GradSkip redesigns ProxSkip to allow clients with less important data to take fewer local training steps.
result GradSkip converges linearly with reduced local training steps and same accelerated communication complexity.
Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' w…
A new algorithm reduces communication costs for collaborative decision-making across clients.
problem Collaborative decision-making with sparse rewards and heterogeneous contexts.
method Federated Lasso algorithm for sparse linear contextual bandits.
result Achieves near-optimal regret with logarithmic communication costs.
FedSTaS stratifies and samples clients for efficient FL.
problem Inefficient client sampling in federated learning.
method Stratifies clients based on compressed gradients, uses Neyman allocation for sampling, and samples local data uniformly.
result FedSTaS achieves higher accuracy than FedSTS in fixed training rounds.
Synthetic data replaces model updates in federated learning.
problem High computational costs in transmitting model parameters.
method Transmitting synthetic data instead of model updates.
result Reduces communication costs by more than an order of magnitude.
LotteryFL improves federated learning efficiency and personalization.
problem Statistical heterogeneity and communication efficiency in federated learning.
method Exploiting Lottery Ticket hypothesis to learn compact lottery networks.
result LotteryFL achieves better personalization and reduces communication cost.
LC-FL uses generative models to reduce communication costs in federated learning.
problem High communication costs and strict model homogeneity in federated learning.
method LC-FL employs generative models to transmit data and aggregate models.
result LC-FL reduces communication costs and supports heterogeneous models.
FedGLOMO accelerates FL convergence for non-convex functions.
problem Efficiently solving non-convex optimization problems in federated learning with client heterogeneity.
method Combines global and local momentum updates to reduce variance and improve convergence rate.
result Achieves O(ε−1.5) convergence to ε-stationary point, compared to O(ε−2). Paper introduces MoTEF for faster decentralized optimization with compressed communication.
problem Efficiency bottleneck in decentralized machine learning applications.
method Integrates communication compression with Momentum Tracking and Error Feedback.
result Significantly outperforms existing methods under arbitrary data heterogeneity.
Federated learning (FL) rests on the notion of training a global model in a decentralized manner. Under this setting, mobile devices perform computations on their local data before uploading the required updates to improve the global model. However, when the participating clients implement an uncoordinated computation …