FedAMD framework improves federated learning with partial client participation.
problem Data heterogeneity and inactive client updates in partial client participation.
method Anchor sampling divides clients into anchor and miner groups, using large and small batches respectively.
result FedAMD achieves faster convergence and improved model performance compared to state-of-the-art methods.
Unified analysis of FL with arbitrary client participation.
problem Challenges of intermittent client availability and efficiency in FL.
method Introduces a generalized FedAvg and novel analysis capturing client participation.
result Unified convergence upper bounds for various participation patterns.
FedAvg converges linearly to global minimum in federated learning with partial participation.
problem Challenges in federated learning with partial client participation.
method Federated averaging (FedAvg) method for over-parameterized neural networks.
result FedAvg converges to global minimum at a linear rate after t iterations.
New framework provides privacy guarantees for practical federated learning.
problem Inadequate privacy guarantees for federated learning due to restrictive assumptions.
method Fed- α \alpha α -NormEC, integrating multiple local updates, partial client participation, and standard assumptions. result Provably convergent and differentially private federated learning framework.
This paper analyzes and improves convergence in federated learning with biased client selection.
problem Analyzing convergence in federated learning with biased client selection.
method First convergence analysis of federated optimization for biased client selection strategies, proposing Power-of-Choice framework.
result Power-of-Choice strategies converge up to 3 times faster and give 10% higher test accuracy than random selection.
FedSGM tackles constrained federated learning with unified framework.
problem Functional constraints, communication bottlenecks, local updates, and partial client participation in federated learning.
method Unified framework based on switching gradient method, incorporating bi-directional error feedback, and soft switching for stability.
result Achieves O ( 1 / T ) \boldsymbol{\mathcal{O}}(1/\sqrt{T}) O ( 1/ T ) convergence rate with high-probability bounds decoupling from sampling noise. This paper analyzes error feedback in compressed federated learning for non-convex optimization problems.
problem Reducing communication cost in federated learning with biased gradient compression.
method Proposes Fed-EF, a compressed federated learning scheme with error feedback, and analyzes its convergence rate and performance under partial client participation.
result Fed-EF can match the convergence rate of full-precision FL under data heterogeneity with a linear speedup and no extra slow-down factor due to stale error compensation.
Proposes SROF for row-wise fusion in federated learning for multivariate responses.
problem Heterogeneous client models with shared variable-level structure.
method Sparse Row-wise Fusion (SROF) regularizer and RowFed algorithm.
result Empirically shows consistent error reduction and stronger variable-level cluster recovery.
Paper tackles unknown participation in FL, proposing FedAU for better performance.
problem Unknown participation statistics in federated learning impact performance.
method Adapting aggregation weights in FedAvg based on participation history.
result FedAU converges to optimal solution and has desirable properties.
This paper addresses privacy in federated learning with wireless clients and base stations.
problem Privacy of clients' data in federated learning with hierarchical wireless architecture.
method Derives communication cost limits and introduces private aggregation schemes tailored for hierarchical wireless systems.
result Private aggregation schemes reduce communication costs by multiplicative factors compared to information-theoretic limits.
FedAVOT improves federated learning by aligning user distributions.
problem Partial client participation leads to biased and unstable updates in federated learning.
method Formulates aggregation as masked optimal transport to align availability and importance distributions.
result Achieves a standard O(1/√T) rate, independent of the number of participating users per round.
A new method for efficient online federated learning reduces communication overhead.
problem Real-world limitations in online federated learning, such as heterogeneous client participation and communication delays.
method Proposes a communication-efficient asynchronous online federated learning (PAO-Fed) strategy.
result Achieves the same convergence properties as online federated stochastic gradient while reducing communication overhead by 98 percent.
New algorithm achieves linear speedup in non-i.i.d. federated bilevel learning.
problem Linear speedup in convergence for non-i.i.d. datasets in federated bilevel optimization.
method Proposes FedMBO with a novel client sampling scheme for non-i.i.d. datasets.
result Achieves a convergence rate of O(1/√(nK) + 1/K + √n/K³/²).
FedSTaS stratifies and samples clients for efficient FL.
problem Inefficient client sampling in federated learning.
method Stratifies clients based on compressed gradients, uses Neyman allocation for sampling, and samples local data uniformly.
result FedSTaS achieves higher accuracy than FedSTS in fixed training rounds.
A federated minimax framework for heterogeneous clients.
problem Training with edge devices having different datasets and capabilities.
method Proposes a federated minimax optimization framework with normalized updates.
result Improves convergence and communication complexity for nonconvex functions.
Entropy regularization improves sparse model discovery in federated learning.
problem Sparse model discovery in federated learning with limited data.
method Entropy regularization of gate distributions for probabilistic sparse model exploration.
result Entropy regularization leads to better sparse model recovery and performance.
Federated learning (FL) rests on the notion of training a global model in a decentralized manner. Under this setting, mobile devices perform computations on their local data before uploading the required updates to improve the global model. However, when the participating clients implement an uncoordinated computation …
This work analyzes generalization in federated learning using information theory.
problem Generalization performance in federated learning is less explored compared to centralized learning.
method The work applies an information-theoretic analysis via the conditional mutual information (CMI) framework to study federated learning's two-level generalization.
result The work derives multiple CMI-based bounds, including hypothesis-based CMI bounds and fast-rate evaluated CMI bounds, which improve convergence rates for specific model aggregation strategies and structured loss functions.
Optimal privacy and accuracy in distributed mean estimation with compression.
problem Achieving optimal accuracy under privacy and communication constraints.
method Compression to reduce communication while maintaining privacy and accuracy.
result Achieves optimal error with significantly reduced communication.
This paper optimizes brokerage contracts for multiple clients trading a single asset.
problem Optimizing brokerage contracts for multiple clients trading a single asset.
method Endogenously determines clients' reservation values and strategically chooses clients. Characterizes optimal portfolios computationally.
result Characterizes optimal portfolios of clients and their profits, showing dependence on price impact coefficients.
DaringFed incentivizes clients in OFL with dynamic rewards under TII.
problem Designing incentives for OFL clients under dynamic, incomplete information.
method Formulated as a dynamic signaling and pricing allocation problem in a Bayesian persuasion game.
result Optimal design of DaringFed improves accuracy and convergence speed by 16.99%.
Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to s…
Personalized federated learning for diverse client objectives.
problem Training a single global model across diverse local datasets is not optimal.
method Efficiently calculates optimal weighted model combinations for each client based on their specific objectives.
result Our method outperforms existing alternatives and enables new personalized features.
FetchSGD reduces communication in federated learning with sketching.
problem Communication bottlenecks and convergence issues in federated learning.
method FetchSGD uses Count Sketch to compress and merge model updates efficiently.
result FetchSGD achieves high compression rates and good convergence without sparse client participation.
A novel incentive mechanism improves fairness and participation in federated learning.
problem Low-quality clients and lack of fairness in federated learning.
method Client selection process and money transfer mechanism to ensure fairness and participation.
result The proposed incentive mechanism improves the duration and fairness of federated learning.
Paper explores how to design federated learning protocols that benefit all participants while maintaining privacy.
problem Privacy concerns undermine the accuracy benefits of federated learning in privacy-sensitive domains.
method The paper provides conditions for mutually beneficial federated learning protocols and designs protocols that maximize total utility and accuracy.
result The paper demonstrates that federated learning can be designed to be mutually beneficial, striking a balance between privacy and model accuracy.
A new algorithm improves Bayesian federated learning by reducing communication overhead.
problem Bayesian federated learning constraints, including privacy, data ownership, and communication overhead.
method Proposes Quantised Langevin Stochastic Dynamics (QLSD) for Bayesian federated learning, using gradient compression and variance reduction techniques.
result Non-asymptotic and asymptotic convergence guarantees for QLSD and its improved versions.
Federated learning studies separate client data and distribution gaps.
problem Understanding performance differences in federated learning across different datasets.
method Proposed a framework to disentangle out-of-sample and participation gaps.
result Dataset synthesis strategy is crucial for realistic simulations of federated learning generalization.
Unified Bayesian framework for clustered federated learning improves model performance.
problem Handling non-IID client data in federated learning.
method A unified Bayesian framework for clustered federated learning that associates clients to clusters and proposes practical algorithms for data associations.
result The proposed framework increases model performance by circumventing the need for unique client-cluster associations.
SimFBO simplifies FBO, making it more efficient and flexible.
problem Complex nested optimization in machine learning and edge computing.
method Proposes SimFBO, a simple and flexible FBO framework with improved communication efficiency.
result SimFBO and ShroFBO achieve linear convergence speedup and improved sample and communication complexities.
Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue, we introduce two novel strategies to reduce communication costs: (1) the use of lossy compression on the global model sent server-to-client;…
This paper analyzes deep federated learning for low-dimensional data, revealing intrinsic dimensionality's role in convergence rates.
problem Insufficient investigation of generalization error in heterogeneous federated learning, especially for low-dimensional data.
method Statistical analysis of deep federated regression in a two-stage sampling model.
result Intrinsic dimensionality, characterized by entropic dimension, determines convergence rates for deep learners.
Collaborative learning allows participants to jointly train a model without data sharing. To update the model parameters, the central server broadcasts model parameters to the clients, and the clients send updating directions such as gradients to the server. While data do not leave a client device, the communicated gra…
This paper surveys techniques to personalize federated learning models.
problem Personalized models outperform shared models for some clients, reducing participation.
method Surveys recent research on personalizing federated learning models.
result Personalization techniques improve model performance for individual clients.
Federated Learning allows multiple parties to jointly train a deep learning model on their combined data, without any of the participants having to reveal their local data to a centralized server. This form of privacy-preserving collaborative learning however comes at the cost of a significant communication overhead du…
This paper proposes a cooperative mechanism for mitigating the performance degradation due to non-independent-and-identically-distributed (non-IID) data in collaborative machine learning (ML), namely federated learning (FL), which trains an ML model using the rich data and computational resources of mobile clients with…
Two algorithms improve federated learning efficiency and resilience.
problem Scalability issues in federated learning due to communication, privacy, and Byzantine attacks.
method Proposes two algorithms, Ada-StoSign and β β β -StoSign, that compress gradients into bit vectors to reduce communication. result Ada-StoSign converges with a rate of O ( log T / T + 1 / M ) O(\log T/\sqrt{T} + 1/\sqrt{M}) O ( log T / T + 1/ M ) and outperforms existing methods. FLANDERS detects and blocks extreme model poisoning in federated learning.
problem Resilience against large-scale model poisoning attacks in federated learning.
method FLANDERS treats client updates as matrix-valued time series and identifies outliers using autoregressive forecasting.
result FLANDERS significantly improves robustness in federated learning across various attacks.
Artemis framework improves distributed learning with bidirectional compression and partial participation.
problem Learning in distributed or federated settings with communication constraints and device partial participation.
method Artemis framework using bidirectional compression, memory mechanism, and Polyak-Ruppert averaging.
result Fast rates of convergence (linear up to a threshold) under weak assumptions on stochastic gradients.
A new Federated Learning approach balances personalization and global training.
problem Breaking the curse of data heterogeneity in Federated Learning.
method Splitting variables into global and local parameters, using a simple algorithm.
result The approach allows each client to fit their data perfectly, breaking the curse of data heterogeneity.
New algorithm reduces communication time in federated learning.
problem Intermittent connectivity and non-i.i.d. data slow federated learning convergence.
method Lyapunov optimization for efficient device scheduling.
result Significant reduction in communication time with improved convergence rates.
FastSecAgg improves federated learning security and efficiency.
problem Privacy leakage in federated learning due to model parameter sharing.
method Introduces FastSecAgg, a secure aggregation protocol with FFT-based multi-secret sharing (FastShare).
result Efficient in computation and communication, robust to client dropouts.
SA-PEF improves federated learning efficiency by correcting gradient mismatches.
problem Slow decay of residual error in federated learning under non-IID data.
method Integrates step-ahead correction with partial error feedback.
result Achieves faster convergence and target accuracy compared to standard EF.
FedSmart optimizes federated learning models for non-IID data.
problem Model performance on non-IID data is poor and privacy is at risk.
method FedSmart optimizes models by sharing global gradients and adjusting weights based on local validation set accuracy.
result FedSmart improves model performance by allocating more weight to similar data distributions.
A two-stage optimization framework reduces label noise in federated learning.
problem Label noise from noisy clients degrades federated learning model performance.
method MaskedOptim framework: detects noisy clients, corrects labels, and aggregates models robustly.
result Our framework improves model robustness and data quality in federated learning.
Federated Learning with L0 constraint improves sparsity and performance.
problem Inherent sparsity in data and models leads to dense models with poor generalizability.
method L0 constraint on model density achieved through probabilistic gates and federated stochastic gradient descent.
result Achieves target sparsity (rho) in FL with minimal loss in statistical performance.
AAggFF improves federated learning fairness through sequential decision making.
problem Achieving client-level fairness in federated learning systems.
method Unified online convex optimization framework for adaptive aggregation strategies.
result AAggFF achieves better client-level fairness in federated learning.
This paper tackles federated incremental learning with dynamic memory allocation for improved model performance in non-IID data.
problem Catastrophic forgetting in federated healthcare systems with non-IID data.
method Dynamic memory allocation strategy based on data replay mechanism.
result Significant performance improvements in medical image datasets compared to baseline models.