New algorithm for federated learning with non-smooth regularizers.
problem Federated Learning with non-smooth composite optimization problems.
method Proposed Federated Dual Averaging (FedDualAvg) algorithm to overcome convergence issues.
result FedDualAvg outperforms other algorithms in federated composite optimization.
Partial model averaging improves Federated Learning performance.
problem Periodic model averaging causes significant model discrepancy in Federated Learning.
method Proposes a partial model averaging framework that encourages local models to stay close to each other.
result Partial averaging achieves up to 2.2% higher validation accuracy than full averaging.
Federated learning technique improves convergence speed with communication delays.
problem Communication delays between edge nodes and aggregator in federated learning.
method Developed FedDelAvg, a technique that generalizes federated averaging to incorporate a weighting between current local model and delayed global model.
result FedDelAvg achieves a significant improvement in convergence speed, especially when optimizing the weighting scheme to account for delays.
This paper analyzes the convergence of Federated Average under relaxed assumptions.
problem Lack of theoretical analysis for Federated Average under assumptions beyond smoothness.
method Relaxing assumptions of strong smoothness to semi-smoothness and semi-Lipschitz properties, and introducing a bound on the gradient.
result Provides a theoretical convergence study on Federated Learning under new assumptions.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
This dissertation advances the theoretical foundation of local optimization methods in Federated Learning.
problem Theoretical understanding of local optimization methods in Federated Learning is lacking.
method The dissertation proposes and analyzes new methods to improve convergence rates and communication efficiency in Federated Learning.
result Sharp bounds and convergence rates for FedAvg are established, and new methods like FedAc and Federated Dual Averaging are proposed.
Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural ne…
FedSPDnet improves federated learning for SPD matrices, outperforming existing methods.
problem Federated learning for SPD matrices with orthogonality constraints.
method Two efficient aggregation strategies: ProjAvg and RLAvg, preserving geometric structure.
result FedSPDnet outperforms federated EEGnet in F1 score and robustness to federation and partial participation.
We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mi…
This work improves Q-learning for average-reward MDPs, reducing sample and communication complexities in federated settings.
problem Improving sample complexity of Q-learning for average-reward MDPs.
method Simple Q-learning algorithm with carefully chosen parameters for both single-agent and federated scenarios.
result Established first federated Q-learning algorithm for average-reward MDPs with provable efficiency in sample and communication complexities.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
Compressed Federated Distillation reduces communication in federated learning.
problem Communication constraints in Federated Learning.
method Compressed Federated Distillation (CFD) leverages soft labels and quantization techniques.
result Reduces communication by more than 4 orders of magnitude compared to Federated Averaging.
Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-oriented challenges which include (i) communication bottleneck since a large number of devices upload their local updates to a parameter server, a…
A framework to compare federated learning algorithms in high-dimensional settings.
problem Comparing the performance of federated learning algorithms in high-dimensional settings.
method Formulating federated learning as a multi-criterion objective and analyzing a linear regression model.
result Federated Averaging with simple client fine-tuning achieves the same asymptotic risk as more intricate approaches and outperforms without personalization.
FedAvg converges linearly to global minimum in federated learning with partial participation.
problem Challenges in federated learning with partial client participation.
method Federated averaging (FedAvg) method for over-parameterized neural networks.
result FedAvg converges to global minimum at a linear rate after t iterations.
New algorithm reduces communication in federated learning.
problem Communication inefficiency in distributed federated learning.
method Distributionally Robust Federated Averaging (DRFA) with snapshotting scheme.
result DRFA achieves efficient convergence in various settings.
Sharp bounds established for Federated Averaging (FedAvg), improving convergence rates.
problem Undetermined convergence rate of Federated Averaging (FedAvg) in Federated Learning.
method Developed novel iterate bias concept and proved sharp bounds on it, leading to improved convergence results.
result Lower bounds for FedAvg match existing upper bounds, showing no improvable capacity.
Unified analysis of Federated Averaging and Nesterov FedAvg for linear speedup.
problem Understanding convergence of FL algorithms under non-i.i.d. data and partial participation.
method Systematic study of convergence guarantees for FedAvg and Nesterov FedAvg under different conditions.
result Unified analysis of linear speedup for FedAvg and Nesterov FedAvg in various settings.
FA-HMC improves Bayesian federated learning with rigorous guarantees.
problem Parameter estimation and uncertainty quantification in non-iid distributed data.
method Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC) with convergence guarantees.
result FA-HMC achieves better convergence and communication efficiency than existing methods.
A federated learning framework using superquantile aggregation for robust performance across heterogeneous data.
problem Robust predictive performance across clients with heterogeneous data.
method Superquantile-based learning objective and stochastic training algorithm with differential privacy.
result Proves finite time convergence guarantees and demonstrates competitive performance with tail statistics improvement.
Mobile crowdsensing has gained significant attention in recent years and has become a critical paradigm for emerging Internet of Things applications. The sensing devices continuously generate a significant quantity of data, which provide tremendous opportunities to develop innovative intelligent applications. To utiliz…
Federated learning enables training collaborative machine learning models at scale with many participants whilst preserving the privacy of their datasets. Standard federated learning techniques are vulnerable to Byzantine failures, biased local datasets, and poisoning attacks. In this paper we introduce Adaptive Federa…
FedAc accelerates Federated Averaging for distributed optimization.
problem Efficiently optimizing distributed machine learning models.
method Federated Accelerated Stochastic Gradient Descent (FedAc) using a potential-based perturbed iterate analysis.
result FedAc achieves faster convergence and lower communication costs than previous methods.
Paper analyzes \FedAvg's convergence and introduces a new algorithm to reduce bias.
problem Analyzing convergence and bias in Federated Averaging.
method Markov property, first-order bias expansion, Richardson-Romberg extrapolation.
result Bias in \FedAvg can be decomposed into noise and client heterogeneity components.
New federated learning algorithms improve model aggregation robustness.
problem Improving model aggregation in federated learning.
method Complete mathematical convergence analysis and novel aggregation algorithms.
result Derived novel algorithms that modify model architecture based on client contributions.
Federated multi-mini-batch improves efficiency in non-IID environments.
problem Performance and communication efficiency challenges in federated learning with non-IID data.
method Introduces federated multi-mini-batch approach to balance performance and communication.
result Federated multi-mini-batch outperforms federated averaging in non-IID settings.
A new method for federated learning with only positive labels.
problem Learning with only positive labels leads to poor classifier performance.
method Federated Averaging with Spreadout (FedAwS) framework.
result FedAwS can almost match the performance of conventional learning with negative labels.
Federated Q-learning achieves linear speedup with heterogeneity, improving sample complexity.
problem Collaborative learning in distributed RL settings with limited data sharing.
method Analyzes synchronous and asynchronous federated Q-learning, proposes importance averaging.
result Achieves linear speedup with heterogeneity, robust to local trajectory heterogeneity.
FedHDPrivacy uses DP to improve FL in IoT, maintaining high accuracy.
problem Privacy threats in FL, especially in IoT environments.
method Integrates DP with neuro-symbolic computing, actively monitoring and adjusting noise.
result Maintains high performance in manufacturing monitoring, surpassing other FL methods.
FA-LD algorithm improves uncertainty quantification and mean predictions in federated learning.
problem Uncertainty quantification and mean predictions in federated learning with distributed clients.
method FA-LD algorithm for strongly log-concave distributions with non-i.i.d data, considering general models.
result The FA-LD algorithm provides theoretical guarantees for convergence and optimal noise injection.
New federated method preserves privacy and estimates treatment effects.
problem Privacy-preserving causal inference for multi-site studies.
method Multiply robust nuisance function estimation, transfer learning.
result Efficient and optimal treatment effect estimation under different scenarios.
DFedAvgM is a decentralized FedAvg with momentum for privacy and communication efficiency.
problem Efficiently train models with privacy and communication efficiency in federated learning.
method Decentralized Federated Averaging with Momentum (DFedAvgM) on clients connected by an undirected graph, using stochastic gradient descent with momentum and quantization.
result DFedAvgM converges under trivial assumptions and can be improved with the PŁ property, numerically verified.
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data. Standard federated optimization methods such as Federated Averaging (FedAvg) are often difficult to tune and exhibit unfavorable con…
A scalable protocol for federated averaging with privacy and correctness guarantees.
problem Privacy and correctness in federated learning from multiple parties.
method Scalable protocol using correlated and independent Gaussian noise, analyzed for differential privacy and graph topology.
result Nearly matches trusted curator model's utility with minimal communication.
We present one-shot federated learning, where a central server learns a global model over a network of federated devices in a single round of communication. Our approach - drawing on ensemble learning and knowledge aggregation - achieves an average relative gain of 51.5% in AUC over local baselines and comes within 90.…
FGPR uses averaging and SGD for federated GP regression, excelling in personalization and multi-fidelity modeling.
problem Privacy-preserving multi-fidelity data modeling and personalization.
method Federated Gaussian process framework with averaging and SGD for local computations.
result FGPR converges to a critical point of the full log-likelihood function, excels in personalization and multi-fidelity modeling.
This paper advances FL algorithms for composite optimization and statistical recovery.
problem Federated learning optimization and statistical recovery in composite settings.
method Proposes Fast Federated Dual Averaging for strongly convex and smooth loss, and Multi-stage Federated Dual Averaging for restricted strongly convex and smooth loss.
result Establishes state-of-the-art iteration and communication complexity, and high probability complexity bound with linear speedup.
FedCluster accelerates federated learning convergence by cycling device groups.
problem Federated learning convergence issues with device-level data heterogeneity.
method FedCluster groups devices into clusters that cycle through learning rounds, boosting convergence with meta-updates.
result FedCluster achieves faster convergence in nonconvex optimization compared to FedAvg.
This work tackles semi-supervised federated learning by reducing model gradient diversity.
problem Improving test accuracy in semi-supervised federated learning with limited labeled data.
method Investigates and compares various design choices including consistency regularization loss, Batch Normalization, and Group Normalization.
result Grouping-based model averaging combined with Group Normalization and consistency regularization loss improves test accuracy.
FedNNNN improves FL by adjusting model update vector norms.
problem Slow convergence and low prediction accuracy in FL.
method Norm-Normalized Neural Network Aggregation (FedNNNN) with momentum control.
result Up to 5.4% accuracy improvement on multiple datasets.
Federated Learning allows multiple parties to jointly train a deep learning model on their combined data, without any of the participants having to reveal their local data to a centralized server. This form of privacy-preserving collaborative learning however comes at the cost of a significant communication overhead du…
Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' w…
Fed-ensemble improves FL by averaging predictions from multiple models.
problem Improving generalization in federated learning.
method Random permutations to update K models, averaging predictions.
result Predictions from all K models have the same predictive posterior distribution.
This study optimizes model averaging for personalized collaborative learning.
problem Differences in data or objectives between nodes in federated learning.
method Weighted averaging between local and global models for scalar mean estimation.
result There is always some positive model averaging that reduces expected squared error.
In federated distributed learning, the goal is to optimize a global training objective defined over distributed devices, where the data shard at each device is sampled from a possibly different distribution (a.k.a., heterogeneous or non i.i.d. data samples). In this paper, we generalize the local stochastic and full gr…
This paper adapts PATE for semantic segmentation while maintaining privacy.
problem Preserving privacy in medical machine learning, especially for sensitive information.
method Adapting PATE for semantic segmentation using low-dimensional representations and low-sensitivity queries.
result An Autoencoder-based PATE variant achieves a higher Dice coefficient for the same privacy guarantee.
The emerging paradigm of federated learning strives to enable collaborative training of machine learning models on the network edge without centrally aggregating raw data and hence, improving data privacy. This sharply deviates from traditional machine learning and necessitates the design of algorithms robust to variou…
Federated CTMC model estimates bridge deterioration hazards without sharing raw data.
problem Bridge inspection data privacy and cross-organizational data sharing constraints.
method Federated CTMC hazard model with local optimization and FedAvg aggregation.
result Federated model converges on global benchmark parameters without data transfer.