This paper develops a federated EM algorithm for unsupervised learning of mixture models.
problem Theoretical foundations of unsupervised federated learning are lacking.
method Introduces a federated gradient EM algorithm (FedGrEM) for unsupervised learning of mixture models.
result Theoretical analysis shows FedGrEM outperforms local single-task learning.
Federated MTL learns personalized models under mixed distributions.
problem Heterogeneity of local data distributions leads to poor global model performance.
method Proposes federated MTL under mixture of distributions, using penalized optimization and federated EM-like algorithms.
result Models with higher accuracy and fairness than state-of-the-art methods.
Bayesian mixture models inference in federated learning
problem Inference of Bayesian mixture models in federated learning
method Consensus Monte Carlo approach
result Recovery of small clusters with greater accuracy than standard MCMC
EM algorithm speeds up convergence in federated learning with heterogenous data.
problem Understanding convergence rates of federated learning algorithms under data heterogeneity.
method Characterized convergence rate of EM algorithm for FMLR model under various regimes.
result EM algorithm converges to ground truth with SNR ≥ √K in all regimes.
A new method speeds up DPMM inference for federated learning.
problem Slow inference for large datasets in DPMMs.
method Distributed collapsed Gibbs sampler (DisCGS) for DPMMs.
result Significant reduction in execution time (200x faster) for large datasets.
New algorithm reduces communication in federated learning.
problem Communication inefficiency in distributed federated learning.
method Distributionally Robust Federated Averaging (DRFA) with snapshotting scheme.
result DRFA achieves efficient convergence in various settings.
Wasserstein framework solves mixed linear regression problems.
problem Mixed linear regression with multi-modal distributions.
method Wasserstein distance minimization for nonconvex-concave minimax optimization.
result WMLR achieves global convergence and generalization guarantees for two linear models.
FDR-SVM improves classification robustness in federated learning with uncertain data.
problem Federated learning with uncertain and private client data.
method Develops FDR-SVM, a robust SVM approach using a mixture of Wasserstein balls ambiguity set.
result Establishes theoretical guarantees and derives algorithms with performance bounds.
New algorithms improve Gaussian mixture model estimation in DFL with heterogeneous data.
problem Bias in EM algorithm for Gaussian mixtures in DFL with heterogeneous data.
method MNEM and semi-MNEM algorithms integrating historical and partially labeled data.
result MNEM and semi-MNEM achieve asymptotic efficiency and improved convergence.
Efficient federated algorithm for calculating transportation barycenter.
problem Efficiently calculating the free-support transportation barycenter in a federated setting.
method Single-loop dual decomposition algorithm that uses only aggregated information.
result Significantly scalable and low-complexity algorithm for federated computation.
Adaptive personalized federated learning improves local model personalization.
problem Maximizing global model performance limits local model personalization.
method APFL algorithm trains local models while contributing to global model, with optimal mixing parameter and communication-efficient optimization.
result Demonstrates effectiveness of personalization schema and correctness of generalization theories.
DPMM-CFL clusters clients for federated learning without fixed K, improving performance.
problem Improving federated learning performance under non-IID client heterogeneity.
method DPMM-CFL uses a Dirichlet Process Mixture Model to infer both cluster number and client assignments.
result DPMM-CFL optimizes per-cluster federated objectives and jointly infers cluster number and assignments.
Federated learning for Bayesian clustering of large datasets.
problem Bayesian model-based clustering of large-scale binary and categorical data.
method Federated variational inference with local merge and delete moves in parallel batches, followed by global merge moves.
result Empirical validation shows superior performance compared to existing algorithms.
New optimization for federated learning with local models.
problem Training models with private data from multiple devices.
method Proposes a new optimization formulation and efficient SGD variants.
result Local steps can improve communication for heterogeneous data.
A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new f…
SPIRE enables efficient federated learning for diffusion models by separating client-specific embeddings from a shared backbone.
problem Large diffusion models are impractical for federated learning due to their size.
method SPIRE separates the network into a global backbone and client-specific embeddings, enabling efficient finetuning.
result SPIRE achieves parameter-efficient finetuning, updating only a small fraction of weights.
Study federates measurement of demographic disparities from quantile sketches.
problem Misalignment of fairness goals with siloed data collection and privacy regulations.
method Federated auditing of demographic parity through score distributions, using Wasserstein--Frechet variance and quantile summaries.
result Proposes a one-shot, communication-efficient protocol to estimate global disparity and its decomposition.
The paper improves EM for clustering in a mixture of linear regression models.
problem Improving EM for clustering in a mixture of linear regression models with clustered data.
method Employed Expectation-Maximization (EM) method to estimate parameters from dependent samples.
result EM on structured data requires only O(1) iterations for statistical accuracy, compared to O(log(mn/d)) for unstructured data.
In clinical research, the lack of events of interest often necessitates imbalanced learning. One approach to resolve this obstacle is data integration or sharing, but due to privacy concerns neither is practical. Therefore, there is an increasing demand for a platform on which an analysis can be performed in a federate…
Paper proposes CCVR to improve classifier calibration in federated learning with non-IID data.
problem Improving classifier calibration in federated learning with non-IID data.
method Proposes CCVR, a simple algorithm that adjusts the classifier using virtual representations.
result CCVR achieves state-of-the-art performance on popular federated learning benchmarks.
GRAFFL learns Bayesian models without gradients for federated learning.
problem Privacy protection in federated learning during gradient updates.
method Gradient-free approach using summary statistics from SuffiAE.
result Feasibility and usefulness demonstrated in Gaussian mixture model.
Improved Bayesian FL method calibrates predictions for federated learning.
problem Overconfident predictions in Bayesian FL methods for federated learning.
method β-Predictive Bayes algorithm interpolates between mixture and product of local predictive posteriors, tuning parameter β for better calibration.
result Demonstrated superior calibration compared to other baselines, even with increased data heterogeneity.
This paper studies generalization in machine learning with mixture data.
problem Generalization performance and statistical rates in heterogeneous data.
method Characterization of heterogeneity via pairwise total variation distance, analysis of Rademacher and Gaussian complexities.
result The requirement on heterogeneity increases as function classes get more complex.
New method estimates SW distance using CDFs for scalable data parallelism.
problem Estimating SW distance efficiently for large datasets.
method Estimators based on CDFs of projected measures, avoiding sorting.
result Efficient estimation for large datasets and federated learning.
Compressed Federated Distillation reduces communication in federated learning.
problem Communication constraints in Federated Learning.
method Compressed Federated Distillation (CFD) leverages soft labels and quantization techniques.
result Reduces communication by more than 4 orders of magnitude compared to Federated Averaging.
Adaptive optimizers improve federated learning performance.
problem Inadequate optimization methods in federated learning.
method Federated versions of adaptive optimizers (Adagrad, Adam, Yogi).
result Adaptive optimizers enhance federated learning performance.
Federated multi-mini-batch improves efficiency in non-IID environments.
problem Performance and communication efficiency challenges in federated learning with non-IID data.
method Introduces federated multi-mini-batch approach to balance performance and communication.
result Federated multi-mini-batch outperforms federated averaging in non-IID settings.
FedIV uses federated GMM for IV analysis in non-i.i.d. data.
problem Efficient IV analysis in non-i.i.d. federated data.
method Federated GMM via FedGDA algorithm.
result Federated solution consistently estimates local moment conditions.
A practical one-shot federated learning algorithm for cross-silo setting.
problem Limited applicability of existing one-shot federated learning algorithms due to specific model support and lack of privacy guarantees.
method FedKT, a one-shot federated learning algorithm that supports any classification models and provides differential privacy guarantees.
result FedKT significantly outperforms other state-of-the-art federated learning algorithms with a single communication round.
Paper introduces OARF benchmark suite for federated learning systems.
problem Limited diversity in federated learning benchmarks.
method Characterizes OARF benchmark suite with diverse data and applications.
result Federated learning can effectively increase end-to-end throughput.
New federated f-differential privacy for collaborative learning.
problem Privacy in federated learning.
method Introducing federated f-differential privacy and proposing a generic private federated learning framework. result Proves federated f-differential privacy provides privacy guarantee on each record of one client's data. New algorithm solves federated minimax optimization problems.
problem Federated minimax optimization challenges.
method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).
result FESS-GDA uniformly solves federated minimax problems.
A framework to compare federated learning algorithms in high-dimensional settings.
problem Comparing the performance of federated learning algorithms in high-dimensional settings.
method Formulating federated learning as a multi-criterion objective and analyzing a linear regression model.
result Federated Averaging with simple client fine-tuning achieves the same asymptotic risk as more intricate approaches and outperforms without personalization.
Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing s…
pFedGame uses game theory for decentralized federated learning in dynamic networks.
problem Performance bottlenecks, data bias, model convergence issues, and model poisoning attacks in federated learning.
method pFedGame employs game theory to decentralize federated learning, avoiding a central aggregation server and addressing dynamic network challenges.
result pFedGame achieves higher accuracy (over 70%) in heterogeneous data compared to existing methods.
This paper analyzes privacy threats in federated matrix factorization.
problem Privacy threats in federated matrix factorization models.
method Categorizes federated matrix factorization into three types and analyzes privacy threats.
result This is the first study of privacy threats in federated matrix factorization.
Research improves federated text models for next word prediction.
problem Training models on distributed devices efficiently and effectively.
method Employ transfer learning in federated learning for next word prediction.
result Enhancements to current baselines with pretrained embeddings and whole model pretraining.
Federated learning method improves covariate shift adaptation for missing target values.
problem Missing target values in federated learning.
method Federated covariate shift adaptation algorithm for missing target output values.
result Asymptotically unbiased and efficient algorithm for federated learning.
In federated learning, a central server coordinates the training of a single model on a massively distributed network of devices. This setting can be naturally extended to a multi-task learning framework, to handle real-world federated datasets that typically show strong statistical heterogeneity among devices. Despite…
This work improves fairness in federated learning by using zero-shot data augmentation.
problem Statistical heterogeneity leads to biased and less uniform accuracy across clients in federated learning.
method Proposes a federated learning system with zero-shot data augmentation to mitigate statistical heterogeneity and improve fairness.
result Empirical results show improved test accuracy and fairness across clients.
A fair reward system boosts participation in federated learning.
problem Fairness in federated learning among competitive agents with siloed data.
method Hierarchically fair federated learning (HFFL) framework with proportional rewards based on contribution levels.
result Efficacy of HFFL in maintaining fairness and facilitating federated learning in competitive settings.
Federated learning studies separate client data and distribution gaps.
problem Understanding performance differences in federated learning across different datasets.
method Proposed a framework to disentangle out-of-sample and participation gaps.
result Dataset synthesis strategy is crucial for realistic simulations of federated learning generalization.
Federated multi-view matrix factorization learns from multiple data sources without centralizing user data.
problem Cold-start federated recommendations and multi-view data structure.
method Federated learning framework extended to multi-view matrix factorization.
result Federated multi-view matrix factorization outperforms simpler methods in cold-start federated recommendations.
Federated learning linked to mean-field games for large-scale learning.
problem Large-scale distributed and privacy-preserving learning algorithms.
method Established a connection between federated learning and mean-field games, presenting federated learning as a differential game.
result Properties of the equilibrium of the federated learning game were discussed.
A new federated learning method reduces communication costs and improves adaptivity.
problem Large communication overhead and lack of adaptivity in federated learning.
method FedCAMS: A novel communication-efficient adaptive federated learning method with theoretical guarantees.
result FedCAMS achieves the same convergence rate as non-compressed federated learning methods.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
FedSPDnet improves federated learning for SPD matrices, outperforming existing methods.
problem Federated learning for SPD matrices with orthogonality constraints.
method Two efficient aggregation strategies: ProjAvg and RLAvg, preserving geometric structure.
result FedSPDnet outperforms federated EEGnet in F1 score and robustness to federation and partial participation.
Novel algorithm resists Byzantine attacks in federated learning for PCA and LRCS.
problem Byzantine attacks in federated learning for PCA and LRCS.
method Subspace-Median algorithm for federated PCA, altGDmin for LRCS.
result Provably Byzantine-resilient communication-efficient and sample-efficient algorithms.