Study finds non-IID data causes FL performance issues.
problem Reduced performance in federated learning due to non-IID data.
method Investigated from IID to non-IID settings, categorized methods into two strategies.
result Inconsistencies in client loss landscapes are the primary cause of performance degradation.
LotteryFL improves federated learning efficiency and personalization.
problem Statistical heterogeneity and communication efficiency in federated learning.
method Exploiting Lottery Ticket hypothesis to learn compact lottery networks.
result LotteryFL achieves better personalization and reduces communication cost.
FedCD improves non-IID federated learning performance.
problem Learning from non-IID data in federated learning.
method FedCD dynamically groups devices with similar data.
result FedCD achieves higher accuracy and faster convergence.
A new algorithm improves federated learning by combining knowledge distillation and weighted combination loss.
problem Non-IID client data in federated learning leads to model drift and poor generalization.
method pFedKD-WCL integrates knowledge distillation with bi-level optimization to address non-IID challenges.
result pFedKD-WCL outperforms state-of-the-art algorithms in accuracy and convergence speed.
FedFMC improves federated learning on non-iid data without sharing data or increasing communication costs.
problem Efficiently updating a global model on non-iid data in federated learning.
method FedFMC dynamically forks devices into different global models, merges and consolidates them.
result FedFMC substantially improves upon earlier approaches to non-iid data in federated learning.
A framework for large-scale federated learning with non-IID data.
problem Stability of models trained on non-IID data in federated learning.
method Generation of non-IID datasets and modular evaluation framework.
result Open-source benchmark for large-scale federated learning research.
Clust-PSI-PFL uses PSI to improve accuracy and fairness in federated learning.
problem Non-IID data biases federated learning performance.
method Clust-PSI-PFL uses clustering and PSI to form homogeneous groups of clients.
result Clust-PSI-PFL delivers up to 18% higher global accuracy and improves client fairness.
Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on …
This study assesses the impact of non-IID data in federated learning, revealing significant performance drops.
problem The impact of non-IID data on federated learning model performance.
method Empirical analysis using Hellinger Distance to measure distribution differences, benchmarking four strategies for handling non-IID data.
result Significant performance drops occur at specific HD thresholds, especially for extreme non-IIDness.
This work improves federated learning privacy and accuracy with non-private data sharing and approximate gradient coding.
problem Challenges of non-IID data and stragglers in federated learning.
method Data-driven strategy combining offline data sharing and approximate gradient coding.
result Achieves a trade-off between privacy and utility, leading to improved model convergence and accuracy.
Paper proposes CCVR to improve classifier calibration in federated learning with non-IID data.
problem Improving classifier calibration in federated learning with non-IID data.
method Proposes CCVR, a simple algorithm that adjusts the classifier using virtual representations.
result CCVR achieves state-of-the-art performance on popular federated learning benchmarks.
Fed-TDA augments federated tabular data to improve performance and privacy.
problem Non-IID data challenges in federated learning.
method Synthesizes tabular data using simple statistics for augmentation.
result Fed-TDA improves test performance and communication efficiency.
DPMM-CFL clusters clients for federated learning without fixed K, improving performance.
problem Improving federated learning performance under non-IID client heterogeneity.
method DPMM-CFL uses a Dirichlet Process Mixture Model to infer both cluster number and client assignments.
result DPMM-CFL optimizes per-cluster federated objectives and jointly infers cluster number and assignments.
Federated multi-mini-batch improves efficiency in non-IID environments.
problem Performance and communication efficiency challenges in federated learning with non-IID data.
method Introduces federated multi-mini-batch approach to balance performance and communication.
result Federated multi-mini-batch outperforms federated averaging in non-IID settings.
Quantum federated learning improves with non-IID data using one-shot communication.
problem Performance degradation in federated learning with non-IID data.
method Quantum algorithms and local density estimators for non-IID data.
result One-shot communication complexity for non-IID quantum federated learning.
On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples. To enjoy this benefit, inter-device communication overhead should be minimized. With this end, we propose federated distillation (FD), a distributed model training algorithm whose communicati…
Client adaptation improves federated learning performance with non-IID data.
problem Improving model performance in federated learning with non-identically and non-independently distributed data.
method Simulates heterogeneous clients to learn client-specific conditioning using a conditional gated activation unit.
result Client adaptation enhances model performance across balanced and imbalanced data sets from audio and image domains.
Paper proposes FedAMP for improved federated learning with non-IID data.
problem Challenges of federated learning with non-IID data.
method FedAMP: Federated attentive message passing for pairwise client collaborations.
result FedAMP improves federated learning performance with non-IID data.
FL+HC improves federated learning on non-iid data by clustering local updates.
problem FL struggles with non-iid data, leading to suboptimal models.
method Introduce hierarchical clustering to separate and train clusters of clients independently.
result FL+HC converges faster and achieves higher accuracy than standard FL.
A framework for federated graph classification over non-IID graphs.
problem Training graph mining models collaboratively across different domains with non-IID graphs.
method Graph Clusters Federated Learning (GCFL) framework, dynamically finding clusters based on GNN gradients, and a gradient sequence-based clustering mechanism (GCFL+).
result Demonstrated effectiveness of GCFL+ in reducing structure and feature heterogeneity among graphs.
Identifying anomalous patterns in real-world data is essential for understanding where, when, and how systems deviate from their expected dynamics. Yet methods that separately consider the anomalousness of each individual data point have low detection power for subtle, emerging irregularities. Additionally, recent dete…
This paper tackles federated incremental learning with dynamic memory allocation for improved model performance in non-IID data.
problem Catastrophic forgetting in federated healthcare systems with non-IID data.
method Dynamic memory allocation strategy based on data replay mechanism.
result Significant performance improvements in medical image datasets compared to baseline models.
New bucketing scheme improves Byzantine robustness for heterogeneous data.
problem Byzantine attacks on federated learning with heterogeneous data.
method Bucketing scheme to adapt robust algorithms to non-iid data.
result Bucketing scheme ensures convergence against Byzantine attacks.
New framework detects outliers in non-IID categorical data.
problem Existing outlier detection methods fail in non-IID data.
method Value-value graph-based representation and outlierness propagation.
result Significant improvement in AUC on complex data sets.
IDA adapts to non-iid data in federated learning for medical imaging.
problem Statistical heterogeneity in federated learning data, especially in medical imaging.
method IDA (Inverse Distance Aggregation) is a novel adaptive weighting approach for clients based on meta-information.
result IDA outperforms Federated Averaging in handling unbalanced and non-iid data in federated learning.
New federated learning methods improve model performance on non-IID data.
problem Improving model performance on non-IID decentralized data.
method Proposed Federated AGMs using adaptive gradient methods with first-order and second-order momenta.
result The proposed Federated AGMs converge to a first-order stationary point under non-IID and unbalanced data settings for nonconvex optimization.
This paper proposes a cooperative mechanism for mitigating the performance degradation due to non-independent-and-identically-distributed (non-IID) data in collaborative machine learning (ML), namely federated learning (FL), which trains an ML model using the rich data and computational resources of mobile clients with…
A new federated learning method clusters users into multiple models for better data distribution handling.
problem Non-IID data from heterogeneous sources in federated learning.
method Proposes a multi-center aggregation mechanism to learn multiple global models and optimally match users to centers.
result Our method outperforms existing federated learning methods on benchmark datasets.
FedSmart optimizes federated learning models for non-IID data.
problem Model performance on non-IID data is poor and privacy is at risk.
method FedSmart optimizes models by sharing global gradients and adjusting weights based on local validation set accuracy.
result FedSmart improves model performance by allocating more weight to similar data distributions.
Study proposes BFEL framework for privacy-preserving FL in personalized healthcare.
problem Privacy and security concerns in traditional cloud-centric ML, especially in wearable devices.
method Develops a blockchain-enhanced federated edge learning (BFEL) framework based on FedCurv, incorporating fisher information matrix and public key encryption.
result Significant reduction in communication cost and high efficiency for federated training on non-iid and heterogeneous data.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
FedACS uses attention to select clients with similar data for federated learning.
problem Non-IID data and data scarcity in federated learning.
method FedACS integrates an attention mechanism to prioritize clients with similar data distributions.
result FedACS improves federated learning performance by addressing non-IID data and data scarcity.
FedMAX tackles activation divergence in FL, improving accuracy and efficiency.
problem Activation divergence in Federated Learning due to non-IID data.
method Introduced a prior based on maximum entropy to minimize information about per-device activation vectors and make similar classes more alike.
result Significantly more similar activation vectors across multiple devices, leading to better accuracy and efficiency.
GraphFL tackles semi-supervised node classification on graphs using federated learning.
problem Real-world graph-based problems often require collecting the entire graph and labeling a reasonable number of labels, which is impractical and costly.
method GraphFL is a federated learning framework that addresses non-IID data, new label domains, and unlabeled data issues in graph-based semi-supervised node classification.
result GraphFL significantly outperforms compared FL baselines and self-training methods.
Semi-Federated Learning clusters clients for efficient model training.
problem Efficiency and data distribution challenges in Federated Learning.
method Local clustering and in-cluster training with a sequential training manner.
result Semi-Federated Learning reduces communication costs and improves robustness to Non-IID data.
New learnability criteria for non-iid processes equivalent to online learning.
problem Statistical learning under non-iid stochastic processes is underdeveloped.
method Defined two learnability notions and showed their equivalence to online learning.
result Learnability criteria for non-iid processes are equivalent to online learning.
New methods improve distributed optimization on non-iid data.
problem Communication bottleneck in distributed machine learning models.
method Two types of distributed gradient compression methods (D-QSGD and D-EF-SGD) analyzed for non-iid data.
result D-EF-SGD performs better than D-QSGD on non-iid data but can still slow down with high data skewness.
A federated method for feature selection in multi-label data.
problem Feature selection in multi-label data for distributed and federated environments.
method Semi-Supervised Federated Multi-Label Feature Selection (SSFMLFS) using fuzzy information measures.
result SSFMLFS outperforms other methods in feature selection for multi-label data in federated settings.
Survey classifies Clustered Federated Learning into three types of approaches.
problem Non-independent and identically distributed (non-IID) data in Federated Learning.
method Systematic review of CFL literature, principled taxonomy.
result Core CFL and Metadata-based approaches have distinct focuses.
New research shows semantic data matching can degrade SSDL performance.
problem The limits of semantic data set matching in semi-supervised learning.
method Demonstrated through simulations and a new dissimilarity measure.
result Semantic data matching can degrade SSDL performance under non-IID data.
Unified Bayesian framework for clustered federated learning improves model performance.
problem Handling non-IID client data in federated learning.
method A unified Bayesian framework for clustered federated learning that associates clients to clusters and proposes practical algorithms for data associations.
result The proposed framework increases model performance by circumventing the need for unique client-cluster associations.
A Bayesian treatment of latent directed graph structure for non-iid data is provided where each child datum is sampled with a directed conditional dependence on a single unknown parent datum. The latent graph structure is assumed to lie in the family of directed out-tree graphs which leads to efficient Bayesian inferen…
Paper addresses generalization error bounds for learning with censored feedback.
problem Impact of censored feedback on generalization error bounds.
method Derives an extension of DKW inequality for non-IID data due to censored feedback and uses it to bound generalization error.
result Existing generalization error bounds fail to account for censored feedback, necessitating new bounds.
This work tackles sequential data learning challenges by improving neural network robustness to non-iid distribution shifts.
problem Sequential data learning challenges, particularly non-iid distribution shifts across batches.
method Cramér-Rao-based regularization using Fisher Information Matrix to adapt to sequential covariate shifts.
result Achieves 19% accuracy improvement over state-of-the-art methods.
We tackle the problem of Federated Learning in the non i.i.d. case, in which local models drift apart, inhibiting learning. Building on an analogy with Lifelong Learning, we adapt a solution for catastrophic forgetting to Federated Learning. We add a penalty term to the loss function, compelling all local models to con…
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\texttt{FedAvg}) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once …
This work addresses a new problem that learns generative adversarial networks (GANs) from multiple data collections that are each i) owned separately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such non-iid data as input, we aim to learn a distribution in…
SA-PEF improves federated learning efficiency by correcting gradient mismatches.
problem Slow decay of residual error in federated learning under non-IID data.
method Integrates step-ahead correction with partial error feedback.
result Achieves faster convergence and target accuracy compared to standard EF.