SAFLe solves federated learning's trade-off between non-linearity and scalability.
problem Federated Learning's high communication overhead and performance collapse on non-IID data.
method SAFLe introduces a structured head of bucketed features and sparse, grouped embeddings, mathematically equivalent to a high-dimensional linear regression.
result SAFLe achieves a new state-of-the-art in analytic FL, outperforming linear AFL and multi-round DeepAFL.
Improved neural network predicts spectral functions more accurately than traditional methods.
problem Reconstructing real-time spectral functions from imaginary-time Green's functions is ill-posed and challenging.
method Feature Learning Network (FL-net) for enhanced prediction accuracy.
result FL-net achieves at least 20% improvement over traditional methods like MEM.
FLORAS uses orthogonal sequences for SISO FL, offering both DP and convergence guarantees.
problem Privacy-preserving wireless federated learning in SISO systems.
method Leverages orthogonal sequences to eliminate CSIT requirement and provide DP guarantees.
result FLORAS achieves a smooth tradeoff between convergence rate and DP levels.
Analyzes gaming in federated learning systems and provides design principles.
problem Gaming and cooperation in federated learning systems under partial observability.
method Developed an analytical framework to separate welfare-improving behavior from metric gaming.
result Derived threshold conditions for deterring harmful gaming and preserving cooperation.
Study on theoretical limits of ℓ0 sparse-regression algorithms using Fl RDT.
problem Understanding the performance limits of ℓ0 norm based optimization algorithms in compressed sensing and sparse regression. method Utilized Fully lifted random duality theory (Fl RDT) to analyze the maximum-likelihood (ML) decoding performance.
result Uncovered phase-transition (PT) and descending ℓ0 (dℓ0) curves that separate successful and unsuccessful algorithm performance. Simplifies deep learning scaling analysis without sacrificing accuracy.
problem Interpreting feature learning mechanisms and determining network implicit bias in high-dimensional settings.
method Developed a heuristic approach for predicting data and width scales of feature learning patterns.
result Predictions align with known results and extend to complex architectures.
Paper optimizes FL communication efficiency with stochastic optimization.
problem Intermittent connectivity and non-i.i.d. data in FL.
method Convergence analysis of non-convex loss functions, stochastic optimization for client selection and power allocation.
result Significant reduction in communication time compared to random participation.
Optimal privacy and accuracy in distributed mean estimation with compression.
problem Achieving optimal accuracy under privacy and communication constraints.
method Compression to reduce communication while maintaining privacy and accuracy.
result Achieves optimal error with significantly reduced communication.
In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a cen…
New method lowers spherical perceptron capacity using fully lifted random duality theory.
problem Tackles the negative spherical perceptron capacity, a long-standing open problem.
method Develops fully lifted random duality theory (fl RDT) to characterize capacity.
result Shows remarkable closed-form analytical relations for practical capacity values.
In this paper, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In particular, in the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station …
This work bridges federated learning and contextual bandits, enhancing FL's utility.
problem Limited use of federated learning in contextual bandits despite its potential.
method Proposes FedIGW, a novel federated contextual bandits design that leverages regression-based algorithms and integrates various FL components.
result FedIGW better harnesses FL innovations and provides flexible, modular, and seamless integration of FL elements.
Group personalization improves FL performance in heterogeneous client data.
problem Mitigating client drift in federated learning with heterogeneous data.
method Fine-tuning a global FL model over homogeneous groups of clients, then personalizing each group's model.
result The proposed method achieves superior personalization performance compared to other FL approaches.
This paper analyzes convergence of FL for neural networks using NTK.
problem Theoretical guarantees of FL for neural networks with explicit forms and multi-step updates are unexplored.
method FL-NTK framework for federated learning of ReLU neural networks trained by gradient descent.
result FL-NTK converges to a global-optimal solution at a linear rate with proper learning parameters.
Efficient Bayesian FL method improves predictive accuracy and uncertainty estimates.
problem Federated Learning with model and predictive uncertainty and personalization.
method Second-order optimization approach for Bayesian FL.
result Improved predictive accuracies and uncertainty estimates.
Flower framework simplifies federated learning experiments on edge devices.
problem Realistic implementation of Federated Learning on edge devices is challenging.
method Developed a comprehensive federated learning framework, Flower, supporting large-scale experiments on heterogeneous devices.
result Flower enables federated learning experiments with up to 15M client size using only two high-end GPUs.
Semi-Federated Learning clusters clients for efficient model training.
problem Efficiency and data distribution challenges in Federated Learning.
method Local clustering and in-cluster training with a sequential training manner.
result Semi-Federated Learning reduces communication costs and improves robustness to Non-IID data.
Study quantifies impacts of heterogeneity in FL on smartphone data.
problem Heterogeneity in FL devices causes performance degradation.
method Collected 136k smartphone data, built heterogeneity-aware FL platform, conducted extensive experiments.
result Heterogeneity causes up to 9.2% accuracy drop and 2.32x training time increase.
FL's early training phase significantly impacts final test accuracy.
problem Understanding how early phases affect FL's final test accuracy.
method Generalized Fisher Information Matrix (FedFIM) to FL.
result FL exhibits critical learning periods where small errors can have large impacts.
Flashback Learning balances model stability and plasticity in continual learning.
problem Balancing model stability and plasticity in continual learning.
method Flashback Learning (FL) uses a bidirectional regularization approach to balance stability and plasticity.
result FL improves model accuracy by up to 4.91% in Class-Incremental and 3.51% in Task-Incremental settings.
A new asynchronous method for vertical federated learning improves privacy and efficiency.
problem Solving vertical federated learning in an asynchronous manner with privacy and efficiency.
method A simple FL method that allows clients to run stochastic gradient algorithms asynchronously with a new perturbed local embedding technique.
result The method improves privacy and communication efficiency compared to centralized and synchronous FL methods.
Survey on threats to federated learning models.
problem Vulnerabilities in federated learning protocols.
method Taxonomy of threat models and attacks.
result Important future research directions.
FL+HC improves federated learning on non-iid data by clustering local updates.
problem FL struggles with non-iid data, leading to suboptimal models.
method Introduce hierarchical clustering to separate and train clusters of clients independently.
result FL+HC converges faster and achieves higher accuracy than standard FL.
NAC-FL optimizes model updates in FL systems by adapting compression to network congestion.
problem Federated Learning systems face congestion and delays in data exchanges.
method NAC-FL dynamically adjusts client compression based on network congestion.
result NAC-FL reduces training time and achieves robust performance improvements.
Optimal feature learning strength improves generalization in deep networks.
problem Understanding how feature learning strength affects generalization in practical settings.
method Empirical studies and theoretical analysis of gradient flow dynamics in two-layer ReLU nets.
result Optimal feature learning strength yields substantial generalization gains, contrary to the prevailing intuition.
Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.
problem Reducing communication overhead in Federated Learning with data heterogeneity and partial participation.
method Revisit and simplify stochastic controlled averaging, proposing SCALLION and SCAFCOM for unbiased and biased compression.
result SCALLION and SCAFCOM outperform existing methods in communication and computation complexities.
FL-Sailer enables federated learning for scATAC-seq data, reducing dimensionality and noise.
problem Privacy-preserving federated learning for ultra-high dimensional, sparse, and heterogeneous scATAC-seq data.
method FL-Sailer integrates adaptive leverage score sampling and an invariant VAE architecture.
result FL-Sailer converges to an approximate solution with bounded error, surpassing centralized methods.
FedML aims to improve FL research by providing a library and benchmark.
problem Inconsistent FL algorithm development and performance comparison.
method FedML offers an open research library and benchmark supporting diverse computing paradigms and flexible API design.
result FedML facilitates fair algorithm comparison and development in federated learning.
Enhances OTA FL algorithms by defining inverse feasibility for linear models.
problem Improving security and privacy in over-the-air federated learning.
method Defines inverse feasibility as an upper bound on condition number, analyzes existing model, proposes new model.
result Proposes a new OTA FL model with enhanced characteristics.
This work analyzes generalization in federated learning using information theory.
problem Generalization performance in federated learning is less explored compared to centralized learning.
method The work applies an information-theoretic analysis via the conditional mutual information (CMI) framework to study federated learning's two-level generalization.
result The work derives multiple CMI-based bounds, including hypothesis-based CMI bounds and fast-rate evaluated CMI bounds, which improve convergence rates for specific model aggregation strategies and structured loss functions.
SIGMA prior enables federated learning for non-factorizable models.
problem Current FL methods assume conditional independence, limiting applicability to non-factorizable models.
method SIGMA prior approximates deep generative model to induce conditional independence structure.
result SIGMA prior expands FL applicability to fields requiring modeling dependencies.
FLeet improves online FL for mobile apps with better performance and privacy.
problem Federated Learning's offline nature limits its applicability for online updates.
method Combines staleness awareness and performance prediction with adaptive learning.
result 2.3x quality boost with minimal battery consumption.
Multi-model FL improves performance without sharing data.
problem Training multiple models in a federated setting without data sharing.
method Proposed two variants of FedAvg for multi-model FL with provable convergence guarantees.
result Multi-model FL can have better performance than training each model separately.
Local adaptation improves federated learning models.
problem Improving accuracy of federated learning models on non-iid data.
method Local adaptation techniques (fine-tuning, multi-task learning, knowledge distillation).
result Participants benefit from local adaptation, improving federated model accuracy.
Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers…
This thesis tackles FL challenges with new methods and algorithms.
problem Privacy-preserving machine learning with decentralized data.
method Compression, client selection, and heterogeneity handling.
result Practical FL solutions with mathematically rigorous guarantees.
LC-FL uses generative models to reduce communication costs in federated learning.
problem High communication costs and strict model homogeneity in federated learning.
method LC-FL employs generative models to transmit data and aggregate models.
result LC-FL reduces communication costs and supports heterogeneous models.
Ubiquitous systems with End-Edge-Cloud architecture are increasingly being used in healthcare applications. Federated Learning (FL) is highly useful for such applications, due to silo effect and privacy preserving. Existing FL approaches generally do not account for disparities in the quality of local data labels. Howe…
This paper studies how label noise affects Federated Learning.
problem The impact of label noise on Federated Learning.
method The paper derives an upper bound for the generalization error and conducts experiments on MNIST and CIFAR-10 datasets.
result The global model accuracy decreases linearly with increasing label noise, consistent with theoretical analysis.
pFedMe uses Moreau envelopes to improve personalized FL performance.
problem Statistical diversity among clients limits FL performance.
method pFedMe employs Moreau envelopes as clients' regularized loss functions in a bi-level optimization problem.
result pFedMe achieves state-of-the-art convergence rates and superior empirical performance.
Random orthogonalization improves FL in massive MIMO systems without CSI.
problem Efficient model aggregation in FL with minimal channel estimation overhead.
method Combining FL with massive MIMO's channel hardening and favorable propagation, random orthogonalization reduces channel estimation overhead.
result Achieves model aggregation without CSI, significantly reducing channel estimation overhead.
The octonionic flag manifold Fl(O) is the space of all pairs in OP2×OP2 (where OP2 denotes the octonionic projective plane) which satisfy a certain "incidence" relation. It comes equipped with the projections π1,π2:Fl(O)→OP2, which are $\mat…
Improved federated learning over-the-air reduces noise impact.
problem Noise in OTA FL degrades model accuracy.
method Developed COTAF algorithm with precoding and scaling.
result COTAF achieves similar convergence rate to error-free channels.
Survey combines FL and control for better adaptability and privacy.
problem Combining FL and control for better adaptability and privacy.
method Combining Federated Learning (FL) and control methods.
result Combining FL and control enhances adaptability, scalability, generalization, and privacy.
New framework provides privacy guarantees for practical federated learning.
problem Inadequate privacy guarantees for federated learning due to restrictive assumptions.
method Fed-α-NormEC, integrating multiple local updates, partial client participation, and standard assumptions. result Provably convergent and differentially private federated learning framework.
Federated Learning (FL) enables learning a shared model across many clients without violating the privacy requirements. One of the key attributes in FL is the heterogeneity that exists in both resource and data due to the differences in computation and communication capacity, as well as the quantity and content of data…
Federated learning (FL) provides a communication-efficient approach to solve machine learning problems concerning distributed data, without sending raw data to a central server. However, existing works on FL only utilize first-order gradient descent (GD) and do not consider the preceding iterations to gradient update w…
New method addresses class imbalance in federated learning.
problem Class imbalance in federated learning training data.
method Proposes a monitoring scheme to infer training data composition and a new loss function, Ratio Loss, to mitigate imbalance.
result Demonstrates effectiveness in mitigating class imbalance in federated learning, outperforming previous methods.