Adaptive optimizers improve federated learning performance.
problem Inadequate optimization methods in federated learning.
method Federated versions of adaptive optimizers (Adagrad, Adam, Yogi).
result Adaptive optimizers enhance federated learning performance.
New algorithm solves federated minimax optimization problems.
problem Federated minimax optimization challenges.
method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).
result FESS-GDA uniformly solves federated minimax problems.
A novel decentralized algorithm improves minimax optimization in federated learning.
problem Minimax optimization in federated learning with data heterogeneity.
method Decentralized Gradient Tracking (K-GT-Minimax) for nonconvex-strongly-concave optimization.
result Demonstrates superior convergence rate for NC-SC minimax optimization.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
A novel Federated Learning scheme using Optimal Transport for personalized model training.
problem Training models with data from clients having non-identically distributed data.
method Personalized Federated Learning scheme based on Optimal Transport (FedOT).
result FedOT scheme effectively transfers data from multiple distributions to a common domain and optimizes the prediction model.
New algorithm achieves linear speedup in non-i.i.d. federated bilevel learning.
problem Linear speedup in convergence for non-i.i.d. datasets in federated bilevel optimization.
method Proposes FedMBO with a novel client sampling scheme for non-i.i.d. datasets.
result Achieves a convergence rate of O(1/√(nK) + 1/K + √n/K³/²).
A new federated learning method reduces communication costs and improves adaptivity.
problem Large communication overhead and lack of adaptivity in federated learning.
method FedCAMS: A novel communication-efficient adaptive federated learning method with theoretical guarantees.
result FedCAMS achieves the same convergence rate as non-compressed federated learning methods.
New method FedEx accelerates federated hyperparameter tuning.
problem Federated hyperparameter tuning challenges in distributed learning.
method FedEx method connecting to weight-sharing, adapted for federated optimization.
result FedEx outperforms natural baselines on various benchmarks.
Second-order guarantees for federated learning algorithms.
problem Non-convex optimization in federated learning with saddle-points as bottlenecks.
method Drawing on recent results on second-order optimality in centralized and decentralized settings, establish second-order guarantees for federated learning algorithms.
result Established second-order guarantees for federated learning algorithms.
A new algorithm optimizes local objectives in federated learning with heterogeneous clients.
problem Optimizing local objectives in federated learning with heterogeneous client data.
method Proposes PF-PNE algorithm with double elimination strategy.
result PF-PNE algorithm optimizes local objectives with arbitrary heterogeneity and protects client data confidentiality.
FedCluster accelerates federated learning convergence by cycling device groups.
problem Federated learning convergence issues with device-level data heterogeneity.
method FedCluster groups devices into clusters that cycle through learning rounds, boosting convergence with meta-updates.
result FedCluster achieves faster convergence in nonconvex optimization compared to FedAvg.
This dissertation advances the theoretical foundation of local optimization methods in Federated Learning.
problem Theoretical understanding of local optimization methods in Federated Learning is lacking.
method The dissertation proposes and analyzes new methods to improve convergence rates and communication efficiency in Federated Learning.
result Sharp bounds and convergence rates for FedAvg are established, and new methods like FedAc and Federated Dual Averaging are proposed.
Paper proposes AggITD for efficient federated hypergradient computation.
problem Computing hypergradient in federated settings is challenging due to distributed and nonlinear construction of global Hessian matrices.
method AggITD: a novel communication-efficient federated hypergradient estimator via aggregated iterative differentiation.
result AggITD achieves the same sample complexity as AID-based approaches but with fewer communication rounds, especially in heterogeneous data environments.
FedIV uses federated GMM for IV analysis in non-i.i.d. data.
problem Efficient IV analysis in non-i.i.d. federated data.
method Federated GMM via FedGDA algorithm.
result Federated solution consistently estimates local moment conditions.
New algorithm for federated learning with non-smooth regularizers.
problem Federated Learning with non-smooth composite optimization problems.
method Proposed Federated Dual Averaging (FedDualAvg) algorithm to overcome convergence issues.
result FedDualAvg outperforms other algorithms in federated composite optimization.
Personalized federated learning for diverse client objectives.
problem Training a single global model across diverse local datasets is not optimal.
method Efficiently calculates optimal weighted model combinations for each client based on their specific objectives.
result Our method outperforms existing alternatives and enables new personalized features.
The paper analyzes privacy leakage in federated learning using linear algebra and optimization theory.
problem Privacy leakage in federated learning despite its promise for data privacy.
method Theoretical analysis from linear algebra and optimization theory perspectives.
result Derives sufficient conditions to prevent data reconstruction attacks and establishes an upper bound on privacy leakage.
We address challenges in collaborative black-box optimization through three frameworks.
problem Challenges in distributed experimentation, heterogeneity, and privacy in black-box optimization.
method Three unifying frameworks: global, local, and predictive.
result Shift from descriptive/predictive to prescriptive federated learning in black-box optimization.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
problem Byzantine attacks and communication inefficiency in federated learning.
method Transformed robust aggregation for zero-order optimization under client heterogeneity.
result CyBeR-0 achieves stable performance with minimal communication costs and reduced memory usage.
A real-time federated neural architecture search approach reduces costs and improves performance.
problem High communication and computational demands in federated learning for large models.
method Evolutionary approach with double-sampling technique to optimize model performance and reduce costs.
result Effective real-time federated neural architecture search for deep models on edge devices.
This paper tackles objective inconsistency in federated optimization with heterogeneous clients.
problem Objective inconsistency due to heterogeneity in clients' datasets and computation speeds.
method General framework for analyzing federated heterogeneous optimization algorithms, including FedAvg and FedProx, and proposing FedNova.
result FedNova eliminates objective inconsistency while preserving fast error convergence.
Efficiently addresses federated learning challenges with reduced communication and sample complexity.
problem Heterogeneity in data volumes and distributions at different clients compromises model generalization ability.
method Introduces algorithms for communication-efficient Federated Group Distributionally Robust Optimization (FGDRO).
result Communication complexity reduced to O(1/ε4) for FGDRO-CVaR and O(1/ε3) for FGDRO-KL. TCT improves federated learning by convexifying neural networks.
problem Federated learning performance disparity due to nonconvexity.
method Train-Convexify-Train procedure using neural tangent kernels.
result Up to +36% accuracy improvement on FMNIST and +37% on CIFAR10.
FeDualEx tackles saddle point optimization in federated learning with composite objectives.
problem Saddle point optimization with constraints and non-smooth regularization in federated learning.
method Federated Dual Extrapolation (FeDualEx) algorithm for saddle point optimization and composite objectives.
result FeDualEx effectively solves saddle point optimization problems with composite objectives in federated learning.
New algorithm reduces sample and communication complexities in federated Q-learning.
problem Optimal Q-function learning in federated Q-learning with limited communication.
method Introduced Fed-DVR-Q algorithm for order-optimal sample and communication complexities.
result Complete characterization of sample-communication complexity trade-off.
FedSplit improves federated learning by ensuring correct convergence to optimal solutions.
problem Federated learning's fixed points do not always correspond to optimal solutions in simple convex settings.
method FedSplit uses operator splitting procedures to solve distributed convex minimization problems with additive structure.
result FedSplit ensures that the fixed points correspond to optima of the original optimization problem.
FedCBO solves clustered federated learning by optimizing groups of users without knowing their structure.
problem Training models for multiple users with privacy and communication constraints, especially in clustered settings.
method FedCBO uses a particle system approach inspired by consensus-based optimization to train models for each user group.
result FedCBO outperforms other methods in training models for clustered federated learning.
Locally adaptive federated learning improves convergence in distributed machine learning.
problem Balancing local updates in federated learning leads to slow convergence.
method Locally adaptive federated learning algorithms that use uncoordinated stepsizes based on local geometric information.
result Locally adaptive methods can be particularly efficient in overparameterized settings and outperform standard federated algorithms.
Muon optimizer improves federated learning performance.
problem Improving federated learning performance with Muon optimizer.
method Proposed FedMuon algorithm with orthonormalized update direction.
result FedMuon's learning rate is independent of problem-specific parameters.
Adaptive personalized federated learning improves local model personalization.
problem Maximizing global model performance limits local model personalization.
method APFL algorithm trains local models while contributing to global model, with optimal mixing parameter and communication-efficient optimization.
result Demonstrates effectiveness of personalization schema and correctness of generalization theories.
FedForest adapts RF for federated learning, improving performance and efficiency.
problem Adapting RF for federated learning with heterogeneous data.
method FedForest uses a novel splitting procedure to aggregate client statistics, allowing non-parametric personalization.
result FedForest's federated RF achieves performance close to centralized models while being communication-efficient.
Federated learning adapts to data and model drifts.
problem Performance guarantees in federated learning for dynamic problems.
method Random subset updates, non-stationary random walk model, tracking term.
result Performance depends on data variability, model variability, and learning rate.
Federated learning is an emerging technique used to prevent the leakage of private information. Unlike centralized learning that needs to collect data from users and store them collectively on a cloud server, federated learning makes it possible to learn a global model while the data are distributed on the users' devic…
New algorithm improves fairness and robustness in federated learning.
problem Ensuring fairness and robustness in federated learning.
method Formulated federated learning as multi-objective optimization and proposed FedMGDA+.
result FedMGDA+ converges to Pareto stationary solutions, improving performance.
New algorithms solve nonconvex federated learning problems efficiently.
problem Nonconvex federated composite optimization in federated learning.
method FedDR and asyncFedDR algorithms combining Douglas-Rachford splitting, randomized block-coordinate strategies, and asynchronous implementation.
result Match communication complexity lower bound up to a constant factor.
New federated learning protocols resist Byzantine failures and offer privacy guarantees.
problem Resisting Byzantine failures in federated learning.
method Proposes robust federated learning protocols with optimal statistical rates and privacy guarantees.
result Achieves nearly optimal statistical rates and tight rate in terms of all parameters for strongly convex losses.
Paper proposes a federated learning framework for relative fairness.
problem Traditional fairness in federated learning overlooks performance disparities between client subgroups.
method Uses a minimax problem approach to minimize relative unfairness, introducing a fairness index based on loss ratios.
result Empirical evaluations confirm the framework's effectiveness in maintaining model performance while reducing disparity.
Unified approach for federated learning using MM optimization.
problem Scaling stochastic optimization to federated learning.
method Unified Majorize-Minimize (MM) framework for stochastic optimization, extended to federated learning.
result Unified algorithm \QSMM\ for federated learning that aggregates surrogate majorizing functions.
Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.
problem Near-optimal federated Q-learning with low communication cost.
method Reference-advantage decomposition for variance reduction, synchronization between agents and server, policy update.
result Achieves almost optimal regret and near-linear regret speedup compared to single-agent learning.
New study tackles free-rider attacks in federated learning models.
problem Free-rider attacks compromise federated learning models by non-contributing to data updates.
method Theoretical and experimental analysis of iterative federated learning schemes, including FedAvg and FedProx.
result Formal guarantees for free-rider attacks to converge to aggregated models of fair participants.
FedAc accelerates Federated Averaging for distributed optimization.
problem Efficiently optimizing distributed machine learning models.
method Federated Accelerated Stochastic Gradient Descent (FedAc) using a potential-based perturbed iterate analysis.
result FedAc achieves faster convergence and lower communication costs than previous methods.
Study on collaborative vs. non-collaborative online and bandit convex optimization.
problem Minimizing average regret in distributed online and bandit convex optimization.
method Analyzes the impact of collaboration in adaptive and zeroth-order feedback settings.
result Collaboration is beneficial in high-dimensional federated online optimization with limited feedback.
Federated learning improves SPCA for sparse components.
problem Data privacy and sharing constraints in machine learning.
method Federated learning framework applied to SPCA with L1 regularization and smoothing.
result Federated SPCA achieves sparse component loadings with improved interpretability.
Federated learning method improves covariate shift adaptation for missing target values.
problem Missing target values in federated learning.
method Federated covariate shift adaptation algorithm for missing target output values.
result Asymptotically unbiased and efficient algorithm for federated learning.
Paper analyzes minimax risks of personalized federated learning algorithms.
problem Statistical heterogeneity among clients in federated learning.
method Minimax analysis of FedAvg and local training approaches.
result Threshold for optimality between FedAvg and local training depends on data heterogeneity.
A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new f…
FLIX simplifies federated learning with efficient communication.
problem Handling constraints specific to federated learning.
method Introduces FLIX, a new framework for federated learning that addresses communication and personalization challenges.
result FLIX achieves dissimilarity regularization similar to local methods without requiring local steps.
FedSplit improves federated learning for heterogeneous data.
problem Data heterogeneity in federated learning degrades convergence and performance.
method FedSplit splits data into shared and personalized groups, optimizing a novel objective function.
result FedSplit converges faster and performs better than standard federated learning.