This article improves communication efficiency in distributed ML over wireless networks.
problem Achieving high ML inference accuracy at scale with zero communication latency.
method Optimizing communication payload types, techniques, scheduling, and ML architectures.
result Communication-efficient and distributed learning frameworks are presented.
Efficient decentralized learning framework reduces communication costs.
problem Efficiently solve optimization problems in distributed learning networks.
method Censored and Quantized Generalized GADMM (CQ-GGADMM) framework.
result Achieves linear convergence rate under strong convexity assumptions.
DP-REC combines privacy and communication efficiency in federated learning.
problem Combining privacy and communication efficiency in federated learning.
method DP-REC uses Relative Entropy Coding (REC) for compression and a minor modification for differential privacy.
result DP-REC reduces communication costs while maintaining privacy comparable to state-of-the-art methods.
IC3Net improves communication efficiency in multiagent tasks.
problem Efficient communication in multiagent tasks, especially in semi-cooperative and competitive settings.
method IC3Net uses a gating mechanism and individualized rewards to control continuous communication, improving scalability and performance.
result IC3Net yields improved performance and convergence rates compared to baselines as task scale increases.
Distributed stochastic gradient descent is an important subroutine in distributed learning. A setting of particular interest is when the clients are mobile devices, where two important concerns are communication efficiency and the privacy of the clients. Several recent works have focused on reducing the communication c…
Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalability of distributed SGD. However, a rigorous convergence analysis and comparative study of different communication-reduction strategies rem…
DiffSketch combines privacy and communication efficiency in distributed learning.
problem Privacy and communication efficiency in distributed machine learning.
method DiffSketch uses Count Sketch for data stream summarization to achieve both privacy and efficiency.
result DiffSketch provides strong differential privacy guarantees and significant communication compression.
This paper proposes a communication-efficient distributed algorithm for high-dimensional data mining.
problem Reducing communication time and rounds in distributed data mining.
method Straggler-agnostic and bandwidth-efficient distributed primal-dual algorithm.
result Guaranteed linear convergence rate for convex problems.
Efficient algorithm reduces communication costs in sparse regression.
problem Sparse linear regression with massive data.
method CESDAR algorithm, communication-efficient surrogate likelihood.
result Achieves same statistical accuracy as global estimator with reduced communication.
This paper improves hierarchical community detection efficiency using local structural properties.
problem Efficiency of hierarchical community detection methods in large networks.
method Use of local structural network properties as proxies to improve efficiency.
result Achieves competitive results in modularity with improved efficiency.
New algorithms reduce communication costs in collaborative learning.
problem Reducing communication costs in collaborative learning.
method Distributed boosting and adaptation to classification noise.
result Communication-efficient algorithms for collaborative PAC learning robust to noise.
This paper compares communication efficiency of split learning and federated learning in various scenarios.
problem Comparing communication efficiency of split learning and federated learning in different settings.
method Examined various practical scenarios of distributed learning setups and compared the two methods.
result Communication efficiency of split learning and federated learning depends on the number of clients, model size, and data samples.
New algorithm for multi-agent reinforcement learning with reduced communication.
problem Cooperative learning among multiple agents with limited communication.
method Randomized multi-agent actor-critic algorithm for directed graphs.
result Algorithm solves problem for strongly connected graphs with reduced communication.
Efficient communication improves MARL performance in StarCraft II.
problem Achieving efficient communication among agents in multi-agent reinforcement learning.
method Variance Based Control (VBC) to limit message variance and preserve useful information.
result VBC reduces communication overhead by 2-10 times compared to state-of-the-art MARL algorithms.
LD-SGD improves communication in decentralized SGD.
problem Efficiently combining local updates and decentralized communication.
method Proposes LD-SGD integrating local updates and decentralized SGD, with a convergence analysis.
result LD-SGD converges to a critical point for non-convex objectives with non-identically distributed data.
Flexible framework improves communication efficiency across various systems.
problem Reducing communication between nodes in machine learning tasks.
method Adapts compression level to true gradient at each iteration, optimizing per-bit improvement.
result Automatic tuning strategies significantly increase communication efficiency.
Paper studies fundamental limits of communication in distributed learning.
problem Communication efficiency in model aggregation for distributed learning.
method Rate-Distortion approach to model aggregation as a vector Gaussian CEO problem.
result Derives rate region bound and sum-rate-distortion function for model aggregation.
We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For…
HSQ reduces communication costs in federated learning.
problem High cost of communicating gradients in federated learning.
method Hyper-sphere quantization (HSQ) framework for efficient gradient compression.
result HSQ achieves O(logd) per-iteration communication cost, significantly reducing costs without compromising accuracy. Efficient method for vertex embedding and community detection.
problem Vertex embedding and community detection.
method Normalized one-hot graph encoder and rank-based cluster size measure.
result Excellent numerical performance of graph encoder ensemble algorithm.
Paper proposes AggITD for efficient federated hypergradient computation.
problem Computing hypergradient in federated settings is challenging due to distributed and nonlinear construction of global Hessian matrices.
method AggITD: a novel communication-efficient federated hypergradient estimator via aggregated iterative differentiation.
result AggITD achieves the same sample complexity as AID-based approaches but with fewer communication rounds, especially in heterogeneous data environments.
Quantized Frank-Wolfe reduces communication costs in distributed optimization.
problem Efficiently reducing communication overhead in distributed machine learning optimization.
method Quantized Frank-Wolfe (QFW), a projection-free algorithm for constrained optimization.
result Strong theoretical guarantees on convergence rate, efficient compression of gradients.
DESTRESS optimizes decentralized nonconvex optimization with optimal IFO complexity and efficient communication.
problem Decentralized nonconvex finite-sum optimization in multi-agent systems.
method DESTRESS uses stochastic recursive gradient updates, gradient tracking, and careful hyper-parameter choices to achieve optimal IFO complexity with efficient communication.
result DESTRESS matches the optimal IFO complexity of centralized algorithms while maintaining communication efficiency.
LSGD improves deep learning training efficiency by synchronizing and decentralizing SGD.
problem Asynchronous SGD's accuracy issues and synchronous SGD's communication inefficiency.
method LSGD divides nodes into subgroups with centralized communication and decentralized computation.
result LSGD achieves better accuracy and efficiency than synchronous and asynchronous SGD.
Compressed Federated Distillation reduces communication in federated learning.
problem Communication constraints in Federated Learning.
method Compressed Federated Distillation (CFD) leverages soft labels and quantization techniques.
result Reduces communication by more than 4 orders of magnitude compared to Federated Averaging.
Deep learning optimizes vehicular communication zones for efficient data dissemination.
problem Overdimensioning and inefficient communication in vehicular floating content.
method Deep learning is used to select optimal broadcasting areas (Anchor Zones) for efficient message dissemination.
result The proposed method achieves an accuracy of 89.7% in predicting optimal Anchor Zones, saving up to 27% of resources.
Local AdaAlter reduces communication in SGD with adaptive learning rates.
problem Communication overhead in distributed training.
method Novel SGD variant with adaptive learning rates and reduced communication.
result Empirically reduces communication overhead by up to 30%.
The paper examines efficient algorithms for linear regression over resource-limited networks.
problem Efficient communication in distributed learning over resource-limited networks.
method Developed algorithms for communication-efficient learning of linear regression tasks.
result The algorithms enable a tradeoff between communication and learning with theoretical performance guarantees.
Paper improves communication in decentralized federated learning for EHRs.
problem Efficiently learn from large, decentralized EHR databases.
method Fully decentralized federated learning with iterative local updates and reduced communication rounds.
result Significant reduction in communication rounds without compromising solution optimality.
GADMM reduces communication costs in distributed machine learning.
problem Efficiently solving distributed machine learning problems with reduced communication costs.
method Group Alternating Direction Method of Multipliers (GADMM) framework.
result GADMM converges to the optimal solution for convex loss functions and is faster and more communication-efficient than state-of-the-art algorithms.
This paper tackles energy-efficient machine learning on low-power devices.
problem Energy consumption in machine learning due to data communication.
method Dynamic averaging for integer exponential families on low-power processors.
result Achieves comparable model quality with significantly less communication and energy.
Scaff-PD improves fairness and robustness in federated learning with reduced communication.
problem Improving fairness and robustness in federated learning with limited communication.
method Scaff-PD uses a family of distributionally robust objectives and an accelerated primal dual algorithm with bias-corrected steps.
result Scaff-PD achieves significant gains in communication efficiency and convergence speed while maintaining fairness and robustness.
D2P-Fed improves privacy and communication in federated learning.
problem Achieving both differential privacy and communication efficiency in federated learning.
method Applying discrete Gaussian noise to private data transmission.
result D2P-Fed outperforms state-of-the-art by 4.7% to 13.0% in model accuracy with one-third less communication cost.
A new gradient quantization scheme improves communication efficiency in distributed training.
problem Efficiently compressing gradients for parallel training of large models.
method Proposes a new gradient quantization scheme with theoretical guarantees and empirical performance.
result The new scheme matches and exceeds the performance of existing methods.
Efficiently tests discrete distributions with limited memory and communication.
problem Testing discrete distributions with constraints on memory and communication.
method Developed efficient algorithms for uniformity/identity and closeness testing in streaming and distributed models.
result Nearly-tight lower bounds on sample complexity and communication cost for uniformity testing.
LASG improves communication efficiency in distributed learning.
problem Efficiently communicating gradients in distributed machine learning.
method Develops a new stochastic gradient descent approach, LASG, that predicts and selects significant communication rounds.
result Achieves communication savings by an order of magnitude in federated learning.
Near-optimal regret in distributed bandit learning with efficient communication protocols.
problem Minimizing total regret in collaborative bandit learning with limited communication.
method Proposed communication protocols for distributed multi-armed and linear bandits with near-optimal regret and efficient communication costs.
result Achieved near-optimal regret with communication costs independent of time horizon and number of arms.
Paper develops efficient algorithms for robust distributed learning with statistical guarantees.
problem Limited communication power and adversarial node behaviors in distributed learning.
method Surrogate likelihood framework and median/trimmed mean operations.
result Provable robustness against Byzantine failures and optimal statistical rates.
New method improves FL efficiency by shuffling data, balancing privacy and accuracy.
problem Balancing privacy, communication, and accuracy in federated learning.
method Developed communication-efficient schemes for private mean estimation, combining privacy amplification and shuffled data.
result Achieved same privacy, optimization performance with lower communication cost.
Secure and efficient distributed learning on devices with limited communication.
problem Limited communication and security in distributed on-device learning.
method Proposes SLSGD, a robust distributed optimization algorithm with efficient communication and attack tolerance.
result Stabilizes convergence and tolerates data poisoning on a small number of workers.
Paper proposes DCT for efficient hybrid parallel training of large recommendation models.
problem Training large recommendation models at scale with efficient communication.
method Dynamic Communication Thresholding (DCT) for both Data Parallelism and Model Parallelism.
result Reduces communication by 100x and 20x during DP and MP, respectively, improving training time by 37%.
Efficiently addresses federated learning challenges with reduced communication and sample complexity.
problem Heterogeneity in data volumes and distributions at different clients compromises model generalization ability.
method Introduces algorithms for communication-efficient Federated Group Distributionally Robust Optimization (FGDRO).
result Communication complexity reduced to O(1/ε4) for FGDRO-CVaR and O(1/ε3) for FGDRO-KL. A new method for efficient online federated learning reduces communication overhead.
problem Real-world limitations in online federated learning, such as heterogeneous client participation and communication delays.
method Proposes a communication-efficient asynchronous online federated learning (PAO-Fed) strategy.
result Achieves the same convergence properties as online federated stochastic gradient while reducing communication overhead by 98 percent.
Study on efficient estimation of Gaussian mean with limited communication.
problem Estimating Gaussian mean under communication constraints.
method Decomposition into localization and refinement stages, development of communication-efficient and statistically optimal procedures.
result Established minimax rates of convergence and developed optimal procedures.
AdaQuantFL reduces communication in federated learning by adaptively quantizing model updates.
problem Efficient communication of model updates in federated learning with high-dimensional models and limited bandwidth.
method AdaQuantFL uses adaptive quantization to reduce the number of bits for model updates while maintaining low error floor.
result AdaQuantFL converges in fewer communicated bits compared to fixed quantization levels, with minimal impact on accuracy.
We consider the problem of learning from distributed data in the agnostic setting, i.e., in the presence of arbitrary forms of noise. Our main contribution is a general distributed boosting-based procedure for learning an arbitrary concept space, that is simultaneously noise tolerant, communication efficient, and compu…
A scalable distributed S-LSR1 algorithm reduces communication costs.
problem Efficiently scaling S-LSR1 for large-scale distributed optimization.
method Proposes DS-LSR1, a communication-efficient variant of S-LSR1.
result DS-LSR1 scales well in problem dimension and data points.
A new method reduces communication in distributed learning by skipping less informative gradient updates.
problem Efficient communication in distributed machine learning.
method Quantizes and skips less informative gradients to reduce communication overhead.
result Proves linear convergence rate similar to gradient descent with significant communication savings.