LASG improves communication efficiency in distributed learning.
problem Efficiently communicating gradients in distributed machine learning.
method Develops a new stochastic gradient descent approach, LASG, that predicts and selects significant communication rounds.
result Achieves communication savings by an order of magnitude in federated learning.
DES training speeds up large-scale recommender systems convergence.
problem Training large-scale recommender systems with dynamic sparse features.
method Distributed Equivalent Substitution (DES) framework for fully synchronous training.
result DES achieves higher AUC and up to 68.7% communication savings.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
problem Byzantine attacks and communication inefficiency in federated learning.
method Transformed robust aggregation for zero-order optimization under client heterogeneity.
result CyBeR-0 achieves stable performance with minimal communication costs and reduced memory usage.
To accommodate heterogeneous tasks in Internet of Things (IoT), a new communication and computing paradigm termed mobile edge computing emerges that extends computing services from the cloud to edge, but at the same time exposes new challenges on security. The present paper studies online security-aware edge computing …
New compression schemes save communication in distributed mean estimation.
problem Efficiently compressing vectors for mean estimation in a limited communication setting.
method Collaborative compression schemes that exploit vector similarities without requiring known correlations.
result Analysis shows varying error types with vector similarity.
Paper improves communication in decentralized federated learning for EHRs.
problem Efficiently learn from large, decentralized EHR databases.
method Fully decentralized federated learning with iterative local updates and reduced communication rounds.
result Significant reduction in communication rounds without compromising solution optimality.
A new method reduces communication in distributed learning by skipping less informative gradient updates.
problem Efficient communication in distributed machine learning.
method Quantizes and skips less informative gradients to reduce communication overhead.
result Proves linear convergence rate similar to gradient descent with significant communication savings.
A new method speeds up community detection in graphs.
problem Efficiently detecting communities in large graphs.
method Subsampled semidefinite programming for faster clustering.
result Statistical limits of sketching for community detection established.
D2P-Fed improves privacy and communication in federated learning.
problem Achieving both differential privacy and communication efficiency in federated learning.
method Applying discrete Gaussian noise to private data transmission.
result D2P-Fed outperforms state-of-the-art by 4.7% to 13.0% in model accuracy with one-third less communication cost.
Common event-triggered state estimation (ETSE) algorithms save communication in networked control systems by predicting agents' behavior, and transmitting updates only when the predictions deviate significantly. The effectiveness in reducing communication thus heavily depends on the quality of the dynamics models used …
Paper reduces communication in distributed machine learning.
problem Reduces burdensome communication in distributed machine learning.
method Introduces communication-censoring technique to reduce transmissions of variables.
result CSGD algorithm achieves same convergence rate as SGD but with significant communication reduction.
Modern distributed training of machine learning models suffers from high communication overhead for synchronizing stochastic gradients and model parameters. In this paper, to reduce the communication complexity, we propose \emph{double quantization}, a general scheme for quantizing both model parameters and gradients. …
New approach uses SPG for semantic communication without a known channel model.
problem Designing efficient semantic communication systems without a known channel model.
method Applying Stochastic Policy Gradient (SPG) for reinforcement learning.
result Achieves comparable performance to model-aware approaches with a decreased convergence rate.
Event-based learning reduces communication in distributed networks.
problem Distributed learning with diverse data distributions and communication inefficiencies.
method A distributed learning algorithm using ADMM with event-triggered communication.
result The algorithm converges even with distinct local data distributions and achieves accelerated convergence in convex settings.
A new compression technique reduces communication costs in distributed deep learning.
problem Communication bottlenecks in distributed deep learning training.
method Natural Compression (NC) and Natural Dithering.
result Natural Compression increases communication savings by 3-4 times with negligible convergence speed impact.
Optimizes network sampling for efficient community detection.
problem Prohibitive cost of observing entire network for community detection.
method Chernoff-optimal dynamic sampling scheme for stochastic blockmodel.
result Significant resource savings while maintaining block structure recovery.
Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random communication delays. This work considers a distributed training framework where each worker node is allowed to perform local model updates a…
MATCHA speeds up decentralized SGD by parallelizing communication.
problem Error-runtime trade-off in decentralized SGD.
method MATCHA decomposes network topology into matchings for efficient communication.
result MATCHA reduces communication time by up to 5x compared to vanilla decentralized SGD.
SPARQ-SGD optimizes communication in decentralized SGD with event-triggered and compressed updates.
problem Efficient communication in decentralized stochastic optimization for large-scale models.
method Event-triggered and compressed algorithm with quantized and sparsified model parameters.
result SPARQ-SGD converges with efficiency comparable to uncompressed training, demonstrating significant communication savings.
New system preserves message meaning in wireless networks, improving data rate.
problem Efficiently transmitting message meaning in wireless networks.
method Modeling semantics as hidden random variables, using Information Bottleneck for compression.
result 20 dB SNR improvement for semantic communication.
Paper uses deep learning to optimize resource allocation in ultra-reliable communications.
problem Optimizing resource allocation for ultra-reliable and low-latency communications.
method Unsupervised deep learning for joint power and bandwidth allocation.
result Deep learning finds an approximated optimal solution with QoS constraints.
Proposes a new algorithm to reduce communication costs in decentralized training.
problem How to apply error-compensated compression to decentralized training.
method Error-compensated stochastic gradient descent for decentralized training.
result Proposed algorithm outperforms existing methods in communication cost reduction.
COMP-AMS optimizes distributed training with compressed gradients, achieving similar accuracy with less communication.
problem Efficiently training large-scale models in distributed environments with reduced communication costs.
method Distributed optimization framework using gradient averaging and adaptive AMSGrad, with gradient compression and error feedback.
result COMP-AMS achieves the same convergence rate and linear speedup as standard AMSGrad with less communication.
IoT nodes compress measurements into DNN outputs for efficient communication.
problem Efficient communication of high-dimensional IoT data with limited bandwidth.
method Modeling IoT node measurements as DNN intermediate outputs and optimizing model parameters.
result Approximately 96% reduction in transmissions with only 2.5% loss in inference accuracy.
COMRADE is a communication-efficient, Byzantine-resilient second-order optimization algorithm.
problem Byzantine failures in distributed optimization.
method COMRADE is a communication-efficient, second-order optimization algorithm that uses a simple norm-based thresholding rule to filter out Byzantine workers.
result COMRADE achieves linear-quadratic convergence and is robust against Byzantine workers.
Patient summarization is essential for clinicians to provide coordinated care and practice effective communication. Automated summarization has the potential to save time, standardize notes, aid clinical decision making, and reduce medical errors. Here we provide an upper bound on extractive summarization of discharge …
Handling the tremendous amount of network data, produced by the explosive growth of mobile traffic volume, is becoming of main priority to achieve desired performance targets efficiently. Opportunistic communication such as FloatingContent (FC), can be used to offload part of the cellular traffic volume to vehicular-to…
This work improves model estimation efficiency and subgroup identification in networked systems.
problem Improving model estimation efficiency and subgroup identification in networked systems.
method A tree-based l1 penalty and decentralized ADMM algorithm are used to solve the objective function in parallel. result The approach outperforms in estimation accuracy, computation speed, and communication cost.
PETRA enables parallel training of deep models with reversible architectures.
problem Challenges in parallelizing deep model training.
method Introduces PETRA, a novel approach for parallelizing gradient computations in reversible architectures.
result Achieves competitive accuracies on CIFAR-10, ImageNet32, and ImageNet using ResNet models.
Quantum networks offer exponential communication savings for large machine learning models.
problem Training and inference of large models require efficient communication.
method Quantum encoding and gradient descent for distributed computation.
result Exponential reduction in communication for gradient descent on quantum networks.
Qsparse-local-SGD reduces communication in large-scale learning models.
problem Communication bottleneck in distributed optimization of large-scale models.
method Combines sparsification, quantization, and local computation with error compensation.
result Converges at the same rate as vanilla distributed SGD for many sparsifiers and quantizers.
We introduce an online tensor decomposition based approach for two latent variable modeling problems namely, (1) community detection, in which we learn the latent communities that the social actors in social networks belong to, and (2) topic modeling, in which we infer hidden topics of text articles. We consider decomp…
Argentum is a crypto coin for saving and investment in unstable countries.
problem Stable purchasing power for savings in unstable economies.
method Designing a crypto coin backed by investment instruments.
result Provides a stabilization instrument for savings in unstable economies.
Optimal privacy and accuracy in distributed mean estimation with compression.
problem Achieving optimal accuracy under privacy and communication constraints.
method Compression to reduce communication while maintaining privacy and accuracy.
result Achieves optimal error with significantly reduced communication.
Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough …
Prize linked savings accounts provide a return in the form of randomly chosen accounts receiving large cash prizes, in lieu of a guaranteed and uniform interest rate. This model became legal for American national banks upon bipartisan passage of the American Savings Promotion Act in December 2014, and many states have …
EF21-Muon optimizes deep learning with error feedback, improving efficiency and accuracy.
problem Lack of principled distributed frameworks for non-Euclidean LMO-based optimizers.
method Introduces EF21-Muon, a communication-efficient, non-Euclidean LMO-based optimizer with convergence guarantees.
result First efficient distributed implementation of non-Euclidean LMO-based optimizers, achieving up to 7x communication savings.
Study on pooled annuity funds and how initial savings affect income stability.
problem Analyzing the stability of income payments in pooled annuity funds.
method Examining the influence of initial savings on income fluctuations and developing a criterion for pooling funds.
result Identification of a term, the 'implied number of homogeneous members', linking initial savings to income fluctuations.
Ricean channel model is widely used in wireless communications to characterize the channels with a line-of-sight path. The Ricean K factor, defined as the ratio of direct path and scattered paths, provides a good indication of the link quality. Most existing works estimate K factor based on either maximum-likelihood cr…
The effects of saving and spending patterns on holding time distribution of money are investigated based on the ideal gas-like models. We show the steady-state distribution obeys an exponential law when the saving factor is set uniformly, and a power law when the saving factor is set diversely. The power distribution c…
We analyze the ideal gas like models of markets and review the different cases where a `savings' factor changes the nature and shape of the distribution of wealth. These models can produce similar distribution of wealth as observed across varied economies. We present a more realistic model where the saving factor can v…
We review a simple model of closed economy, where the economic agents make money transactions and a saving criterion is present. We observe the Gibbs distribution for zero saving propensity, and non-Gibbs distributions otherwise. While the exact solution in the case of zero saving propensity is already known to be give…
We consider a simple model of a closed economic system where the total money is conserved and the number of economic agents is fixed. In analogy to statistical systems in equilibrium, money and the average money per economic agent are equivalent to energy and temperature, respectively. We investigate the effect of the …
SAVED safely learns robot tasks with sparse rewards.
problem Challenges in reinforcement learning for robotics, especially sparse rewards and complex constraints.
method SAVED uses supervision to constrain exploration and learn efficiently, handling complex constraints.
result SAVED outperforms state-of-the-art methods in success rate, constraint satisfaction, and sample efficiency.
This work proposes ACTC for adaptive distributed learning under communication constraints.
problem Adaptive distributed learning in networks with communication constraints.
method ACTC (Adapt-Compress-Then-Combine) strategy with diffusion exchange of compressed updates.
result ACTC iterates converge to the optimizer with significant bit savings.
Study explores factors influencing saving behavior among Dhaka employees.
problem Factors influencing saving behavior among Dhaka employees.
method Quantitative approach with cross-sectional survey design, structured questionnaire, descriptive statistics, reliability analysis, regression analysis.
result Only financial management practices had a significant positive relationship with saving behavior.
We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum soluti…
This paper proposes two mixed models to study a consumer's optimal saving in the presence of two types of risk.