Paper studies fundamental limits of communication in distributed learning.
problem Communication efficiency in model aggregation for distributed learning.
method Rate-Distortion approach to model aggregation as a vector Gaussian CEO problem.
result Derives rate region bound and sum-rate-distortion function for model aggregation.
The paper examines efficient algorithms for linear regression over resource-limited networks.
problem Efficient communication in distributed learning over resource-limited networks.
method Developed algorithms for communication-efficient learning of linear regression tasks.
result The algorithms enable a tradeoff between communication and learning with theoretical performance guarantees.
In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities na and nb respectively. We consider the sparse setting, in which a and b do not scale with n, and…
New method uses word embedding techniques for better graph community discovery.
problem Discovering communities in graphs without labeled data.
method Developed a novel algorithm using neural node embeddings for unsupervised community discovery.
result Empirically attains information-theoretic limits for community recovery and outperforms existing methods.
Study on limits of community detection in various network models.
problem Limits of community detection in network models.
method Analysis of several network models including Stochastic Block Model, Exponential Random Graph Model, Latent Space Model, Directed Preferential Attachment Model, and Directed Small-world Model.
result Information-theoretic limits for recovery of node labels in network models.
Method infers assortative communities in networks without resolution limit.
problem Finding statistically significant assortative modules in networks.
method Nonparametric Bayesian formulation of the planted partition model.
result Method uncovers an arbitrarily large number of communities with statistical evidence.
StreamBP optimally detects communities in growing networks.
problem Detect communities in dynamic networks with limited updates.
method Streaming Stochastic Block Model (StSBM) and StreamBP algorithm.
result StreamBP optimally detects communities in certain network growth regimes.
Study neural communication systems with bandwidth-limited channels.
problem Reliable message transmission despite noisy channels.
method Jointly model compression and error correction with neural networks; introduce prior for missing information; use auxiliary latent variables.
result Joint neural communication systems outperform separate models under expected information loss.
Study explores how to efficiently explore communities with limited budget.
problem Maximizing the number of members met with limited budget in community exploration.
method Systematic study from offline optimization to online learning, including greedy methods and upper confidence algorithms.
result Achieved logarithmic and constant regret bounds in online learning setting.
Study on limits of LLM-based multi-agent planning reliability.
problem Reliability limits of LLM-based multi-agent planning.
method Modeling LLM-based multi-agent architecture as a decision network, showing dominance by centralized Bayes decision maker.
result Optimizing multi-agent directed acyclic graphs under communication budget is equivalent to choosing a constrained experiment.
A new method speeds up community detection in graphs.
problem Efficiently detecting communities in large graphs.
method Subsampled semidefinite programming for faster clustering.
result Statistical limits of sketching for community detection established.
Study information limits for community detection in sub-hypergraphs.
problem Identify limits for exact community detection in sub-hypergraphs.
method Use Fano's inequality to define model parameters and identify success and failure regions.
result Identify regions where algorithms succeed or fail in exact recovery.
Improved PAC guarantees for multi-agent reinforcement learning with noisy communication.
problem Improving exploration in cooperative multi-agent reinforcement learning with communication constraints.
method Develops PAC guarantees for multiple concurrent MDPs with noisy and resource-limited communication.
result Theoretical and empirical improvements in sample complexity for information fusion.
The paper sets communication limits for distributed optimization with feature-based data partitions.
problem Understanding communication limits in distributed convex optimization with feature-based data partitions.
method Developed tight lower bounds on communication rounds for non-incremental and incremental algorithms.
result Established communication limits for a broad class of algorithms under feature-based data partitioning.
Secure and efficient distributed learning on devices with limited communication.
problem Limited communication and security in distributed on-device learning.
method Proposes SLSGD, a robust distributed optimization algorithm with efficient communication and attack tolerance.
result Stabilizes convergence and tolerates data poisoning on a small number of workers.
Shannon's theory sets limits on information transmission.
problem Limits of information transmission in systems.
method Mathematical theory of communication.
result Defines fundamental limits on information transmission.
We optimize distributed learning algorithms to maintain linear convergence with limited communication.
problem Limited communication time affects the convergence of distributed learning algorithms.
method We design quantizers to compress algorithm information while preserving linear convergence and characterize communication time.
result We show how to co-design machine learning and communication protocols for optimal performance.
We propose a semidefinite programming (SDP) algorithm for community detection in the stochastic block model, a popular model for networks with latent community structure. We prove that our algorithm achieves exact recovery of the latent communities, up to the information-theoretic limits determined by Abbe and Sandon (…
A new approach for cooperative multi-agent reinforcement learning with limited communication, reducing the number of communication rounds.
problem Limited communication in decentralized MARL systems leads to outdated information and unstable learning.
method Base policy prediction technique to estimate gradients and collect samples for a sequence of base policies.
result The proposed algorithm converges to an ε-Nash equilibrium with significantly fewer communication rounds and samples.
Study exact community recovery in noisy SBM with limited queries.
problem Community recovery in noisy stochastic block models with limited queries.
method Balanced uniform querying, two-stage adaptive strategy, sublinear queries, subsampled graph.
result Adaptive querying can improve exact recovery limits in noisy SBM.
Graph energy helps detect communities in networks better than traditional methods.
problem Detecting communities in sparse networks where traditional methods fail.
method Using graph energy based on the full spectrum of adjacency matrices.
result The difference in graph energy between a planted partition model and an Erdős--Rényi network has a distinct transition at the detectability threshold.
This work tackles community recovery in hypergraphs with measurements of varying sizes.
problem Cluster data points into distinct communities based on measurements with varying sizes.
method Characterizes the fundamental limits on the number of measurements required for community reconstruction in hypergraphs with homogeneity and parity measurements, possibly corrupted by noise.
result Characterizes fundamental limits on the number of measurements required for community reconstruction in hypergraphs.
Paper reduces communication costs in distributed LMS algorithms.
problem High communication costs in distributed LMS algorithms.
method Introduces a diffusion LMS strategy to reduce communication costs.
result Significantly reduces communication costs without compromising performance.
ECG improves graph clustering and resolves resolution limit issues.
problem Graph clustering resolution limit issue.
method ECG uses consensus clustering to improve graph clustering.
result ECG alleviates the resolution limit issue and improves partition stability.
The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.
problem Cooperative multi-agent reinforcement learning with function approximation.
method Careful message-passing and cooperative value iteration.
result Achieving near-optimal no-regret learning with limited communication in cooperative multi-agent settings.
We study the fundamental limits on learning latent community structure in dynamic networks. Specifically, we study dynamic stochastic block models where nodes change their community membership over time, but where edges are generated independently at each time step. In this setting (which is a special case of several e…
The paper optimizes learning Gaussian Processes in distributed systems with minimal communication.
problem Learning Gaussian Processes in distributed and communication-limited systems.
method Information-theoretic bounds and practical schemes like per-symbol quantization.
result The proposed methods outperform previous zero-rate distributed GP learning schemes.
Study community detection in multi-view data with various types of information.
problem Community detection in multi-view data with different types of information.
method Unified theoretical framework, mutual information analysis, sharp thresholds, iterative algorithms.
result Sharp thresholds for community recovery in various multi-view settings.
Survey of network analysis limits and optimal methods.
problem Graphon estimation, community detection, and hypothesis testing.
method Review of minimax optimal rates and procedures.
result Optimal algorithms for network analysis.
Unified framework detects dynamic community structure in brain networks across individuals.
problem Detecting community structure in functional brain networks across multiple subjects and over time.
method Markov-switching stochastic block model (MSS-SBM) for multilayer brain networks.
result Captures dynamic reconfiguration of modular connectivity in brain networks across different task conditions.
The stochastic block model guides community detection limits and algorithms.
problem Understanding community detection limits in stochastic block models.
method Information-theoretic and computational tradeoffs, algorithms derivation.
result Phase transitions and mutual information tradeoffs for community detection.
Single global merging boosts decentralized learning performance.
problem Limited communication in decentralized learning hinders performance.
method Scheduled communication, focusing on final step with global merging.
result Single global merging improves global test performance.
HSQ reduces communication costs in federated learning.
problem High cost of communicating gradients in federated learning.
method Hyper-sphere quantization (HSQ) framework for efficient gradient compression.
result HSQ achieves O(logd) per-iteration communication cost, significantly reducing costs without compromising accuracy. Paper studies signal detection in noisy environments with limited communication.
problem Signal detection in Gaussian noise with 1-bit communication constraints.
method Derives lower bounds and exhibits optimal testing strategies.
result Optimal distributed testing strategies attain the derived lower bound.
A new framework maximizes influence spread in social networks by accounting for inter-community diffusion.
problem Real-world social networks have inter-community influence that is often overlooked in community-based IM approaches.
method Community-IM++ uses a heuristic based on community-based diffusion degree and progressive budgeting to model and prioritize cross-community diffusion.
result Community-IM++ achieves near-greedy influence spread at up to 100 times lower runtime than existing methods.
Paper sets fundamental limits for distributed covariance estimation with constrained communication.
problem Estimating high-dimensional covariance matrices in a feature-split setting with limited communication.
method Developed a Conditional Strong Data Processing Inequality (C-SDPI) to establish minimax lower bounds and an optimal estimation protocol.
result Achieved nearly optimal estimation protocol with sample and communication requirements matching lower bounds up to logarithmic factors.
Generative model improves local community detection in networks.
problem Finding a single community in a large network using only a small part of it.
method Starting from a generative model for networks with community structure, approximating the unobserved parts to detect local communities.
result The proposed methods show comparable or improved results compared to state-of-the-art local community detection algorithms.
Sparse binary compression reduces communication costs in distributed deep learning.
problem Limited communication bandwidth in distributed deep learning.
method Combines gradient sparsification, binarization, and optimal weight update encoding.
result Reduces upstream communication by more than four orders of magnitude.
Spectral clustering achieves strong consistency in the stochastic block model under certain conditions.
problem Achieving strong consistency in spectral clustering for the stochastic block model.
method Entrywise analysis of the Fielder eigenvector of graph Laplacians.
result Spectral clustering achieves exact recovery of hidden communities under matching information-theoretic limits.
Research shows minimal communication limits adaptive function estimation rates.
problem Adaptive estimation of a smooth function under minimal communication constraints.
method Investigates the L∞-risk and L2-risk under different numbers of servers. result For L∞-risk, optimal rates cannot be achieved under minimal communication. For L2-risk, adaptivity is possible but depends on server number and sample size. Enhances Infomap algorithm to prioritize metadata in community detection.
problem Balancing metadata importance in community detection.
method Introduces a tuning parameter to the Infomap algorithm.
result Improves mutual information with metadata at the cost of structural detectability.
This paper analyzes Local SGD for federated learning, achieving both statistical and communication efficiency.
problem Statistical estimation and inference in federated learning with decentralized data.
method Local SGD, a multi-round estimation procedure using intermittent communication.
result Local SGD achieves both statistical efficiency and communication efficiency.
New uncertainty principle limits compression in distributed learning, suggesting optimal methods.
problem Minimizing communication cost while maintaining message quality in distributed learning.
method Formalized uncertainty principle for compression operators, introduced Kashin compression.
result Explicit formula for Kashin compression's variance bound, dimension independent.
Mathematical framework for cooperative communication explains belief transmission.
problem Lack of understanding why cooperation enables effective belief transmission.
method Connection to optimal transport theory, deriving prior models, statistical interpretations, proofs of robustness and instability.
result Cooperative communication provably enables effective, robust belief transmission.
Efficiently tests discrete distributions with limited memory and communication.
problem Testing discrete distributions with constraints on memory and communication.
method Developed efficient algorithms for uniformity/identity and closeness testing in streaming and distributed models.
result Nearly-tight lower bounds on sample complexity and communication cost for uniformity testing.
Event-triggered learning reduces communication in networked control systems.
problem Reduction of communication in networked control systems.
method Triggered learning experiments when communication performance is poor, using statistical properties of inter-communication times.
result Event-triggered learning improves robustness and communication efficiency.
New model improves community detection in networks with strong assortativity.
problem Classic SBMs fail to recover assortative communities in networks with reduced information.
method Introduced a constrained SBM with strong assortativity constraints and efficient algorithms.
result Significant boost in community recovery capabilities, especially close to information-theoretic threshold.
A scalable distributed S-LSR1 algorithm reduces communication costs.
problem Efficiently scaling S-LSR1 for large-scale distributed optimization.
method Proposes DS-LSR1, a communication-efficient variant of S-LSR1.
result DS-LSR1 scales well in problem dimension and data points.