We optimize distributed learning algorithms to maintain linear convergence with limited communication.
problem Limited communication time affects the convergence of distributed learning algorithms.
method We design quantizers to compress algorithm information while preserving linear convergence and characterize communication time.
result We show how to co-design machine learning and communication protocols for optimal performance.
New algorithm reduces individual regret and communication costs in cooperative bandits.
problem Optimal individual and group regret in cooperative multi-agent bandits.
method Integrates a new communication policy into a learning algorithm.
result Achieves optimal individual regret and constant communication costs.
New algorithms reduce communication costs in collaborative learning.
problem Reducing communication costs in collaborative learning.
method Distributed boosting and adaptation to classification noise.
result Communication-efficient algorithms for collaborative PAC learning robust to noise.
New algorithm reduces communication traffic in decentralized learning.
problem Communication bottleneck in decentralized learning for low-bandwidth workers.
method Sparsification and adaptive peer selection to reduce communication traffic.
result Significant reduction in communication traffic compared to existing methods.
Paper studies community detection in censored hypergraphs using information theory.
problem Community detection in censored hypergraphs with missing values.
method Information-theoretic approach, polynomial-time algorithm, spectral algorithm with refinement.
result Derives information-theoretic threshold for exact recovery of community structure.
Optimized parallel algorithms for identifying strong ties in data.
problem Identifying strong ties in data with varying distances and community sizes.
method Design and analysis of sequential and parallel algorithms for partitioned local depths.
result Optimized algorithms achieve up to 19.4x speedup in parallel execution.
As research into community finding in social networks progresses, there is a need for algorithms capable of detecting overlapping community structure. Many algorithms have been proposed in recent years that are capable of assigning each node to more than a single community. The performance of these algorithms tends to …
New algorithm for multi-agent reinforcement learning with reduced communication.
problem Cooperative learning among multiple agents with limited communication.
method Randomized multi-agent actor-critic algorithm for directed graphs.
result Algorithm solves problem for strongly connected graphs with reduced communication.
Efficient algorithm reduces communication costs in sparse regression.
problem Sparse linear regression with massive data.
method CESDAR algorithm, communication-efficient surrogate likelihood.
result Achieves same statistical accuracy as global estimator with reduced communication.
Bottom-up algorithms outperform top-down in hierarchical community detection at intermediate levels.
problem Finding the optimal hierarchical community structure in networks.
method A bottom-up algorithm for hierarchical clustering of networks.
result Bottom-up algorithms achieve the information-theoretic threshold for exact recovery at intermediate levels of the hierarchy.
Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalability of distributed SGD. However, a rigorous convergence analysis and comparative study of different communication-reduction strategies rem…
This study compares community detection algorithms for delineating health service areas.
problem Delineating health service areas to improve health care services.
method Comparative analysis of community detection algorithms on hospital-patient discharge networks.
result Infomap algorithm produced the best delineation of health service areas.
The problem of community detection in networks is usually formulated as finding a single partition of the network into some "correct" number of communities. We argue that it is more interpretable and in some regimes more accurate to construct a hierarchical tree of communities instead. This can be done with a simple to…
Communities in social networks or graphs are sets of well-connected, overlapping vertices. The effectiveness of a community detection algorithm is determined by accuracy in finding the ground-truth communities and ability to scale with the size of the data. In this work, we provide three contributions. First, we show t…
The paper introduces curvature-based clustering algorithms for graph analysis.
problem Identifying densely connected substructures in graphs for community detection.
method Discrete Ricci curvatures and geometric flows to reveal community structure.
result The curvature-based approach can identify overlapping communities in graphs.
New algorithm reduces sample and communication complexities in federated Q-learning.
problem Optimal Q-function learning in federated Q-learning with limited communication.
method Introduced Fed-DVR-Q algorithm for order-optimal sample and communication complexities.
result Complete characterization of sample-communication complexity trade-off.
A fast spectral algorithm detects community structure in evolving graphs.
problem Detecting community structure in time-evolving sparse graphs.
method Extension of the Bethe-Hessian matrix for spectral community detection.
result The algorithm reaches the optimal detectability threshold and outperforms other methods.
ComEx protocol reduces communication costs in cooperative bandits.
problem Minimizing communication costs in cooperative bandits while maintaining optimal performance.
method Developed ComEx protocol to reduce communication from Θ(T) to O(logT) messages. result Achieves state-of-the-art performance with significantly reduced communication cost.
Community detection has been one of the central problems in network studies and directed network is particularly challenging due to asymmetry among its links. In this paper, we found that incorporating the direction of links reveals new perspectives on communities regarding to two different roles, source and terminal, …
Local network community detection is the task of finding a single community of nodes concentrated around few given seed nodes in a localized way. Conductance is a popular objective function used in many algorithms for local community detection. This paper studies a continuous relaxation of conductance. We show that con…
New algorithm detects community labels in networks using unlabeled data.
problem Detect community labels in networks with partially labeled data.
method Proposes an algorithm using structural similarity metrics.
result Theoretical guarantees for misclassification error.
Statistical estimates can often be improved by fusion of data from several different sources. One example is so-called ensemble methods which have been successfully applied in areas such as machine learning for classification and clustering. In this paper, we present an ensemble method to improve community detection by…
LSGD improves deep learning training efficiency by synchronizing and decentralizing SGD.
problem Asynchronous SGD's accuracy issues and synchronous SGD's communication inefficiency.
method LSGD divides nodes into subgroups with centralized communication and decentralized computation.
result LSGD achieves better accuracy and efficiency than synchronous and asynchronous SGD.
Multiplex networks, a special type of multilayer networks, are increasingly applied in many domains ranging from social media analytics to biology. A common task in these applications concerns the detection of community structures. Many existing algorithms for community detection in multiplexes attempt to detect commun…
Local AdaAlter reduces communication in SGD with adaptive learning rates.
problem Communication overhead in distributed training.
method Novel SGD variant with adaptive learning rates and reduced communication.
result Empirically reduces communication overhead by up to 30%.
Spectral clustering with edge counting detects communities in sparse models.
problem Detecting communities in sparse latent space models.
method Spectral clustering followed by edge counting.
result Algorithm achieves consistency and optimality for a broad class of models.
Community detection is a fundamental problem in network analysis which is made more challenging by overlaps between communities which often occur in practice. Here we propose a general, flexible, and interpretable generative model for overlapping communities, which can be thought of as a generalization of the degree-co…
We propose and analyze a generic method for community recovery in stochastic block models and degree corrected block models. This approach can exactly recover the hidden communities with high probability when the expected node degrees are of order logn or higher. Starting from a roughly correct community partition …
New algorithms reduce communication in GNN training.
problem Higher communication costs in GNNs due to sparse connectivity.
method Parallel algorithms for sparse-dense matrix multiplication.
result Asymptotic reduction in communication compared to previous methods.
Much of the community detection literature studies structural communities, communities defined solely by the connectivity patterns of the network. Often, networks contain additional metadata which can inform community detection such as the grade and gender of students in a high school social network. In this work, we i…
FedLion improves Federated Learning by speeding up convergence and reducing communication costs.
problem Slow convergence and high communication costs in Federated Learning.
method Integrates Lion's adaptive approach into Federated Learning framework, using signed gradients.
result FedLion outperforms existing adaptive algorithms in convergence rate and communication efficiency.
ODC protocol improves learning in asynchronous multi-agent bandits.
problem Cooperative multi-agent bandits with unknown, irregular pull times.
method ODC protocol for on-demand communication between agents.
result ODC protocol reduces communication complexity based on empirical pull times.
New algorithm minimizes regret in multi-agent bandit problem with probabilistic communication.
problem Minimizing group regret in multi-agent multi-armed bandit problem with probabilistic communication.
method Proposes a new UCB-based algorithm for decentralized multi-agent multi-armed bandit problem on d-regular graphs with probabilistic communication. result The proposed algorithm outperforms state-of-the-art algorithms in minimizing group regret.
A new algorithm speeds up deep learning training by decoupling computation and communication.
problem High communication cost limits the speedup of distributed SGD.
method CoCoD-SGD: runs computation and communication in parallel.
result Linear time speedup with respect to hardware resources.
A new algorithm reduces regret in multiplayer bandits with minimal communication.
problem Maximizing rewards in multiplayer multi-armed bandits with collisions.
method DPE (Decentralized Parsimonious Exploration) algorithm.
result Achieves the same regret as optimal centralized algorithms with less communication.
We study distributed optimization algorithms for minimizing the average of convex functions. The applications include empirical risk minimization problems in statistical machine learning where the datasets are large and have to be stored on different machines. We design a distributed stochastic variance reduced gradien…
This paper presents a novel spectral algorithm with additive clustering designed to identify overlapping communities in networks. The algorithm is based on geometric properties of the spectrum of the expected adjacency matrix in a random graph model that we call stochastic blockmodel with overlap (SBMO). An adaptive ve…
Protein interactions constitute the fundamental building block of almost every life activity. Identifying protein communities from Protein-Protein Interaction (PPI) networks is essential to understand the principles of cellular organization and explore the causes of various diseases. It is critical to integrate multipl…
DESTRESS optimizes decentralized nonconvex optimization with optimal IFO complexity and efficient communication.
problem Decentralized nonconvex finite-sum optimization in multi-agent systems.
method DESTRESS uses stochastic recursive gradient updates, gradient tracking, and careful hyper-parameter choices to achieve optimal IFO complexity with efficient communication.
result DESTRESS matches the optimal IFO complexity of centralized algorithms while maintaining communication efficiency.
Efficient algorithm for CLSBM reduces misclassification rate.
problem Reducing misclassification in community detection for CLSBM.
method Spectral-based algorithm for CLSBM, with theoretical misclassification bounds.
result Upper bound on misclassification rate of efficient algorithm.
Estimating the largest community in a mixed population via sequential sampling.
problem Identifying the largest community in a mixed population with limited sampling.
method Sequential, random sampling of individuals across multiple boxes, optimizing sampling strategy and decision rule.
result Proposed algorithms achieve optimal error probability decay rates under fixed budget constraints.
New algorithms reduce communication for sparse mean estimation in noisy distributed systems.
problem Sparse normal means estimation with limited communication in a distributed setting.
method Two distributed algorithms for estimating a sparse mean vector with sublinear communication.
result Correct support of the sparse mean can be recovered with significantly less communication than previously required.
We study the inference of a model of dynamic networks in which both communities and links keep memory of previous network states. By considering maximum likelihood inference from single snapshot observations of the network, we show that link persistence makes the inference of communities harder, decreasing the detectab…
ABCD model generates synthetic graphs for community detection with improved scalability and interpretability.
problem Synthetic graphs for community detection are limited in scalability and interpretability.
method Developed a new random graph model (ABCD) with community structure and power-law distribution.
result ABCD model solves scalability and interpretability issues of LFR model.
New method compares community detection algorithms without ground truth.
problem Comparing community detection algorithms without ground truth.
method Integrates any objective with its implicit generative model.
result More expressive methods consistently outperform others.
Moniqua improves SGD convergence with quantized communication.
problem Efficiently communicating in decentralized SGD with limited bandwidth.
method Modulo quantized communication in decentralized SGD.
result Moniqua converges at the same rate as full-precision communication with less bits.
This paper proposes a communication-efficient distributed algorithm for high-dimensional data mining.
problem Reducing communication time and rounds in distributed data mining.
method Straggler-agnostic and bandwidth-efficient distributed primal-dual algorithm.
result Guaranteed linear convergence rate for convex problems.
Efficiently addresses federated learning challenges with reduced communication and sample complexity.
problem Heterogeneity in data volumes and distributions at different clients compromises model generalization ability.
method Introduces algorithms for communication-efficient Federated Group Distributionally Robust Optimization (FGDRO).
result Communication complexity reduced to O(1/ε4) for FGDRO-CVaR and O(1/ε3) for FGDRO-KL.