Spectral clustering with edge counting detects communities in sparse models.
problem Detecting communities in sparse latent space models.
method Spectral clustering followed by edge counting.
result Algorithm achieves consistency and optimality for a broad class of models.
GMC uses global momentum for sparse communication in distributed learning.
problem Latency and bandwidth limitations in network communication for large-scale deep learning models.
method GMC utilizes global momentum for sparse communication in DMSGD, improving convergence and accuracy.
result GMC and GMC+ achieve higher test accuracy and faster convergence compared to local momentum methods.
Sparse spectral decomposition identifies overlapping communities in networks.
problem Estimating overlapping community memberships in networks where nodes can belong to multiple communities.
method Sparse principal subspace estimation with iterative thresholding.
result The fixed point of the algorithm corresponds to correct node memberships under the stochastic block model.
Unified spectral clustering for sparse networks with heterogeneous degrees.
problem Efficiently detecting communities in sparse networks with varying degrees.
method Developed a parametrized regularized Laplacian matrix for spectral clustering.
result Improved parametrization accounts for network heterogeneity and community hardness.
New algorithms reduce communication for sparse mean estimation in noisy distributed systems.
problem Sparse normal means estimation with limited communication in a distributed setting.
method Two distributed algorithms for estimating a sparse mean vector with sublinear communication.
result Correct support of the sparse mean can be recovered with significantly less communication than previously required.
ACOWA improves distributed sparse classification with extra communication round.
problem Efficiently optimizing sparse classification with limited communication.
method Introducing ACOWA, a new technique with an extra communication round.
result ACOWA achieves better approximation quality and higher accuracy.
New algorithms reduce communication in GNN training.
problem Higher communication costs in GNNs due to sparse connectivity.
method Parallel algorithms for sparse-dense matrix multiplication.
result Asymptotic reduction in communication compared to previous methods.
Spectral method detects communities in sparse hypergraphs, achieving detection threshold.
problem Community detection in sparse hypergraphs.
method Non-backtracking operator and spectral approach.
result Spectral method achieves detection threshold for sparse HSBMs.
We consider the community detection problem in sparse random hypergraphs. Angelini et al. (2015) conjectured the existence of a sharp threshold on model parameters for community detection in sparse hypergraphs generated by a hypergraph stochastic block model. We solve the positive part of the conjecture for the case of…
Applying machine learning techniques to the quickly growing data in science and industry requires highly-scalable algorithms. Large datasets are most commonly processed "data parallel" distributed across many nodes. Each node's contribution to the overall gradient is summed using a global allreduce. This allreduce is t…
Paper identifies sparse structures and communities in heterogeneous graphical models.
problem Detecting community structures in graphical models.
method Novel decomposition into sparse and low-rank parts, three-stage estimation procedure.
result Consistent model selection for adaptive ℓ1 penalized estimator. A fast spectral algorithm detects community structure in evolving graphs.
problem Detecting community structure in time-evolving sparse graphs.
method Extension of the Bethe-Hessian matrix for spectral community detection.
result The algorithm reaches the optimal detectability threshold and outperforms other methods.
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
New method for community detection in sparse directed SBMs with exact recovery guarantees.
problem Exact recovery in sparse directed SBMs, especially with growing communities.
method Two-stage procedure: neighborhood-smoothing followed by K-means clustering. result Exact recovery of all community labels with probability tending to one under mild sparsity and separation conditions.
Sparse random networks reduce communication in federated learning.
problem Large communication cost in federated learning.
method Freeze random weights, train stochastic binary mask to sparsify.
result Improves accuracy, reduces communication, speeds convergence.
In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities na and nb respectively. We consider the sparse setting, in which a and b do not scale with n, and…
Efficient algorithm reduces communication costs in sparse regression.
problem Sparse linear regression with massive data.
method CESDAR algorithm, communication-efficient surrogate likelihood.
result Achieves same statistical accuracy as global estimator with reduced communication.
Generative model captures hubs and dense communities in social networks.
problem Capturing both hubs and dense communities in social networks.
method Graphon mixture model with a new condition on sparse graphs.
result Estimation of hub normalized degree and graphon for sparse components.
Many algorithms have been proposed for fitting network models with communities, but most of them do not scale well to large networks, and often fail on sparse networks. Here we propose a new fast pseudo-likelihood method for fitting the stochastic block model for networks, as well as a variant that allows for an arbitr…
Distributed-OMP recovers sparse vectors with low communication costs.
problem High-dimensional sparse linear regression with limited computation and communication.
method Distributed orthogonal matching pursuit (OMP) scheme.
result Support of the regression vector can be recovered with linear communication per machine and logarithmic in dimension.
Paper proposes a method to improve variational inference for sparse networks.
problem Variational inference struggles with sparse networks, leading to inaccurate community detection.
method The method involves hard thresholding the posterior of community assignment after each iteration.
result The proposed method accurately recovers true community labels in sparse networks.
A new method reduces communication costs in distributed learning.
problem Reduces communication bottlenecks in distributed learning.
method Local SGD with communication-computation overlap and delay-corrected sparse model averaging.
result Theoretical convergence guarantees for smooth non-convex objectives.
Graph energy helps detect communities in networks better than traditional methods.
problem Detecting communities in sparse networks where traditional methods fail.
method Using graph energy based on the full spectrum of adjacency matrices.
result The difference in graph energy between a planted partition model and an Erdős--Rényi network has a distinct transition at the detectability threshold.
New findings support a new community recovery threshold for Stochastic Block Model with many communities.
problem Recovering communities in Stochastic Block Model with more than sqrt(n) communities.
method Counting specific motifs to achieve polynomial-time community recovery above a new threshold.
result LDP fails below the new threshold, but polynomial-time recovery is possible above it.
We propose a novel statistical model for sparse networks with overlapping community structure. The model is based on representing the graph as an exchangeable point process, and naturally generalizes existing probabilistic models with overlapping block-structure to the sparse regime. Our construction builds on vectors …
Spectral algorithms are classic approaches to clustering and community detection in networks. However, for sparse networks the standard versions of these algorithms are suboptimal, in some cases completely failing to detect communities even when other algorithms such as belief propagation can do so. Here we introduce a…
Optimized parallel algorithms for identifying strong ties in data.
problem Identifying strong ties in data with varying distances and community sizes.
method Design and analysis of sequential and parallel algorithms for partitioned local depths.
result Optimized algorithms achieve up to 19.4x speedup in parallel execution.
Recently, the decentralized optimization problem is attracting growing attention. Most existing methods are deterministic with high per-iteration cost and have a convergence rate quadratically depending on the problem condition number. Besides, the dense communication is necessary to ensure the convergence even if the …
New method quantifies network cycles to enhance community detection.
problem Challenges in detecting communities in networks, especially in sparse graphs.
method Renewal non-backtracking random walks (RNBRW) to quantify cyclic structure.
result RNBRW improves community detection algorithms, especially in sparse graphs.
We study the tradeoff between the statistical error and communication cost of distributed statistical estimation problems in high dimensions. In the distributed sparse Gaussian mean estimation problem, each of the m machines receives n data points from a d-dimensional Gaussian distribution with unknown mean θ w…
Sparse DNNs face scalability issues; MIT/IEEE/Amazon challenge analyzes best solutions.
problem Scalability issues in Sparse Deep Neural Networks (DNNs).
method Mathematically defined DNN inference computation, community submissions from various fields.
result Sparse DNN execution time, TmDNN, is strongly dependent on the number of operations, Nmop. New findings on community recovery in SBM with many communities.
problem Determining community recovery conditions in SBM with more than sqrt(n) communities.
method Constructing motifs and counting them to prove community recovery above the proposed threshold.
result Proving community recovery above the proposed threshold in SBM with K >= sqrt(n) communities.
FedDST trains sparse sub-networks to improve efficiency in federated learning.
problem Efficiently train large models on resource-limited edge devices with limited network bandwidth.
method Dynamic sparse training to reduce model size and communication.
result FedDST outperforms existing methods in federated learning, especially in non-i.i.d. settings.
Sparse covariance estimation in the vertical-split model achieves exponential improvement over dense estimates.
problem Minimax estimation error for distributed covariance matrix estimation in the vertical-split setting.
method Elementwise s-sparsity is shown to reduce communication and sample complexity. result Minimax lower bounds for 1-sparse cross-covariance estimation are established. DES training speeds up large-scale recommender systems convergence.
problem Training large-scale recommender systems with dynamic sparse features.
method Distributed Equivalent Substitution (DES) framework for fully synchronous training.
result DES achieves higher AUC and up to 68.7% communication savings.
New scheme for sparse feature selection in networked data.
problem Sparse feature selection in distributed, communication-restricted networks.
method Distributed sparse linear regression and feature selection method.
result True causal features can be reliably recovered with minimal bandwidth usage.
Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
The study extends stochastic block models to geometric settings, focusing on community detection and information flow.
problem Generalizing community detection and information flow models to geometric settings.
method Considered a geometric random graph over a homogeneous metric space, defined a geometric counterpart of flow of information on trees.
result Sufficient conditions for recovering locations and for percolation of information in geometric settings.
Paper uses SSC for identifying layers with identical community structures in DIMPLE networks.
problem Identifying layers with identical community structures in DIMPLE networks.
method Sparse Subspace Clustering (SSC) for identifying groups of layers with identical community structures.
result SSC leads to strongly consistent between-layer clustering under mild conditions.
A novel method for learning Bayesian network structures from decentralized data, balancing privacy and efficiency.
problem Privacy and communication costs in learning Bayesian network structures from decentralized data.
method Fed-Sparse-BNSL, combining differential privacy with greedy updates targeting only a few relevant edges per participant.
result Achieves utility close to non-private baselines while offering stronger privacy and communication efficiency.
New model allows sparse graphs with many triangles to be represented.
problem Sparse graphs with many triangles cannot be accurately represented in finite dimensions.
method Infinite-dimensional inner product model with manifold representations.
result Local neighborhoods can be represented in lower dimensions.
Paper optimizes Laplacian regularization for sparse network clustering.
problem Improving spectral clustering in sparse networks.
method Formally determines optimal Laplacian regularization.
result Proper regularization is closely tied to state-of-the-art techniques.
Learning sparse combinations is a frequent theme in machine learning. In this paper, we study its associated optimization problem in the distributed setting where the elements to be combined are not centrally located but spread over a network. We address the key challenges of balancing communication costs and optimizat…
New algorithms detect communities in sparse graphs with labeled data.
problem Detecting communities in sparse graphs with limited labeled data.
method Introduces two algorithms: combinatorial and optimization-based, to integrate labeled data with graph structures.
result Detection of communities is feasible throughout the parameter domain with arbitrary labeled data.
Spectral algorithm recovers community structure in sparse hypergraphs.
problem Community detection in sparse random hypergraphs with community structure and higher-order interactions.
method Spectral algorithm with three steps: hyperedge selection, spectral partition, and correction/merging.
result Weak consistency achieved for weak signal-to-noise ratio.
Additive asynchronous and cyclostationary impulsive noise limits communication performance in OFDM powerline communication (PLC) systems. Conventional OFDM receivers assume additive white Gaussian noise and hence experience degradation in communication performance in impulsive noise. Alternate designs assume a parametr…
New model captures sparse, evolving multigraph structures.
problem Understanding sparse, evolving multigraph structures in dynamic interaction data.
method Dynamic nonparametric Bayesian model combining sparsity and clustering.
result Improved held-out likelihood and predictive performance.
New algorithm reduces communication in distributed learning by sharing compressed beliefs.
problem Efficiently learning from private data in a distributed setting with large hypothesis sets.
method Proposes a belief update rule for distributed cooperative learning with compressed (sparse or quantized) beliefs.
result Beliefs converge almost surely to optimal hypotheses with a linear concentration rate.