Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Communication Avoiding”
This paper extends Newton's method to distributed learning, avoiding saddle points and handling Byzantine workers.
problem Avoiding saddle points in distributed non-convex optimization, especially in the presence of Byzantine workers.
method Extends cubic-regularized Newton method to distributed framework, addressing communication bottlenecks and Byzantine attacks.
result The method achieves improved iteration complexity compared to first-order methods, with a 25% improvement in experiments.
Parallelizes LARS for high-dimensional data with speedups and accuracy trade-offs.
problem Fitting linear regression models to high-dimensional data efficiently.
method Two parallel and communication avoiding versions of LARS: bLARS and Tournament-bLARS.
result Speedups up to 4x compared to LARS, with trade-offs in solution quality.
New algorithm reduces communication in distributed eigenspace estimation.
problem Efficiently estimating eigenspaces in distributed settings without excessive communication.
method Communication-efficient distributed algorithm using Procrustean alignment.
result Achieves similar error rate to centralized estimator for PCA.
Due to economic globalization, each country's economic law, including tax laws and tax treaties, has been forced to work as a single network. However, each jurisdiction (country or region) has not made its economic law under the assumption that its law functions as an element of one network, so it has brought unexpecte…
Mixed-precision CA-SGD for generalized linear models on GPUs
problem SGD communication bottleneck
method Mixed-precision CA-SGD
result Matches FP32 SGD loss within 0.5% on various problems
Descriptive vs. inferential community detection in networks: pitfalls, myths, and half-truthsphysics.soc-ph
New categorization of community detection methods to avoid pitfalls.
problem Misuse of community detection methods in practice.
method Descriptive vs. inferential approaches.
result Inferential methods provide clearer insights into network formation.
Efficient algorithm reduces communication costs in sparse regression.
problem Sparse linear regression with massive data.
method CESDAR algorithm, communication-efficient surrogate likelihood.
result Achieves same statistical accuracy as global estimator with reduced communication.
New algorithm reduces communication traffic in decentralized learning.
problem Communication bottleneck in decentralized learning for low-bandwidth workers.
method Sparsification and adaptive peer selection to reduce communication traffic.
result Significant reduction in communication traffic compared to existing methods.
A framework uses complex networks for image segmentation.
problem Over-segmentation in image segmentation.
method Initial segmentation, adaptive network construction, community detection.
result The proposed framework improves segmentation performance.
Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes)…
Proposes COLA, a communication-efficient algorithm for decentralized optimization.
problem Decentralized consensus optimization over a network.
method Linearization and communication-censoring strategy to reduce computation and communication costs.
result Proven convergence and established convergence rates for COLA.
This paper tackles energy-efficient machine learning on low-power devices.
problem Energy consumption in machine learning due to data communication.
method Dynamic averaging for integer exponential families on low-power processors.
result Achieves comparable model quality with significantly less communication and energy.
Our recent study of a nation-wide production network uncovered a community structure, namely how firms are connected by supplier-customer links into tightly-knit groups with high density in intra-groups and with lower connectivity in inter-groups. Here we propose a method to visualize the community structure by a graph…
CSE-FSL reduces communication and storage costs in federated learning.
problem High communication and storage costs in federated learning.
method CSE-FSL uses an auxiliary network to locally update client models and sends only selected epochs' smashed data.
result Significant communication reduction with state-of-the-art convergence and model accuracy.
Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and research progress but entails two complications. First, the training library must support inter-GPU communication. Depending on the particular …
Self-supervised learning of visual semantics in image games.
problem Learning visual semantics in referential emergent language games.
method Investigating the impact of feature extractor weights and tasks on visual semantics, using various image augmentations and additional tasks.
result Communication systems can learn visual semantics in a self-supervised manner by playing the right types of games.
The deep learning trend has recently impacted a variety of fields, including communication systems, where various approaches have explored the application of neural networks in place of traditional designs. Neural networks flexibly allow for data/simulation-driven optimization, but are often employed as black boxes det…
We introduce the nonparametric metadata dependent relational (NMDR) model, a Bayesian nonparametric stochastic block model for network data. The NMDR allows the entities associated with each node to have mixed membership in an unbounded collection of latent communities. Learned regression models allow these memberships…
A novel method for learning Bayesian network structures from decentralized data, balancing privacy and efficiency.
problem Privacy and communication costs in learning Bayesian network structures from decentralized data.
method Fed-Sparse-BNSL, combining differential privacy with greedy updates targeting only a few relevant edges per participant.
result Achieves utility close to non-private baselines while offering stronger privacy and communication efficiency.
Paper proposes a DRL-based controller for networked AP systems that reduces communication frequency.
problem Reduce communication frequency in networked AP systems while maintaining control performance.
method Develops a DRL-based controller that avoids explicit update timing learning, using a semi-Markov decision process (SMDP).
result Improves communication efficiency without sacrificing control performance.
New findings support a new community recovery threshold for Stochastic Block Model with many communities.
problem Recovering communities in Stochastic Block Model with more than sqrt(n) communities.
method Counting specific motifs to achieve polynomial-time community recovery above a new threshold.
result LDP fails below the new threshold, but polynomial-time recovery is possible above it.
We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD). Our goal is to find an efficient parallelization strategy for a fixed batch size using process…
We consider a communication scenario, in which an intruder tries to determine the modulation scheme of the intercepted signal. Our aim is to minimize the accuracy of the intruder, while guaranteeing that the intended receiver can still recover the underlying message with the highest reliability. This is achieved by per…
New methods improve distributed optimization on non-iid data.
problem Communication bottleneck in distributed machine learning models.
method Two types of distributed gradient compression methods (D-QSGD and D-EF-SGD) analyzed for non-iid data.
result D-EF-SGD performs better than D-QSGD on non-iid data but can still slow down with high data skewness.
Peer review is the foundation of scientific publication, and the task of reviewing has long been seen as a cornerstone of professional service. However, the massive growth in the field of machine learning has put this community benefit under stress, threatening both the sustainability of an effective review process and…
We focus on the commonly used synchronous Gradient Descent paradigm for large-scale distributed learning, for which there has been a growing interest to develop efficient and robust gradient aggregation strategies that overcome two key system bottlenecks: communication bandwidth and stragglers' delays. In particular, R…
New method for community detection in sparse directed SBMs with exact recovery guarantees.
problem Exact recovery in sparse directed SBMs, especially with growing communities.
method Two-stage procedure: neighborhood-smoothing followed by -means clustering.
result Exact recovery of all community labels with probability tending to one under mild sparsity and separation conditions.
Community moderation drifts towards majority, study finds.
problem How to ensure crowd-sourced moderation systems trust and reward accurate evaluations.
method Consensus-based auditing with a two-stage algorithm that weights contributors by the stability of their past residuals.
result Minority contributors' evaluations drift towards the majority, and their participation share falls on controversial topics.
Paper combines scalable BMF algorithms for web-scale datasets.
problem High computational cost of Bayesian Matrix Factorization.
method Combines Posterior Propagation and asynchronous distributed implementation.
result Substantial improvements in scalability on web-scale datasets.
Graph transformers outperform graph convolutions by preserving community information.
problem Understanding why graph transformers perform well in node-level prediction tasks.
method Analyzing the Gaussian process limits of graph transformers with infinite width and infinite heads.
result Graph transformers maintain discriminative node representations even in deep layers, preventing oversmoothing.
We introduce an online tensor decomposition based approach for two latent variable modeling problems namely, (1) community detection, in which we learn the latent communities that the social actors in social networks belong to, and (2) topic modeling, in which we infer hidden topics of text articles. We consider decomp…
We propose a decentralized Maximum Likelihood solution for estimating the stochastic renewable power generation and demand in single bus Direct Current (DC) MicroGrids (MGs), with high penetration of droop controlled power electronic converters. The solution relies on the fact that the primary control parameters are se…
We consider the fully decentralized machine learning scenario where many users with personal datasets collaborate to learn models through local peer-to-peer exchanges, without a central coordinator. We propose to train personalized models that leverage a collaboration graph describing the relationships between user per…
Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classificationstat.ML
This article guides data scientists on avoiding discrimination in machine learning.
problem Machine learning systems can create or exacerbate societal disparities.
method Provides a taxonomy of practices and measures to mitigate discrimination.
result Data scientists should be intentional about modeling and reducing discriminatory outcomes.
Learning-based link scheduling improves network performance in millimeter-wave multi-connectivity.
problem Efficient link scheduling is crucial for maximizing network performance in millimeter-wave multi-connectivity.
method A learning-based approach to predict optimal link scheduling.
result The learning-based solution outperforms base line methods and approaches the optimal solution.
Analysis of spectral clustering algorithms for community detection: the general bipartite settingmath.ST
We consider spectral clustering algorithms for community detection under a general bipartite stochastic block model (SBM). A modern spectral clustering algorithm consists of three steps: (1) regularization of an appropriate adjacency or Laplacian matrix (2) a form of spectral truncation and (3) a k-means type algorithm…
NAS best practices guide reduces evaluation issues.
problem Lack of scientific evaluation quality in NAS.
method Described NAS best practices and checklist.
result Reduces evaluation issues in NAS.
Autonomous driving is getting a lot of attention in the last decade and will be the hot topic at least until the first successful certification of a car with Level 5 autonomy. There are many public datasets in the academic community. However, they are far away from what a robust industrial production system needs. Ther…
Flexible inference model for multilayer networks with heterogeneous data.
problem Complexity of heterogeneous data in networked datasets.
method Probabilistic generative model using Bayesian framework and Laplace matching.
result Effective detection of overlapping community structures and prediction tasks.
Paper proposes a new strategy to improve initial performance of federated models.
problem Weight divergence in Federated Averaging (FedAvg) leads to poor initial performance in federated models.
method Local continual training with importance weights evaluated on a proxy dataset.
result The method significantly improves the initial performance of federated models with minimal extra communication costs.
The performance and efficiency of distributed machine learning (ML) depends significantly on how long it takes for nodes to exchange state changes. Overly-aggressive attempts to reduce communication often sacrifice final model accuracy and necessitate additional ML techniques to compensate for this loss, limiting their…
Nonnegative matrix factorization (NMF) has become a workhorse for signal and data analytics, triggered by its model parsimony and interpretability. Perhaps a bit surprisingly, the understanding to its model identifiability---the major reason behind the interpretability in many applications such as topic mining and hype…
Paper formalizes continual learning, proposing a feature extraction approach.
problem Learning from multiple environments without forgetting previous ones.
method Feature extraction framework and gradient-based algorithm DPGD.
result Efficient algorithm DPGD avoids catastrophic forgetting.
Decentralized detection avoids sharing data, controls false discoveries.
problem Global false discovery rate control in decentralized novelty detection.
method Quantized surrogate models for low-precision sharing, preserving exchangeability.
result Quantized composite scores maintain competitive statistical power with reduced communication.
A new method avoids overfitting in network reconstruction by using the minimum description length principle.
problem Determining the optimal model complexity in network reconstruction to prevent overfitting.
method Hierarchical Bayesian inference and weight quantization based on the minimum description length principle.
result The method yields increased accuracy in reconstructing both artificial and empirical networks.
FedSGM tackles constrained federated learning with unified framework.
problem Functional constraints, communication bottlenecks, local updates, and partial client participation in federated learning.
method Unified framework based on switching gradient method, incorporating bi-directional error feedback, and soft switching for stability.
result Achieves convergence rate with high-probability bounds decoupling from sampling noise.
PRISM-FCP improves federated prediction robustness against Byzantine attacks.
problem Byzantine attacks in federated learning.
method Partial model sharing and distance-based maliciousness scores.
result Maintains nominal coverage guarantees under Byzantine attacks.