A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper tightens lower bounds on decentralized training complexity.
Novel periodic momentum SGD method for decentralized training with linear speedup.
Communication is a key bottleneck in distributed training. Recently, an \emph{error-compensated} compression technology was particularly designed for the \emph{centralized} learning and receives huge successes, by showing significant advantages over state-of-the-art compression based methods in saving the communication…
Decentralized Gaussian processes for multi-agent learning.
Decentralized machine learning is a promising emerging paradigm in view of global challenges of data ownership and privacy. We consider learning of linear classification and regression models, in the setting where the training data is decentralized over many user devices, and the learning algorithm must run on-device, …
RelaySum improves decentralized deep learning by uniformly distributing data across workers.
New algorithm improves decentralized learning in the presence of Byzantine faults.
Single global merging boosts decentralized learning performance.
Low complexity decentralized neural net with centralized performance.
Many large-scale machine learning (ML) applications need to perform decentralized learning over datasets generated at different devices and locations. Such datasets pose a significant challenge to decentralized learning because their different contexts result in significant data distribution skew across devices/locatio…
New algorithm reduces communication traffic in decentralized learning.
This paper develops efficient federated learning and unlearning methods in Bayesian models.
New method for decentralized learning from diverse clients.
Optimizing distributed learning systems is an art of balancing between computation and communication. There have been two lines of research that try to deal with slower networks: {\em communication compression} for low bandwidth networks, and {\em decentralization} for high latency networks. In this paper, We explore a…
New algorithm solves min-max optimization problems in a decentralized manner.
We propose a decentralized learning algorithm over a general social network. The algorithm leaves the training data distributed on the mobile devices while utilizing a peer to peer model aggregation method. The proposed algorithm allows agents with local data to learn a shared model explaining the global training data …
Decentralized solutions to finite-sum minimization are of significant importance in many signal processing, control, and machine learning applications. In such settings, the data is distributed over a network of arbitrarily-connected nodes and raw data sharing is prohibitive often due to communication or privacy constr…
FedFaiREE addresses fairness in decentralized learning with small samples.
Moniqua improves SGD convergence with quantized communication.
DMF improves POI recommendation privacy and efficiency.
While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}. Ironically, recent analysis of decentralized parallel stochastic gradient descent (D-PS…
A novel decentralized algorithm improves minimax optimization in federated learning.
Flexible decentralized MARL framework for cooperative multi-agent learning.
A framework for decentralized optimization using first-order methods.
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
SQuARM-SGD improves decentralized SGD efficiency with momentum.
Algorithm for decentralized competition among adaptive agents.
Data parallelism has become the de facto standard for training Deep Neural Network on multiple processing units. In this work we propose DC-S3GD, a decentralized (without Parameter Server) stale-synchronous version of the Delay-Compensated Asynchronous Stochastic Gradient Descent (DC-ASGD) algorithm. In our approach, w…
This chapter deals with decentralized learning algorithms for in-network processing of graph-valued data. A generic learning problem is formulated and recast into a separable form, which is iteratively minimized using the alternating-direction method of multipliers (ADMM) so as to gain the desired degree of paralleliza…
In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we assume that each node has access to a summand of a global objective function and nodes are allowed to exchange information only with their n…
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters. As current approaches suffer from limited bandwidth of the network, we propose the use of communication compression in the decentral…
We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar object…
Novel privacy model for decentralized data analysis.
We propose an efficient protocol for decentralized training of deep neural networks from distributed data sources. The proposed protocol allows to handle different phases of model training equally well and to quickly adapt to concept drifts. This leads to a reduction of communication by an order of magnitude compared t…
It has been shown that injecting noise into the neural network weights during the training process leads to a better generalization of the resulting model. Noise injection in the distributed setup is a straightforward technique and it represents a promising approach to improve the locally trained models. We investigate…
Improves decentralized learning by optimizing graph mixing for data heterogeneity.
Decentralized methods to solve finite-sum minimization problems are important in many signal processing and machine learning tasks where the data is distributed over a network of nodes and raw data sharing is not permitted due to privacy and/or resource constraints. In this article, we review decentralized stochastic f…
Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it is still unclear how to efficiently combine local updates and decentralized communication. In this work, we propose an algorithm named as L…
Federated learning opens a number of research opportunities due to its high communication efficiency in distributed training problems within a star network. In this paper, we focus on improving the communication efficiency for fully decentralized federated learning over a graph, where the algorithm performs local updat…
Experts predict significant adoption of decentralized finance by 2034, with traditional finance adapting.
RESIST improves decentralized learning resilience against MITM attacks.
Wide and Deep GNN learns from distributed graphs and retrain online.
This study compares decentralized banks and finds some lack decentralization.
Improves decentralized learning by teleporting active nodes for better convergence.
This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network. While a denser (sparser) network topology results in faster (slower) error convergence in terms of iterations, it incurs more (less) communication …
DIGing-SGLD improves SGLD for scalable Bayesian learning in dynamic networks.
New method for decentralized learning reduces data and computation needs.
Paper tackles low sample and communication complexities in decentralized bilevel optimization.