arXiv research

CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.

Paper tackles Byzantine resilience in distributed multi-task learning.

We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience…

Novel algorithm resists Byzantine attacks in federated learning for PCA and LRCS.

Machine learning has begun to play a central role in many applications. A multitude of these applications typically also involve datasets that are distributed across multiple computing devices/machines due to either design constraints (e.g., multiagent systems) or computational/privacy reasons (e.g., learning on smartp…

Machine Learning (ML) solutions are nowadays distributed and are prone to various types of component failures, which can be encompassed in so-called Byzantine behavior. This paper introduces LiuBei, a Byzantine-resilient ML algorithm that does not trust any individual component in the network (neither workers nor serve…

Secure federated learning framework resists adversarial users.

Byrd-SAGA reduces variance to robustify SGD against Byzantine attacks.

COMRADE is a communication-efficient, Byzantine-resilient second-order optimization algorithm.

Paper develops Byzantine-resilient algorithms for decentralized learning.

Byzantine-resilient federated learning with local iterations and robust mean estimation.

Machine Learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component failures, which can be all encompassed within the spectrum of a …

Distributed machine learning algorithms enable learning of models from datasets that are distributed over a network without gathering the data at a centralized location. While efficient distributed algorithms have been developed under the assumption of faultless networks, failures that can render these algorithms nonfu…

Paper shows data poisoning and Byzantine attacks are equivalent, impacting federated learning security.

Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers. In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, corrupt data, or even malicious attacks. We introduce \emph{Karda…

Paper develops a robust federated recommendation system against poisoning attacks.

We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilie…

To improve the resilience of distributed training to worst-case, or Byzantine node failures, several recent approaches have replaced gradient averaging with robust aggregation methods. Such techniques can have high computational costs, often quadratic in the number of compute nodes, and only have limited robustness gua…

The growth of data, the need for scalability and the complexity of models used in modern machine learning calls for distributed implementations. Yet, as of today, distributed machine learning frameworks have largely ignored the possibility of arbitrary (i.e., Byzantine) failures. In this paper, we study the robustness …

Proposes a method for private aggregation in heterogeneous federated learning.

PRISM-FCP improves federated prediction robustness against Byzantine attacks.

Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through multi-Bulyan. Given $n$ workers, $f$ of which are arbitrary malicious (Byzantine) and $m=n-f$ are not, we prove that multi-Bulyan can ensure a strong form of Byzantine resilien…

In this work, we consider the resilience of distributed algorithms based on stochastic gradient descent (SGD) in distributed learning with potentially Byzantine attackers, who could send arbitrary information to the parameter server to disrupt the training process. Toward this end, we propose a new Lipschitz-inspired c…

Two algorithms improve federated learning efficiency and resilience.

New algorithm resists Byzantine attacks in distributed SGD for heterogeneous data.

While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD against adversarial (Byzantine) workers sending poisoned gradients …

This paper extends Newton's method to distributed learning, avoiding saddle points and handling Byzantine workers.

New algorithms for collaborative learning in uncertain, decentralized environments.

In this paper, we show synchronization for a group of output passive agents that communicate with each other according to an underlying communication graph to achieve a common goal. We propose a distributed event-triggered control framework that will guarantee synchronization and considerably decrease the required comm…

Novel asynchronous SGD method resists Byzantine attacks without server storage.

New framework detects and removes malicious model updates in federated learning.

While the last few decades have witnessed a huge body of work devoted to inference and learning in distributed and decentralized setups, much of this work assumes a non-adversarial setting in which individual nodes---apart from occasional statistical failures---operate as intended within the algorithmic framework. In r…

Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a parameter server (PS). To guarantee some form of robustness, recent work suggests using variants of the geometric median as an aggregation r…

Paper tackles Byzantine attacks in Federated Learning by clustering and robustifying.

Paper addresses Byzantine attacks in decentralized optimization over networks.

Paper tackles Byzantine attacks in distributed learning with a new ADMM method.

Proposes Stochastic-Sign SGD for federated learning with theoretical guarantees.

DynBRO learns robustly from dynamic Byzantine workers.

Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robu…

SignSGD improves distributed learning by tolerating faulty devices, including Byzantine adversaries.

New method improves model accuracy in Byzantine-robust distributed learning by optimizing batch size.

New algorithm improves decentralized learning in the presence of Byzantine faults.

New methods improve Byzantine robustness in distributed learning.

New attack strategy circumvents CC framework's defences in federated learning.

Paper improves statistical efficiency of median-of-means estimator for Byzantine robust distributed inference.

Robust algorithm for distributed optimization resistant to Byzantine failures.

Two novel algorithms improve distributed machine learning in the presence of Byzantine adversaries.

New federated learning protocols resist Byzantine failures and offer privacy guarantees.