A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
DynBRO learns robustly from dynamic Byzantine workers.
Paper tackles Byzantine attacks in distributed learning with a new ADMM method.
Machine Learning (ML) solutions are nowadays distributed and are prone to various types of component failures, which can be encompassed in so-called Byzantine behavior. This paper introduces LiuBei, a Byzantine-resilient ML algorithm that does not trust any individual component in the network (neither workers nor serve…
Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers. In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, corrupt data, or even malicious attacks. We introduce \emph{Karda…
A method to robustly federate learning with non-i.i.d. data and Byzantine workers.
We propose Zeno++, a new robust asynchronous Stochastic Gradient Descent~(SGD) procedure which tolerates Byzantine failures of the workers. In contrast to previous work, Zeno++ removes some unrealistic restrictions on worker-server communications, allowing for fully asynchronous updates from anonymous workers, arbitrar…
Machine Learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component failures, which can be all encompassed within the spectrum of a …
COMRADE is a communication-efficient, Byzantine-resilient second-order optimization algorithm.
Byrd-SAGA reduces variance to robustify SGD against Byzantine attacks.
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robu…
New bucketing scheme improves Byzantine robustness for heterogeneous data.
The growth of data, the need for scalability and the complexity of models used in modern machine learning calls for distributed implementations. Yet, as of today, distributed machine learning frameworks have largely ignored the possibility of arbitrary (i.e., Byzantine) failures. In this paper, we study the robustness …
New algorithm improves decentralized learning in the presence of Byzantine faults.
New algorithm resists Byzantine attacks in distributed SGD for heterogeneous data.
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilie…
New methods improve Byzantine robustness in distributed learning.
We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience…
Proposes a method to improve Byzantine-robustness in compressed federated learning.
Paper shows data poisoning and Byzantine attacks are equivalent, impacting federated learning security.
New research shows Byzantine failures hurt generalization more than data poisoning in robust distributed learning.
This paper extends Newton's method to distributed learning, avoiding saddle points and handling Byzantine workers.
We develop a communication-efficient distributed learning algorithm that is robust against Byzantine worker machines. We propose and analyze a distributed gradient-descent algorithm that performs a simple thresholding based on gradient norms to mitigate Byzantine failures. We show the (statistical) error-rate of our al…
Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through multi-Bulyan. Given workers, of which are arbitrary malicious (Byzantine) and are not, we prove that multi-Bulyan can ensure a strong form of Byzantine resilien…
While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD against adversarial (Byzantine) workers sending poisoned gradients …
Two novel algorithms improve distributed machine learning in the presence of Byzantine adversaries.
We consider unreliable distributed learning systems wherein the training data is kept confidential by external workers, and the learner has to interact closely with those workers to train a model. In particular, we assume that there exists a system adversary that can adaptively compromise some workers; the compromised …
We study robust distributed learning that involves minimizing a non-convex loss function with saddle points. We consider the Byzantine setting where some worker machines have abnormal or even arbitrary and adversarial behavior. In this setting, the Byzantine machines may create fake local minima near a saddle point tha…
Novel asynchronous SGD method resists Byzantine attacks without server storage.
New method improves model accuracy in Byzantine-robust distributed learning by optimizing batch size.
In this work, we consider the resilience of distributed algorithms based on stochastic gradient descent (SGD) in distributed learning with potentially Byzantine attackers, who could send arbitrary information to the parameter server to disrupt the training process. Toward this end, we propose a new Lipschitz-inspired c…
The recent advances in sensor technologies and smart devices enable the collaborative collection of a sheer volume of data from multiple information sources. As a promising tool to efficiently extract useful information from such big data, machine learning has been pushed to the forefront and seen great success in a wi…
We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different de…
Paper tackles Byzantine attacks in Federated Learning by clustering and robustifying.
Paper addresses Byzantine attacks in decentralized optimization over networks.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
Distributed learning is central for large-scale training of deep-learning models. However, they are exposed to a security threat in which Byzantine participants can interrupt or control the learning process. Previous attack models and their corresponding defenses assume that the rogue participants are (a) omniscient (k…
Paper develops Byzantine-resilient algorithms for decentralized learning.
Novel algorithm resists Byzantine attacks in federated learning for PCA and LRCS.
Distributed machine learning algorithms enable learning of models from datasets that are distributed over a network without gathering the data at a centralized location. While efficient distributed algorithms have been developed under the assumption of faultless networks, failures that can render these algorithms nonfu…
SignSGD improves distributed learning by tolerating faulty devices, including Byzantine adversaries.
New attack strategy circumvents CC framework's defences in federated learning.
PRISM-FCP improves federated prediction robustness against Byzantine attacks.
Paper improves statistical efficiency of median-of-means estimator for Byzantine robust distributed inference.
Robust algorithm for distributed optimization resistant to Byzantine failures.
Paper develops a robust federated recommendation system against poisoning attacks.
Machine learning has begun to play a central role in many applications. A multitude of these applications typically also involve datasets that are distributed across multiple computing devices/machines due to either design constraints (e.g., multiagent systems) or computational/privacy reasons (e.g., learning on smartp…
New federated learning protocols resist Byzantine failures and offer privacy guarantees.
Paper tackles Byzantine resilience in distributed multi-task learning.