arXiv research

Lapse improves parameter servers by dynamically allocating parameters, achieving near-linear scaling.

Asynch-SGBDT speeds up GBDT training on Parameter Server.

New robust aggregation rules for SGD in the presence of Byzantine failures.

Paper proposes a GPU-based system for training massive deep learning models in ads systems.

Stanza separates convolutional and fully connected layers for faster deep learning training.

This work provides bounds on generalization error and privacy leakage in federated learning.

New algorithm solves distributed ML with no trusted central server.

LiuBei is a resilient ML algorithm that tolerates Byzantine workers and servers without trusting any component.

DFedAvgM is a decentralized FedAvg with momentum for privacy and communication efficiency.

DSCOVR improves distributed optimization for big data with less communication and synchronization.

Paper explores differential privacy in high-dimensional federated learning, tackling server trustworthiness and estimation.

Mix2FLD improves FL accuracy with FD, reducing convergence time.

AD-PSGD is an asynchronous decentralized parallel SGD that converges as fast as AllReduce-SGD but is much faster in a heterogeneous environment.

This paper presents an asynchronous incremental aggregated gradient algorithm and its implementation in a parameter server framework for solving regularized optimization problems. The algorithm can handle both general convex (possibly non-smooth) regularizers and general convex constraints. When the empirical data loss…

A novel decentralized deep learning algorithm using gradient-based optimization.

Corella protects client data privacy in multi-server learning with correlated queries.

This work improves communication efficiency in federated learning over wireless networks by optimizing energy consumption.

Coded Federated Learning speeds up model convergence by preemptively computing on parity data.

Phocas improves robustness in SGD against Byzantine failures.

This paper makes two contributions to Bayesian machine learning algorithms. Firstly, we propose stochastic natural gradient expectation propagation (SNEP), a novel alternative to expectation propagation (EP), a popular variational inference algorithm. SNEP is a black box variational algorithm, in that it does not requi…

Enhanced federated learning reduces communication costs and improves model accuracy.

MQGrad uses reinforcement learning to dynamically adjust gradient quantization bits.

More frequent model updates in FL increase generalization error.

This paper speeds up large-scale deep learning training.

Develops a new deep learning framework for privacy-preserving text representations.

Deep learning (DL) training-as-a-service (TaaS) is an important emerging industrial workload. The unique challenge of TaaS is that it must satisfy a wide range of customers who have no experience and resources to tune DL hyper-parameters, and meticulous tuning for each user's dataset is prohibitively expensive. Therefo…

FURL improves model accuracy in FL by locally training user embeddings.

Paper develops a robust federated recommendation system against poisoning attacks.

DBCL defends collaborative learning by sketching parameters to prevent gradient-based privacy inference attacks.

Federated learning is viewed as a hierarchical latent variable model for new algorithm development.

Hippo optimizes deep learning hyper-parameters by reducing redundant trials.

A new federated learning algorithm improves on existing methods by exploiting data smoothness.

DSSP improves deep learning training speed by dynamically adjusting staleness thresholds.

A new scheme for private computation splits client data into shares for server operations.

Pipe-SGD improves deep net training time by 5.4x with pipelined AllReduce.

FedPAQ improves federated learning efficiency by averaging and quantizing updates.

Asynchronous distributed stochastic gradient descent methods have trouble converging because of stale gradients. A gradient update sent to a parameter server by a client is stale if the parameters used to calculate that gradient have since been updated on the server. Approaches have been proposed to circumvent this pro…

In distributed ML applications, shared parameters are usually replicated among computing nodes to minimize network overhead. Therefore, proper consistency model must be carefully chosen to ensure algorithm's correctness and provide high throughput. Existing consistency models used in general-purpose databases and moder…

New algorithm reduces communication costs in distributed deep learning.

New algorithm for federated learning without a central server.

Paper speeds up DGD by allowing multiple transmissions from non-straggling servers.

As Machine Learning (ML) applications increase in data size and model complexity, practitioners turn to distributed clusters to satisfy the increased computational and memory demands. Unfortunately, effective use of clusters for ML requires considerable expertise in writing distributed code, while highly-abstracted fra…

DaringFed incentivizes clients in OFL with dynamic rewards under TII.

New theory helps MCMC algorithms work well in parallel systems.

Algorithm stabilizes queues in asymmetric systems with unknown service rates.

We consider the problem of distributed statistical machine learning in adversarial settings, where some unknown and time-varying subset of working machines may be compromised and behave arbitrarily to prevent an accurate model from being learned. This setting captures the potential adversarial attacks faced by Federate…

Federated learning improves by training central model with client model outputs.

New method for semi-supervised learning in federated learning with and without labels at clients.