A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New bucketing scheme improves Byzantine robustness for heterogeneous data.
RelaySum improves decentralized deep learning by uniformly distributing data across workers.
A distributed SGD method for heterogeneous networks with hubs and workers.
Ringmaster ASGD improves Asynchronous SGD's efficiency under varying worker times.
Rescaled ASGD optimizes distributed learning under heterogeneous data.
MindFlayer SGD improves parallel SGD for heterogeneous, random compute times.
Study improves communication efficiency in RIS-assisted downlink communication.
Freya PAGE optimizes nonconvex optimization with heterogeneous, asynchronous workers.
Unified analysis of asynchronous-SGD algorithms for distributed learning.
New algorithm resists Byzantine attacks in distributed SGD for heterogeneous data.
Ringmaster LMO accelerates training in distributed systems by asynchronously updating neural networks.
A method to robustly federate learning with non-i.i.d. data and Byzantine workers.
We consider the problem faced by a service platform that needs to match limited supply with demand but also to learn the attributes of new users in order to match them better in the future. We introduce a benchmark model with heterogeneous "workers" (demand) and a limited supply of "jobs" that arrive over time. Job typ…
ATA optimizes task allocation in distributed machine learning.
Fault-tolerant federated learning for non-uniform data.
Ringleader ASGD optimizes SGD for diverse edge devices with varying data and computation speeds.
New model optimizes worker-task specialization for crowdsourcing.
Generative AI boosts productivity and improves customer service quality.
Crowdsourcing platforms emerged as popular venues for purchasing human intelligence at low cost for large volume of tasks. As many low-paid workers are prone to give noisy answers, a common practice is to add redundancy by assigning multiple workers to each task and then simply average out these answers. However, to fu…
In crowd labeling, a large amount of unlabeled data instances are outsourced to a crowd of workers. Workers will be paid for each label they provide, but the labeling requester usually has only a limited amount of the budget. Since data instances have different levels of labeling difficulty and workers have different r…
The study uses a multi-armed bandit model to analyze and mitigate hiring discrimination.
Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art parallel mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due to different computational capabilities or different communication delays), s…
New asynchronous SGD algorithms achieve optimal performance in distributed learning.
A new method reduces communication costs in distributed learning.
This work tackles resource allocation in asynchronous and stochastic systems.
Most commonly used distributed machine learning systems are either synchronous or centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a heterogeneous environment, while asynchronous algorithms using a parameter server suffer from 1) communication bottleneck at parameter servers when wo…
Model uses Preisach hysteresis to predict gig worker acceptance, reducing costs and improving fill rates.
Graph embedding is a popular algorithmic approach for creating vector representations for individual vertices in networks. Training these algorithms at scale is important for creating embeddings that can be used for classification, ranking, recommendation and other common applications in industry. While industrial syst…
Crowdsourced labeling recovers task types with minimal queries.
Paper shows data poisoning and Byzantine attacks are equivalent, impacting federated learning security.
Crowdsourcing provides a popular paradigm for data collection at scale. We study the problem of selecting subsets of workers from a given worker pool to maximize the accuracy under a budget constraint. One natural question is whether we should hire as many workers as the budget allows, or restrict on a small number of …
A new algorithm reduces training time for distributed machine learning by dynamically assigning backup workers.
Due to concerns about human error in crowdsourcing, it is standard practice to collect labels for the same data point from multiple internet workers. We here show that the resulting budget can be used more effectively with a flexible worker assignment strategy that asks fewer workers to analyze easy-to-label data and m…
Consider unsupervised clustering of objects drawn from a discrete set, through the use of human intelligence available in crowdsourcing platforms. This paper defines and studies the problem of universal clustering using responses of crowd workers, without knowledge of worker reliability or task difficulty. We model sto…
New principle reduces load imbalance in LLM serving systems, saving up to 52% energy.
We present a macroeconomic agent-based model that combines several mechanisms operating at the same timescale, while remaining mathematically tractable. It comprises enterprises and workers who compete in a job market and a commodity goods market. The model is stock-flow consistent; a bank lends money charging interest…
We propose Zeno++, a new robust asynchronous Stochastic Gradient Descent~(SGD) procedure which tolerates Byzantine failures of the workers. In contrast to previous work, Zeno++ removes some unrealistic restrictions on worker-server communications, allowing for fully asynchronous updates from anonymous workers, arbitrar…
While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}. Ironically, recent analysis of decentralized parallel stochastic gradient descent (D-PS…
We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different de…
The number of Italian firms in function of the number of workers is well approximated by an inverse power law up to 15 workers but shows a clear downward deflection beyond this point, both when using old pre-1999 data and when using recent (2014) data. This phenomenon could be associated with employent protection legis…
Deep learning is a popular machine learning technique and has been applied to many real-world problems. However, training a deep neural network is very time-consuming, especially on big data. It has become difficult for a single machine to train a large model over large datasets. A popular solution is to distribute and…
DynBRO learns robustly from dynamic Byzantine workers.
In this paper, we focus on approaches to parallelizing stochastic gradient descent (SGD) wherein data is farmed out to a set of workers, the results of which, after a number of updates, are then combined at a central master node. Although such synchronized SGD approaches parallelize well in idealized computing environm…
Recently, there is a growing interest in the study of median-based algorithms for distributed non-convex optimization. Two prominent such algorithms include signSGD with majority vote, an effective approach for communication reduction via 1-bit compression on the local gradients, and medianSGD, an algorithm recently pr…
Cost-efficient distributed learning via combinatorial bandits.
Crowdsourcing is a relatively economic and efficient solution to collect annotations from the crowd through online platforms. Answers collected from workers with different expertise may be noisy and unreliable, and the quality of annotated data needs to be further maintained. Various solutions have been attempted to ob…
We consider distributed optimization problems where forming the Hessian is computationally challenging and communication is a significant bottleneck. We develop unbiased parameter averaging methods for randomized second order optimization that employ sampling and sketching of the Hessian. Existing works do not take the…
To accelerate the training of machine learning models, distributed stochastic gradient descent (SGD) and its variants have been widely adopted, which apply multiple workers in parallel to speed up training. Among them, Local SGD has gained much attention due to its lower communication cost. Nevertheless, when the data …