Paper controls type I error in text classification despite data distortion.
problem Data distortion in open online platforms leads to misclassification.
method Uses Neyman-Pearson (NP) classification paradigm to minimize type I error.
result NP methods control type I error on test data despite data distortion.
Efficiently maximizes AUC with deep nets, reducing communication rounds.
problem Maximizing AUC with deep neural networks in a distributed setting.
method Communication-efficient distributed optimization algorithm for non-convex concave AUC maximization.
result Achieves linear speedup with significantly fewer communication rounds.
Crowdsourced labeling recovers task types with minimal queries.
problem Labeling tasks accurately with minimal queries.
method Worker clustering, skill estimation, weighted majority voting.
result Achieves any targeted recovery accuracy with minimum queries.
New model optimizes worker-task specialization for crowdsourcing.
problem Inferring correct labels from noisy answers across varying worker and task skills.
method Introduced a d d d -type specialization model to account for varying worker and task types, and proposed algorithms achieving optimal sample complexity. result Optimal label inference algorithms for crowdsourcing with unknown worker and task types.
BUOCA optimizes crowdsourcing budgets by assigning more workers to hard tasks.
problem Crowdsourcing budget inefficiency due to uniform worker assignment.
method Optimal worker allocation based on task difficulty and features, without using worker profiles.
result Large budget savings (up to 49%) with minimal accuracy loss.
Crowdsourcing provides a popular paradigm for data collection at scale. We study the problem of selecting subsets of workers from a given worker pool to maximize the accuracy under a budget constraint. One natural question is whether we should hire as many workers as the budget allows, or restrict on a small number of …
A new algorithm reduces training time for distributed machine learning by dynamically assigning backup workers.
problem Time-consuming synchronization phase due to slow workers (stragglers).
method Dynamic allocation of backup workers to minimize waiting time.
result Achieves linear speedup in convergence performance with more workers.
The paper tackles universal clustering using crowdsourcing, proving algorithms' asymptotic consistency and optimality.
problem Unsupervised clustering of objects from a discrete set using human intelligence in crowdsourcing platforms.
method Modeling stochastic worker response distributions and defining clustering algorithms for temporary and long-term workers, integrating them into a unified model.
result Proven asymptotic consistency and optimality of the defined algorithms.
Asynchronous SGD can speed up training with a trade-off of gradient staleness.
problem Asynchronous SGD suffers from gradient staleness, affecting convergence error.
method Theoretical analysis of the error-runtime trade-off considering random straggling delays.
result A method of gradually varying synchronicity in distributed SGD is proposed and demonstrated.
Proposes a new method for parallelizing SGD that combines partial results from all workers.
problem Slow workers (stragglers) cause convergence issues in synchronous SGD.
method Fixes worker computation time and combines partial results from all workers.
result Improves convergence significantly compared to existing methods.
The number of Italian firms in function of the number of workers is well approximated by an inverse power law up to 15 workers but shows a clear downward deflection beyond this point, both when using old pre-1999 data and when using recent (2014) data. This phenomenon could be associated with employent protection legis…
Paper tackles noisy annotations by considering workers' attention levels.
problem Noisy annotations from workers with varying expertise.
method Proposes a probabilistic model that incorporates workers' attention for accurate label quality estimation.
result Improves aggregated labels by quantifying the relationship between workers' attention and label quality.
Bayesian scheme optimally learns worker quality in crowdsourced regression.
problem Learning the quality of workers in crowdsourced regression tasks.
method Iterative Bayesian learning approach.
result Proves optimal mean squared error performance.
Zeno++ improves robustness of asynchronous SGD in fully asynchronous settings.
problem Byzantine failures in fully asynchronous SGD.
method Estimates descent of loss after applying candidate gradient.
result Proves convergence for non-convex problems under Byzantine failures.
Unified distributed SGD improves non-convex optimization for large datasets.
problem Bottleneck in scaling SGD for non-convex functions and large datasets.
method Distributed and parallel implementation of SGD (DPSGD) combining asynchronous distribution and lock-free parallelism.
result DPSGD achieves better convergence rate and speed-up with more cores and workers.
Secure gradient descent method for high-dimensional learning with Byzantine workers.
problem Secure training in high-dimensional statistical learning with unreliable workers.
method Proposes a secure variant of gradient descent method that can tolerate up to a constant fraction of Byzantine workers.
result Converges in O(log N) rounds to O(√(q/N) + √(d/N)) error rate, achieving optimal error rate O(√(d/N)) when q=O(d).
D 2 ^2 2 improves convergence in decentralized SGD for varied data.
problem Decentralized training's sensitivity to data variance among workers.
method D 2 ^2 2 is a variance-aware extension of D-PSGD. result D 2 ^2 2 achieves faster convergence and robustness to data variance. DynBRO learns robustly from dynamic Byzantine workers.
problem Fault-tolerant distributed learning with dynamic Byzantine workers.
method Multi-level Monte Carlo (MLMC) gradient estimation and adaptive learning rate.
result DynaBRO nearly matches static setting's convergence rate with O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) Byzantine worker changes. Generative AI boosts productivity and improves customer service quality.
problem Productivity and quality of customer support agents.
method Staggered introduction of a generative AI-based conversational assistant in customer support.
result AI increases productivity by 15% on average, with significant heterogeneity across workers.
A method to robustly federate learning with non-i.i.d. data and Byzantine workers.
problem Byzantine workers sending malicious messages in federated learning with non-i.i.d. data.
method Resampling strategy to reduce inner and outer variation, stochastic average gradient, robust geometric median aggregation.
result The method reaches a neighborhood of the optimal solution at a linear convergence rate and learning error depends on the number of Byzantine workers.
Cost-efficient distributed learning via combinatorial bandits.
problem Trade-off between error and runtime in distributed SGD.
method Combinatorial multi-armed bandit model to learn fastest workers.
result Significantly lower errors with same computational efforts and less communication.
We consider the problem faced by a service platform that needs to match limited supply with demand but also to learn the attributes of new users in order to match them better in the future. We introduce a benchmark model with heterogeneous "workers" (demand) and a limited supply of "jobs" that arrive over time. Job typ…
Elastic CoCoA improves convergence rate by dynamically adjusting worker count.
problem Achieving optimal convergence rate in distributed training.
method Dynamic adjustment of worker count based on convergence feedback.
result Accelerates training time by up to 5.96x compared to static settings.
Paper improves communication in distributed optimization, reducing worker-to-server data exchanges.
problem Efficiency in server-to-worker communication in distributed optimization.
method MARINA-P, a novel downlink compression method using correlated compressors; M3, combining MARINA-P with uplink compression.
result MARINA-P achieves provably superior server-to-worker communication complexity with increasing number of workers.
Kardam handles Byzantine workers in asynchronous SGD, ensuring convergence and performance.
problem Byzantine behavior in distributed machine learning workers.
method Two complementary components: a filtering and a dampening component.
result Kardam guarantees almost sure convergence in the presence of Byzantine workers.
Neural network predicts nonlinear safety behavior based on personality traits.
problem Predicting construction workers' unsafe behaviors based on personality traits.
method Developed a forecasting model using neural network algorithms.
result Nonlinear relationship exists between personality traits and safety behavior.
RelaySum improves decentralized deep learning by uniformly distributing data across workers.
problem Handling data heterogeneity in decentralized deep learning.
method RelaySum uses spanning trees to distribute information exactly uniformly across all workers with finite delays.
result RelaySum is independent of data heterogeneity and scales to many workers, enabling highly accurate decentralized deep learning.
New algorithm reduces communication traffic in decentralized learning.
problem Communication bottleneck in decentralized learning for low-bandwidth workers.
method Sparsification and adaptive peer selection to reduce communication traffic.
result Significant reduction in communication traffic compared to existing methods.
Crowdsourcing systems, in which numerous tasks are electronically distributed to numerous "information piece-workers", have emerged as an effective paradigm for human-powered solving of large scale problems in domains such as image classification, data entry, optical character recognition, recommendation, and proofread…
VRL-SGD reduces communication complexity in non-identical data settings.
problem Training machine learning models with non-identical data distribution.
method VRL-SGD, which eliminates gradient variance dependency and achieves linear speedup with lower communication complexity.
result VRL-SGD reduces communication complexity from $O(T^{ rac{3}{4}} N^{ rac{3}{4}})$ to $O(T^{ rac{1}{2}} N^{ rac{3}{2}})$ .
Ringmaster ASGD improves Asynchronous SGD's efficiency under varying worker times.
problem Suboptimal performance of Asynchronous SGD under heterogeneous worker computation times.
method Ringmaster ASGD, a novel Asynchronous SGD method with optimal time complexity.
result Ringmaster ASGD achieves optimal time complexity under arbitrary worker heterogeneity.
Zeno improves SGD for distributed learning with faulty nodes.
problem Fault tolerance for distributed SGD with arbitrary faulty workers.
method Suspicion-based fault-tolerance mechanism with ranking-based preference.
result Proved convergence of SGD for non-convex problems under faulty scenarios.
In crowd labeling, a large amount of unlabeled data instances are outsourced to a crowd of workers. Workers will be paid for each label they provide, but the labeling requester usually has only a limited amount of the budget. Since data instances have different levels of labeling difficulty and workers have different r…
Study estimates 163 million online freelancers globally.
problem Estimating the number of online workers globally.
method Combining data from various online labour platforms.
result Headline estimate of 163 million registered profiles.
Paper designs decision trees for minimizing misclassification in crowdsourcing.
problem Minimizing misclassification in crowdsourcing systems with unreliable workers.
method Proposes two algorithms for designing decision trees based on minimizing the probability of misclassification and entropy.
result Demonstrates improved error performance through worker assignment to different tests.
LiuBei is a resilient ML algorithm that tolerates Byzantine workers and servers without trusting any component.
problem Byzantine failures in distributed ML solutions.
method Byzantine-resilient ML algorithm that aggregates gradients and replicates parameter servers, using a filtering mechanism and scatter/gather protocol.
result LiuBei achieves Byzantine resilience to both servers and workers and guarantees convergence, with an accuracy loss of around 5% and a 24% convergence overhead.
CENs learn interpretable embeddings from images considering worker biases and visual context.
problem Learning interpretable embeddings from noisy crowd annotations.
method Context Embedding Networks (CENs) model worker biases and visual context.
result CENs produce more interpretable embeddings than existing approaches.
M 3 ^3 3 RL trains a manager to infer worker minds and assign tasks for optimal collaboration.
problem Optimal coordination among self-interested agents with diverse preferences and skills.
method Mind-aware Multi-agent Management Reinforcement Learning (M^3RL) that infers worker minds and assigns tasks.
result Effective in modeling worker minds and achieving optimal ad-hoc teaming.
The paper uses SVAR modeling to analyze how demographic changes affect the current account and economic growth.
problem The impacts of demographic changes on the current account and economic growth.
method SVAR modeling to track dynamic impacts between population growth, current account, and economic growth.
result The long-run net impact on economic growth of the domestic working population growth and demand labor for emigrants is positive.
Improves distributed SGD convergence speed with reduced computation load.
problem Mitigating stragglers in distributed SGD to speed up convergence.
method Modeling communication and computation times, adapting number of workers and computation load dynamically.
result Significantly reduces computation load while improving convergence speed.
Partial model averaging improves Federated Learning performance.
problem Periodic model averaging causes significant model discrepancy in Federated Learning.
method Proposes a partial model averaging framework that encourages local models to stay close to each other.
result Partial averaging achieves up to 2.2% higher validation accuracy than full averaging.
Study user engagement in mobile health apps for health workers in resource-poor settings.
problem Detect churn and tailor content for health workers in mobile health apps.
method Probabilistic and survival analysis of behavioral logs.
result Personalized measures of meaningful engagement can enhance health outcomes.
COMRADE is a communication-efficient, Byzantine-resilient second-order optimization algorithm.
problem Byzantine failures in distributed optimization.
method COMRADE is a communication-efficient, second-order optimization algorithm that uses a simple norm-based thresholding rule to filter out Byzantine workers.
result COMRADE achieves linear-quadratic convergence and is robust against Byzantine workers.
LocalNewton reduces communication in distributed learning.
problem Communication bottleneck in distributed optimization.
method LocalNewton is a distributed second-order algorithm with local averaging, updating models locally and communicating once every few iterations.
result LocalNewton reduces communication rounds and end-to-end running time compared to state-of-the-art algorithms.
Paper derives formulas for volatility swap strike and zero vanna implied volatility.
problem Relationship between volatility swap strike and zero vanna implied volatility.
method Applied Malliavin calculus to derive exact formulas.
result Zero vanna implied volatility is a better approximation for volatility swap strike.
Serverless runtimes boost large-scale optimization efficiency.
problem Efficiently solving large-scale optimization problems.
method Master-worker setup with AWS Lambda, parallel optimization algorithm.
result Relative speedups up to 256 workers and efficiencies above 70% up to 64 workers.
Study improves distributed linear estimation under adversarial conditions.
problem Mean estimation of a random vector with adversarial measurements and asynchrony.
method Two-timescale ℓ1-minimization algorithm with tight convergence rates.
result Unified finite-time characterization of robustness, identifiability, and statistical efficiency.
A2SGD reduces distributed SGD communication to O(1) per worker.
problem Heavy communication costs in distributed SGD for large models.
method Two-level gradient averaging to consolidate gradients to two local averages.
result Achieves O(1) communication complexity per worker, significantly reducing traffic and training time.