arXiv research

New Fourier transform method handles missing data and asynchronous observations.

Ringleader ASGD optimizes SGD for diverse edge devices with varying data and computation speeds.

Asynchronous method for hyperparameter and neural architecture search.

FedBuff improves federated learning scalability with asynchronous updates.

Asynchronous framework speeds up model-based RL to real-time.

Unified analysis of asynchronous-SGD algorithms for distributed learning.

Asynchronous federated modeling improves spatial data sharing without centralizing raw data.

Asynchronous federated learning for vertically partitioned data improves efficiency and privacy.

Asynchronous SGD generalizes well with enough data, improving stability and reducing error.

New asynchronous SGD algorithms achieve optimal performance in distributed learning.

PARyOpt is a python based implementation of the Bayesian optimization routine designed for remote and asynchronous function evaluations. Bayesian optimization is especially attractive for computational optimization due to its low cost function footprint as well as the ability to account for uncertainties in data. A key…

Secure asynchronous federated learning with differential privacy for edge intelligence.

New algorithm reduces distributed optimization time with stochastic delays.

The paper improves Q-learning by incorporating pessimism for better sample efficiency.

LSAM optimizes deep learning training with improved efficiency.

Triadic-OCD detects changes in data streams robustly and optimally, even in asynchronous settings.

A new estimator for asynchronous tick data shows better correlation estimates.

Ringmaster ASGD improves Asynchronous SGD's efficiency under varying worker times.

Background: Recent developments have made it possible to accelerate neural networks training significantly using large batch sizes and data parallelism. Training in an asynchronous fashion, where delay occurs, can make training even more scalable. However, asynchronous training has its pitfalls, mainly a degradation in…

New method calibrates asynchronous, error-prone covariates for longitudinal data.

Asynchronous algorithms reduce privacy costs in distributed machine learning.

A new asynchronous method for vertical federated learning improves privacy and efficiency.

Freya PAGE optimizes nonconvex optimization with heterogeneous, asynchronous workers.

AEGiS optimizes expensive function evaluations asynchronously.

Secure aggregation for buffered asynchronous federated learning without TEEs.

Standard acquisition functions are sufficient for asynchronous Bayesian optimization.

Paper analyzes convergence and speedup of asynchronous parallel SGD.

Batch Bayesian optimisation (BO) has been successfully applied to hyperparameter tuning using parallel computing, but it is wasteful of resources: workers that complete jobs ahead of others are left idle. We address this problem by developing an approach, Penalising Locally for Asynchronous Bayesian Optimisation on $k$…

Most commonly used distributed machine learning systems are either synchronous or centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a heterogeneous environment, while asynchronous algorithms using a parameter server suffer from 1) communication bottleneck at parameter servers when wo…

This work tackles resource allocation in asynchronous and stochastic systems.

Advances in asynchronous optimization methods for machine learning.

As datasets continue to increase in size and multi-core computer architectures are developed, asynchronous parallel optimization algorithms become more and more essential to the field of Machine Learning. Unfortunately, conducting the theoretical analysis asynchronous methods is difficult, notably due to the introducti…

New algorithm reduces regret in asynchronous multiplayer bandits to constant or logarithmic levels.

Sparsification improves convergence in asynchronous distributed SGD, even in the presence of staleness.

We address the problem of speech act recognition (SAR) in asynchronous conversations (forums, emails). Unlike synchronous conversations (e.g., meetings, phone), asynchronous domains lack large labeled datasets to train an effective SAR model. In this paper, we propose methods to effectively leverage abundant unlabeled …

The asymptotic pseudo-trajectory approach to stochastic approximation of Benaim, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of a…

We show that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal (even to constant factors) convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard stochastic gradient procedures. Roughly, the n…

Distributed asynchronous SGD has become widely used for deep learning in large-scale systems, but remains notorious for its instability when increasing the number of workers. In this work, we study the dynamics of distributed asynchronous SGD under the lens of Lagrangian mechanics. Using this description, we introduce …

Enhances Fourier estimator performance for asynchronous event-data.

This paper describes Plumbing for Optimization with Asynchronous Parallelism (POAP) and the Python Surrogate Optimization Toolbox (pySOT). POAP is an event-driven framework for building and combining asynchronous optimization strategies, designed for global optimization of expensive functions where concurrent function …

New model stabilizes asynchronous LTI systems, independent of synchronous stability.

Paper proves CLTs for Q-learning with asynchronous updates.

Accelerates optimization in asynchronous systems with sparse updates.

Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episo…

Improves asynchronous federated learning with queuing dynamics.

We study optimization algorithms based on variance reduction for stochastic gradient descent (SGD). Remarkable recent progress has been made in this direction through development of algorithms like SAG, SVRG, SAGA. These algorithms have been shown to outperform SGD, both theoretically and empirically. However, asynchro…

The analysis of the intraday dynamics of correlations among high-frequency returns is challenging due to the presence of asynchronous trading and market microstructure noise. Both effects may lead to significant data reduction and may severely underestimate correlations if traditional methods for low-frequency data are…

Combines multi-fidelity and asynchronous batch methods for faster experimental design.