arXiv research

This paper improves parallel belief propagation for scalable machine learning.

This study compares parallel SMC and MCMC for Bayesian deep learning, showing SMC parallel is faster.

Linear algebra approach for parallel deep learning models.

MixML unifies analysis of weakly consistent parallel learning.

Enhances parallelism in decentralized learning for larger networks.

Serial problems can't be efficiently parallelized, affecting machine learning models.

Cyclic Data Parallelism reduces memory usage and balances gradient communications.

We introduce a new embarrassingly parallel parameter learning algorithm for Markov random fields with untied parameters which is efficient for a large class of practical models. Our algorithm parallelizes naturally over cliques and, for graphs of bounded degree, its complexity is linear in the number of cliques. Unlike…

Classical optimization algorithms in machine learning often take a long time to compute when applied to a multi-dimensional problem and require a huge amount of CPU and GPU resource. Quantum parallelism has a potential to speed up machine learning algorithms. We describe a generic mathematical model to leverage quantum…

Randomized block-diagonal preconditioning improves parallel learning convergence.

PHS optimizes hyperparameters in parallel for expensive computations.

OptEx accelerates first-order optimization with parallelized iterations.

Proposes a method to improve SLMC for multimodal distributions.

A single policy suffices for near-optimal parallel exploration in RL.

Stochastic gradient descent~(SGD) and its variants have become more and more popular in machine learning due to their efficiency and effectiveness. To handle large-scale problems, researchers have recently proposed several parallel SGD methods for multicore systems. However, existing parallel SGD methods cannot achieve…

A parallel algorithm learns efficient Kronecker product dictionaries.

Embarrassingly (communication-free) parallel Markov chain Monte Carlo (MCMC) methods are commonly used in learning graphical models. However, MCMC cannot be directly applied in learning topic models because of the quasi-ergodicity problem caused by multimodal distribution of topics. In this paper, we develop an embarra…

We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. Howev…

This paper describes neural-fortran, a parallel Fortran framework for neural networks and deep learning. It features a simple interface to construct feed-forward neural networks of arbitrary structure and size, several activation functions, and stochastic gradient descent as the default optimization algorithm. Neural-f…

A scalable parallel BO method for asynchronous settings.

Speeds up deep neural networks training by 10x using GPU concurrency.

Study examines parallel computing strategies for faster imputation of missing data.

Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.

Communication costs, resulting from synchronization requirements during learning, can greatly slow down many parallel machine learning algorithms. In this paper, we present a parallel Markov chain Monte Carlo (MCMC) algorithm in which subsets of data are processed independently, with very little communication. First, w…

EP-GFlowNets parallelize GFlowNet training for large-scale Bayesian inference.

Proposes MLPSVM for multi-label learning, improving on binary relevance.

A new method combines MCMC results to avoid failures in parallel computing.

There is significant recent interest to parallelize deep learning algorithms in order to handle the enormous growth in data and model sizes. While most advances focus on model parallelization and engaging multiple computing agents via using a central parameter server, aspect of data parallelization along with decentral…

PETRA enables parallel training of deep models with reversible architectures.

Parallel score matching accelerates DPM training and improves density estimation.

A new trading model combines GARCH and PPO for better stock trading profits.

Training examples are not all equally informative. Active learning strategies leverage this observation in order to massively reduce the number of examples that need to be labeled. We leverage the same observation to build a generic strategy for parallelizing learning algorithms. This strategy is effective because the …

A deep-learning inference accelerator is synthesized from a C-language software program parallelized with Pthreads. The software implementation uses the well-known producer/consumer model with parallel threads interconnected by FIFO queues. The LegUp high-level synthesis (HLS) tool synthesizes threads into parallel FPG…

Deploying deep learning (DL) models across multiple compute devices to train large and complex models continues to grow in importance because of the demand for faster and more frequent training. Data parallelism (DP) is the most widely used parallelization strategy, but as the number of devices in data parallel trainin…

Clapping reduces memory usage in distributed optimization by reusing data samples.

New neural network can learn multiple patterns at once.

Improves parallel deep model performance by restructuring and pruning.

In AI research and industry, machine learning is the most widely used tool. One of the most important machine learning algorithms is Gradient Boosting Decision Tree, i.e. GBDT whose training process needs considerable computational resources and time. To shorten GBDT training time, many works tried to apply GBDT on Par…

Due to the need for robust uncertainty quantification, Bayesian neural learning has gained attention in the era of deep learning and big data. Markov Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which faces several challenges given a large number of parameters, complex and multimodal posterio…

Modern learning models are characterized by large hyperparameter spaces and long training times. These properties, coupled with the rise of parallel computing and the growing demand to productionize machine learning workloads, motivate the need to develop mature hyperparameter optimization functionality in distributed …

Parallelizes active learning for Bayesian inference using Nested Sampler.

Mango automates hyperparameter tuning for large-scale ML training.

MindFlayer SGD improves parallel SGD for heterogeneous, random compute times.

Paper improves financial trading models using GPU parallelism.

Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering. Such an organization significantly constricts the types of shared structure that can be learned. The necessity of parallel ordering for deep MTL is first tested by comparing it with permuted ordering of shared la…

In real world industrial applications of topic modeling, the ability to capture gigantic conceptual space by learning an ultra-high dimensional topical representation, i.e., the so-called "big model", is becoming the next desideratum after enthusiasms on "big data", especially for fine-grained downstream tasks such as …

Neural networks compress uninformative input directions, improving test error.

AgEBO-Tabular combines NAS and hyperparameter tuning for fast, high-performing tabular models.