arXiv research

This paper improves parallel belief propagation for scalable machine learning.

This study compares parallel SMC and MCMC for Bayesian deep learning, showing SMC parallel is faster.

Linear algebra approach for parallel deep learning models.

MixML unifies analysis of weakly consistent parallel learning.

Quantum computers can speed up machine learning optimization problems.

Enhances parallelism in decentralized learning for larger networks.

Serial problems can't be efficiently parallelized, affecting machine learning models.

Cyclic Data Parallelism reduces memory usage and balances gradient communications.

Improved voice conversion with semi-supervised learning.

We introduce a new embarrassingly parallel parameter learning algorithm for Markov random fields with untied parameters which is efficient for a large class of practical models. Our algorithm parallelizes naturally over cliques and, for graphs of bounded degree, its complexity is linear in the number of cliques. Unlike…

Randomized block-diagonal preconditioning improves parallel learning convergence.

PHS optimizes hyperparameters in parallel for expensive computations.

OptEx accelerates first-order optimization with parallelized iterations.

Proposes a method to improve SLMC for multimodal distributions.

A single policy suffices for near-optimal parallel exploration in RL.

Stochastic gradient descent~(SGD) and its variants have become more and more popular in machine learning due to their efficiency and effectiveness. To handle large-scale problems, researchers have recently proposed several parallel SGD methods for multicore systems. However, existing parallel SGD methods cannot achieve…

A parallel algorithm learns efficient Kronecker product dictionaries.

Embarrassingly (communication-free) parallel Markov chain Monte Carlo (MCMC) methods are commonly used in learning graphical models. However, MCMC cannot be directly applied in learning topic models because of the quasi-ergodicity problem caused by multimodal distribution of topics. In this paper, we develop an embarra…

We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. Howev…

A fast deep learning method for parallel MRI without calibration.

This paper describes neural-fortran, a parallel Fortran framework for neural networks and deep learning. It features a simple interface to construct feed-forward neural networks of arbitrary structure and size, several activation functions, and stochastic gradient descent as the default optimization algorithm. Neural-f…

A scalable parallel BO method for asynchronous settings.

Speeds up deep neural networks training by 10x using GPU concurrency.

Model predicts web page parallelism for improved browser performance and energy.

Parallelizes feedforward computation using nonlinear equation solving.

Study examines parallel computing strategies for faster imputation of missing data.

Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.

EP-GFlowNets parallelize GFlowNet training for large-scale Bayesian inference.

Communication costs, resulting from synchronization requirements during learning, can greatly slow down many parallel machine learning algorithms. In this paper, we present a parallel Markov chain Monte Carlo (MCMC) algorithm in which subsets of data are processed independently, with very little communication. First, w…

Parallelizes graph embedding for large graphs.

A new method combines MCMC results to avoid failures in parallel computing.

Proposes MLPSVM for multi-label learning, improving on binary relevance.

There is significant recent interest to parallelize deep learning algorithms in order to handle the enormous growth in data and model sizes. While most advances focus on model parallelization and engaging multiple computing agents via using a central parameter server, aspect of data parallelization along with decentral…

PETRA enables parallel training of deep models with reversible architectures.

End-to-end DA method for domain-invariant CNNs using parallel audio recordings.

SySCD improves SCD scalability and speeds up training.

Parallel score matching accelerates DPM training and improves density estimation.

A new trading model combines GARCH and PPO for better stock trading profits.

Training examples are not all equally informative. Active learning strategies leverage this observation in order to massively reduce the number of examples that need to be labeled. We leverage the same observation to build a generic strategy for parallelizing learning algorithms. This strategy is effective because the …

A deep-learning inference accelerator is synthesized from a C-language software program parallelized with Pthreads. The software implementation uses the well-known producer/consumer model with parallel threads interconnected by FIFO queues. The LegUp high-level synthesis (HLS) tool synthesizes threads into parallel FPG…

Deploying deep learning (DL) models across multiple compute devices to train large and complex models continues to grow in importance because of the demand for faster and more frequent training. Data parallelism (DP) is the most widely used parallelization strategy, but as the number of devices in data parallel trainin…

Clapping reduces memory usage in distributed optimization by reusing data samples.

Optimizes deep learning pipelines with novel algorithms for smooth and non-smooth functions.

New neural network can learn multiple patterns at once.

Improves parallel deep model performance by restructuring and pruning.

In AI research and industry, machine learning is the most widely used tool. One of the most important machine learning algorithms is Gradient Boosting Decision Tree, i.e. GBDT whose training process needs considerable computational resources and time. To shorten GBDT training time, many works tried to apply GBDT on Par…

Due to the need for robust uncertainty quantification, Bayesian neural learning has gained attention in the era of deep learning and big data. Markov Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which faces several challenges given a large number of parameters, complex and multimodal posterio…

Parallelizes active learning for Bayesian inference using Nested Sampler.