arXiv research

Randomized block-diagonal preconditioning improves parallel learning convergence.

Proposes a new PBO method with theoretical guarantees.

MindFlayer SGD improves parallel SGD for heterogeneous, random compute times.

We introduce a new embarrassingly parallel parameter learning algorithm for Markov random fields with untied parameters which is efficient for a large class of practical models. Our algorithm parallelizes naturally over cliques and, for graphs of bounded degree, its complexity is linear in the number of cliques. Unlike…

New research shows parallel optimization is ineffective for convex problems.

RandNet-Parareal uses neural networks to speed up time-parallel PDE solving.

The paper tackles scalable simulation of discrete random variables.

We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple parallel processors…

This work proposes an efficient autoregressive model for text generation.

Motivated by a sampling problem basic to computational statistical inference, we develop a nearly optimal algorithm for a fundamental problem in spectral graph theory and numerical analysis. Given an $n\times n$ SDDM matrix ${\bf \mathbf{M}}$, and a constant $-1 \leq p \leq 1$, our algorithm gives efficient access to a…

Async-parallel method solves convex problems with nonseparable linear constraints.

Efficient algorithm approximates discrete random variables with minimal Kolmogorov distance.

Unified framework for randomized exploration in cooperative MARL.

New methods improve prediction performance and reduce computation time in boosting and random forest models.

New algorithm improves inference for flexible models with infinite latent features.

Parallelizes computation of expected values in binomial trees for financial option pricing.

New sampling method improves efficiency for diffusion models.

We introduce a stochastic model for noisy vector fields on manifolds.

Optimizes deep learning pipelines with novel algorithms for smooth and non-smooth functions.

PDTS accelerates chemical space exploration using parallel and distributed Thompson sampling.

Efficiently quantifies uncertainty in DeepONets for function spaces.

Study examines parallel computing strategies for faster imputation of missing data.

GPU computing has become popular in computational finance and many financial institutions are moving their CPU based applications to the GPU platform. Since most Monte Carlo algorithms are embarrassingly parallel, they benefit greatly from parallel implementations, and consequently Monte Carlo has become a focal point …

New algorithm for parallel inference without block partitioning.

Introduces intrinsic Riemannian cross-covariance for manifold-valued random objects.

TBRF improves large-scale regression with two-stage random forest.

In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex function. The theoretical speedup, as compared to the serial method, and referring…

The modern scale of data has brought new challenges to Bayesian inference. In particular, conventional MCMC algorithms are computationally very expensive for large data sets. A promising approach to solve this problem is embarrassingly parallel MCMC (EP-MCMC), which first partitions the data into multiple subsets and r…

EP-GFlowNets parallelize GFlowNet training for large-scale Bayesian inference.

Machine learning identifies chimera states in complex dynamical systems.

Efficient random binning features improve kernel methods for large datasets.

Big Data is one of the major challenges of statistical science and has numerous consequences from algorithmic and theoretical viewpoints. Big Data always involve massive data but they also often include online data and data heterogeneity. Recently some statistical methods have been adapted to process Big Data, like lin…

New method for parallel Gibbs sampling without graph coloring.

In this era of large-scale data, distributed systems built on top of clusters of commodity hardware provide cheap and reliable storage and scalable processing of massive data. Here, we review recent work on developing and implementing randomized matrix algorithms in large-scale parallel and distributed environments. Ra…

Enhanced SMC$^2$ uses gradients from CRN-PF in Langevin proposals for improved state and parameter estimation.

Parallel optimization limits are tight for non-smooth convex functions.

Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, s…

A parallel optimization method for convex functions using Hessian sketching and debiasing.

Geodesic random walks in Riemannian manifolds analyzed for large deviations.

New reshaping method improves hyperparameter search over random search.

New method grows many trees in parallel and selects a compact subset for optimization.

Label ranking aims to learn a mapping from instances to rankings over a finite number of predefined labels. Random forest is a powerful and one of the most successful general-purpose machine learning algorithms of modern times. In this paper, we present a powerful random forest label ranking method which uses random de…

In order to fully utilize "big data", it is often required to use "big models". Such models tend to grow with the complexity and size of the training data, and do not make strong parametric assumptions upfront on the nature of the underlying statistical dependencies. Kernel methods fit this need well, as they constitut…

A new algorithm speeds up solving complex spin system problems.

We describe an embarrassingly parallel, anytime Monte Carlo method for likelihood-free models. The algorithm starts with the view that the stochasticity of the pseudo-samples generated by the simulator can be controlled externally by a vector of random numbers u, in such a way that the outcome, knowing u, is determinis…

We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities t…

The paper introduces a multilevel initialization method for deep neural networks.

Study explores geometric structure and prior for beta-logistic distribution.