arXiv research

Study error bounds in evaluating distributional computational graphs.

The last decades have seen a surge of interests in distributed computing thanks to advances in clustered computing and big data technology. Existing distributed algorithms typically assume {\it all the data are already in one place}, and divide the data and conquer on multiple machines. However, it is increasingly ofte…

To scale Gaussian processes (GPs) to large data sets we introduce the robust Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts model for large-scale distributed GP regression. Unlike state-of-the-art sparse GP approximations, the rBCM is conceptually simple and does not rely on inducing or …

As inductive inference and machine learning methods in computer science see continued success, researchers are aiming to describe ever more complex probabilistic models and inference algorithms. It is natural to ask whether there is a universal computational procedure for probabilistic inference. We investigate the com…

ATA optimizes task allocation in distributed machine learning.

Improves distributed SGD convergence speed with reduced computation load.

We study the problem of identifying a probability distribution for some given randomly sampled data in the limit, in the context of algorithmic learning theory as proposed recently by Vinanyi and Chater. We show that there exists a computable partial learner for the computable probability measures, while by Bienvenu, M…

D-Wave computers struggle with sampling Boltzmann distributions efficiently.

LOT embeds distributions for linear separability and classification.

New framework for distributed nonparametric estimation under slow communication.

The recent decades have seen a surge of interests in distributed computing. Existing work focus primarily on either distributed computing platforms, data query tools, or, algorithms to divide big data and conquer at individual machines etc. It is, however, increasingly often that the data of interest are inherently dis…

A method to compute divergences between decomposable models, useful in supervised learning.

Four new methods for computing generalized chi-square distribution.

Probabilistic inference in graphical models is the task of computing marginal and conditional densities of interest from a factorized representation of a joint probability distribution. Inference algorithms such as variable elimination and belief propagation take advantage of constraints embedded in this factorization …

Fiber simplifies RL and population-based methods for distributed training.

This paper reviews techniques for distributed learning with non-convex models.

Distributed computing offers a high degree of flexibility to accommodate modern learning constraints and the ever increasing size of datasets involved in massive data issues. Drawing inspiration from the theory of distributed computation models developed in the context of gradient-type optimization algorithms, we prese…

Efficiently transforms Gaussian data to simulate various target distributions.

New method accelerates Parallel Tempering using neural samplers.

Machine learning techniques improve Bayesian computation for complex data.

Coded computation techniques provide robustness against straggling servers in distributed computing, with the following limitations: First, they increase decoding complexity. Second, they ignore computations carried out by straggling servers; and they are typically designed to recover the full gradient, and thus, canno…

Develops unbiased averaging methods for second order optimization in distributed systems.

Concrete distribution properties examined on simplex.

A computational technique for calculating nullity vectors and kernel vectors, using the new Finsler package, is introduced. As an application, three interesting counterexamples are given. The first counterexample shows that the two distributions $\mathrm{Ker}_R$ and $\N_R$ do not coincide. The second shows that the nul…

This paper reviews methods for distributed training of machine learning models from high-rate streams.

We show that the flatness of a nonlinear discrete-time system can be checked by computing a unique sequence of involutive distributions. The well-known test for static feedback linearizability is included as a special case. Since the computation of the sequence of distributions requires only the solution of algebraic e…

New method learns distribution shifts caused by predictive models in social computing.

New method for decentralized learning reduces data and computation needs.

We propose a framework for distributed robust statistical learning on {\em big contaminated data}. The Distributed Robust Learning (DRL) framework can reduce the computational time of traditional robust learning methods by several orders of magnitude. We analyze the robustness property of DRL, showing that DRL not only…

The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on two basic abstractions, it offers flexible building blocks for probabilistic computation. Distributions provide fast, numerically stable metho…

Researchers compute gap distributions for saddle connection directions on specific translation surfaces.

As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine learning and predictive modeling, distributed optimization methods have recently garne…

Dealing with the shear size and complexity of today's massive data sets requires computational platforms that can analyze data in a parallelized and distributed fashion. A major bottleneck that arises in such modern distributed computing environments is that some of the worker nodes may run slow. These nodes a.k.a.~str…

We consider the problem of selecting non-zero entries of a matrix $A$ in order to produce a sparse sketch of it, $B$, that minimizes $\|A-B\|_2$. For large $m \times n$ matrices, such that $n \gg m$ (for example, representing $n$ observations over $m$ attributes) we give sampling distributions that exhibit four importa…

For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method. At every iteration, each worker locally finds an Approximate NewTon (ANT) direction, which is sent to the main driver. The main driver, then, …

Deep learning can achieve outstanding results in various fields. However, it requires so significant computational power that graphics processing units (GPUs) and/or numerous computers are often required for the practical application. We have developed a new distributed calculation framework called "Sashimi" that allow…

As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine learning and predictive modeling, distributed optimization methods have recently garne…

New algorithm reduces communication in distributed eigenspace estimation.

Bayesian matrix factorization (BMF) is a powerful tool for producing low-rank representations of matrices and for predicting missing values and providing confidence intervals. Scaling up the posterior inference for massive-scale matrices is challenging and requires distributing both data and computation over many worke…

Coded Federated Learning speeds up training in edge computing networks.

Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions have been extensively studied, basic questions such as: Do max-entropy distributio…

We study the distribution of resonances for geometrically finite hyperbolic surfaces of infinite area by countting resonances numerically. The resonances are computed as zeros of the Selberg zeta function, using an algorithm for computation of the zeta function for Schottky groups. Our particular focus is on three aspe…

A new Wasserstein distance method for comparing incomparable distributions.

Distributed Quantum Gaussian Processes improve modeling in multi-agent systems.

Variational Prediction simplifies Bayesian inference without test time costs.

Develops a method to efficiently compute Wasserstein barycenters with variational distributions.

Paper develops fast method for computing optimal transport.

We extend the Fourier cosine method to discrete probability distributions, achieving faster convergence rates.