Magnetic particle imaging (MPI) data is commonly reconstructed using a system matrix acquired in a time-consuming calibration measurement. The calibration approach has the important advantage over model-based reconstruction that it takes the complex particle physics as well as system imperfections into account. This be…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,878 papers · 148 categories
Trend · papers per month
27 results for “MPI”
We present a lightweight Python framework for distributed training of neural networks on multiple GPUs or CPUs. The framework is built on the popular Keras machine learning library. The Message Passing Interface (MPI) protocol is used to coordinate the training process, and the system is well suited for job submission …
AITuning uses reinforcement learning to optimize communication libraries.
problem Optimizing communication libraries for better performance in parallel applications.
method Deep reinforcement learning applied to tuning communication libraries.
result Demonstrates the effectiveness of reinforcement learning for tuning communication libraries.
Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors and , for the given input matrix , such that . NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, …
This paper considers a cross-layer adaptive modulation system that is modeled as a Markov decision process (MDP). We study how to utilize the monotonicity of the optimal transmission policy to relieve the computational complexity of dynamic programming (DP). In this system, a scheduler controls the bit rate of the m-qu…
Gradient Codingstat.ML
We propose a novel coding theoretic framework for mitigating stragglers in distributed learning. We show how carefully replicating data blocks and coding across gradients can provide tolerance to failures and stragglers for Synchronous Gradient Descent. We implement our schemes in python (using MPI) to run on Amazon EC…
The paper optimizes dynamic scheduling for ring architectures in deep learning training.
problem Optimizing deep learning training times with ring architectures.
method Formulated a non-convex, non-linear, NP-hard integer programming problem and developed a doubling heuristic.
result Dynamic scheduling can significantly reduce job completion times in ring architectures.
Limited annotated data available for the recognition of facial expression and action units embarrasses the training of deep networks, which can learn disentangled invariant features. However, a linear model with just several parameters normally is not demanding in terms of training data. In this paper, we propose an el…
Paper proposes a GPU-based system for training massive deep learning models in ads systems.
problem Training massive deep learning models with terabyte-scale parameters in ads systems.
method Hierarchical GPU parameter server with 3-layer storage (GPU High-Bandwidth Memory, CPU main memory, SSD).
result 4-node hierarchical GPU parameter server trains a model 2X faster than a 150-node in-memory system.
Using Premia and Nsp for Constructing a Risk Management Benchmark for Testing Parallel Architecturecs.CE
Financial institutions have massive computations to carry out overnight which are very demanding in terms of the consumed CPU. The challenge is to price many different products on a cluster-like architecture. We have used the Premia software to valuate the financial derivatives. In this work, we explain how Premia can …
New method speeds up nuclear-norm constrained learning over multiple machines.
problem Synchronization slowdown and high communication costs in large-scale learning.
method Asynchronous Stochastic Frank-Wolfe (SFW-asyn) method.
result SFW-asyn achieves the same convergence rate as vanilla SFW but with speed-ups almost linear to the number of machines.
Stochastic variational inference (SVI) employs stochastic optimization to scale up Bayesian computation to massive data. Since SVI is at its core a stochastic gradient-based algorithm, horizontal parallelism can be harnessed to allow larger scale inference. We propose a lock-free parallel implementation for SVI which a…
Applying machine learning techniques to the quickly growing data in science and industry requires highly-scalable algorithms. Large datasets are most commonly processed "data parallel" distributed across many nodes. Each node's contribution to the overall gradient is summed using a global allreduce. This allreduce is t…
ParaMonte: A high-performance serial/parallel Monte Carlo simulation library for C, C++, Fortrancs.MS
ParaMonte simplifies Monte Carlo simulations for various scientific fields.
problem Efficiently performing Monte Carlo simulations for complex models.
method Unified, high-performance, parallelized library for C, C++, Fortran.
result Automates and streamlines Monte Carlo sampling for arbitrary-dimensional functions.
Combines soft greediness with Modified Policy Iteration for more efficient deep reinforcement learning.
problem Improving sample efficiency in deep reinforcement learning.
method Combines soft greediness with Modified Policy Iteration (MPI) for off-policy learning.
result The proposed algorithm is more sample efficient than original PPO and competitive with SAC.
Linear algebra approach for parallel deep learning models.
problem Training large DNNs in distributed environments.
method Linear algebraic approach to model parallelism.
result Manual development of backward operators for gradient-based training.
ParaMonte::Python streamlines Bayesian data analysis with fast Monte Carlo and MCMC routines.
problem Efficiently sampling posterior distributions in Bayesian modeling and data science.
method Serial and MPI-parallelized Markov Chain Monte Carlo (MCMC) routines.
result Automated model calibration and uncertainty quantification in Bayesian analysis.
Etalumis bridges scientific simulators and probabilistic programming.
problem Infeasibility of rewriting scientific simulators for Bayesian inference.
method Cross-platform probabilistic execution protocol, MCMC and IC engines, distributed training of 3DCNN-LSTM.
result Achieved largest-scale posterior inference in a Turing-complete PPL for LHC use-case.
Enhanced SMC uses gradients from CRN-PF in Langevin proposals for improved state and parameter estimation.
problem Challenges in high-dimensional parameter spaces for SMC.
method Leveraging gradients from a CRN-PF within a Langevin proposal.
result Higher effective sample size and more accurate parameter estimates.
Paper analyzes convergence of entropy-regularized reinforcement learning methods.
problem Ensuring sub-optimality control in entropy-regularized RL.
method Unified analysis of regularized MPI and VI schemes, providing convergence rates.
result Established sufficient conditions for convergence and provided explicit rates.
Study compares three performance metrics of Bangladeshi banks.
problem Comparing different performance metrics of commercial banks.
method Empirical framework using MPI, ROA, TSR over 2011-2015.
result Productivity growth was recorded, but profitability and stock performance were negative.
Enhanced 3D shape analysis using information geometry.
problem Challenges in comparing 3D point clouds due to their unstructured nature and complex geometry.
method Information geometric framework for 3D point cloud shape analysis using Gaussian Mixture Models (GMMs) on a statistical manifold. Proposed MSKL divergence with upper and lower bounds.
result MSKL provides stable and monotonically varying values that directly reflect geometric variation, outperforming traditional distances and existing KL approximations.
Communication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimizationcs.DC
HybridSGD improves SGD performance by balancing computation and communication.
problem Limited scalability and performance of SGD due to communication costs.
method 2D parallel SGD method (HybridSGD) that trades off between 1D -step SGD and 1D Federated SGD (FedAvg).
result HybridSGD achieves better convergence than FedAvg at similar processor scales and up to 121x speedup over FedAvg.
JAMPI improves matrix multiplication in Spark, boosting performance by up to 24%.
problem Efficiently performing matrix multiplication in Spark.
method Combining asynchronous network IO, auto-vectorization, and barrier execution mode.
result Up to 24% performance increase in distributed matrix multiplication.
SKYNET: an efficient and robust neural network training tool for machine learning in astronomyastro-ph.IM
We present the first public release of our generic neural network training algorithm, called SkyNet. This efficient and robust machine learning tool is able to train large and deep feed-forward neural networks, including autoencoders, for use in a wide range of supervised and unsupervised learning applications, such as…
New DCD and BDCD methods for K-SVM and K-RR reduce communication costs.
problem Scalability issues in DCD and BDCD methods for kernel problems.
method Developed scalable -step variants of DCD and BDCD methods.
result Achieved strong scaling speedups of up to 9.8x over existing methods.
Many powerful machine learning models are based on the composition of multiple processing layers, such as deep nets, which gives rise to nonconvex objective functions. A general, recent approach to optimise such "nested" functions is the method of auxiliary coordinates (MAC). MAC introduces an auxiliary coordinate for …