Massively parallel architectures such as the GPU are becoming increasingly important due to the recent proliferation of data. In this paper, we propose a key class of hybrid parallel graphlet algorithms that leverages multiple CPUs and GPUs simultaneously for computing k-vertex induced subgraph statistics (called graph…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “GPU Computing”
GPU speeds up derivatives sensitivity computation for Heston model options.
problem Efficient computation of option Greeks under the Heston model.
method Implemented exact simulation and novel Milstein discretisation methods on GPU.
result GPU speeds up Greeks computation up to 200x compared to CPU methods.
GPU computing has become popular in computational finance and many financial institutions are moving their CPU based applications to the GPU platform. Since most Monte Carlo algorithms are embarrassingly parallel, they benefit greatly from parallel implementations, and consequently Monte Carlo has become a focal point …
Speeds up deep neural networks training by 10x using GPU concurrency.
problem Training deep residual neural networks efficiently.
method Layer-wise parallel training with GPU concurrency and Nonlinear Multigrid.
result 10.2x speedup over traditional techniques.
The graphics processing unit (GPU) has emerged as a powerful and cost effective processor for general performance computing. GPUs are capable of an order of magnitude more floating-point operations per second as compared to modern central processing units (CPUs), and thus provide a great deal of promise for computation…
We describe the multi-GPU gradient boosting algorithm implemented in the XGBoost library (https://github.com/dmlc/xgboost). Our algorithm allows fast, scalable training on multi-GPU systems with all of the features of the XGBoost library. We employ data compression techniques to minimise the usage of scarce GPU memory …
GPU speeds up Monte Carlo simulations for large time steps.
problem Slow convergence and inaccurate solutions with large time steps in Monte Carlo simulations.
method Generalizes the Seven League scheme for GPU acceleration.
result Significantly improved computational speed.
Framework optimizes cloud container sizing for ML tasks.
problem Challenges in cloud container configuration for ML services.
method Autonomous scaling using nested-loop Monte Carlo simulation.
result Reduces compute cost and accelerates ML algorithms.
The realized stochastic volatility (RSV) model that utilizes the realized volatility as additional information has been proposed to infer volatility of financial time series. We consider the Bayesian inference of the RSV model by the Hybrid Monte Carlo (HMC) algorithm. The HMC algorithm can be parallelized and thus per…
A Python package for GPU-accelerated signature kernel computation.
problem Efficient computation of signature kernels for sequential data.
method GPU-accelerated algorithms and tensor sketches.
result New algorithm outperforms existing methods.
New GPU kernels boost deep learning speed and memory efficiency.
problem Sparse deep learning matrices are not well-suited for existing sparse kernels.
method Identified favorable properties of sparse matrices from deep learning, developed high-performance GPU kernels for sparse matrix operations.
result 27% of single-precision peak performance on Nvidia V100 GPUs achieved with new kernels.
cuSLINK clusters data faster on GPUs, saving space and time.
problem Efficiently clustering large datasets on GPUs.
method Reformulated SLINK algorithm on GPU with optimized building blocks.
result cuSLINK uses space and trades off space and time with parameter .
Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and research progress but entails two complications. First, the training library must support inter-GPU communication. Depending on the particular …
One reflection suffices for orthogonal weights, reducing GPU usage.
problem Efficiently computing orthogonal weight matrices without high GPU utilization.
method Use an auxiliary neural network to compute one reflection instead of many.
result One reflection is sufficient for orthogonal weights, improving GPU utilization.
In this work, we present an extension of Gaussian process (GP) models with sophisticated parallelization and GPU acceleration. The parallelization scheme arises naturally from the modular computational structure w.r.t. datapoints in the sparse Gaussian process formulation. Additionally, the computational bottleneck is …
Mixed-precision CA-SGD for generalized linear models on GPUs
problem SGD communication bottleneck
method Mixed-precision CA-SGD
result Matches FP32 SGD loss within 0.5% on various problems
Stochastic simulation techniques employed for the analysis of portfolios of insurance/reinsurance risk, often referred to as `Aggregate Risk Analysis', can benefit from exploiting state-of-the-art high-performance computing platforms. In this paper, parallel methods to speed-up aggregate risk analysis for supporting re…
Quasi-Monte Carlo speeds up option Greeks calculation on GPUs.
problem Efficiently calculating option Greeks for risk management.
method Quasi-Monte Carlo (QMC) combined with GPU acceleration for pathwise sensitivity calculation.
result Increased computational speed and efficiency in estimating option Greeks.
RFX accelerates and compresses Random Forests for large datasets.
problem Memory bottleneck in proximity matrices limits Random Forest analysis.
method QLORA compression, CPU TriBlock storage, GPU batch sizing, 3D MDS visualization.
result Proximity-based Random Forest analysis on larger datasets is feasible.
Paper proposes a GPU-based system for training massive deep learning models in ads systems.
problem Training massive deep learning models with terabyte-scale parameters in ads systems.
method Hierarchical GPU parameter server with 3-layer storage (GPU High-Bandwidth Memory, CPU main memory, SSD).
result 4-node hierarchical GPU parameter server trains a model 2X faster than a 150-node in-memory system.
New GPU algorithm speeds up Gaussian Process analysis.
problem Reducing computational complexity for large spatial datasets.
method Implemented three GPU methods for Vecchia Approximation.
result New GPU method outperforms existing methods.
GPU-accelerated BART speeds up Bayesian regression.
problem Long running time of BART makes it impractical for large datasets.
method GPU-enabled implementation of BART.
result BART is now 200x faster on GPUs.
SparseRT accelerates sparse computations on GPUs for deep learning inference.
problem Efficiently handling unstructured sparsity patterns on GPUs for deep learning.
method SparseRT, a code generator that leverages unstructured sparsity for accelerating sparse linear algebra operations.
result Geometric mean speedups of 3.4x at 90% sparsity and 5.4x at 95% sparsity for 1x1 convolutions and fully connected layers.
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training. Previous GPU based tree building algorithms are based on parallel multi-…
pySigLib speeds up signature-based computations on CPUs and GPUs.
problem Efficient signature-based computations on large datasets and long sequences.
method Optimised Python library for CPU and GPU, novel differentiation scheme.
result Accurate gradients at a fraction of the runtime of existing libraries.
On-device inference of machine learning models for mobile phones is desirable due to its lower latency and increased privacy. Running such a compute-intensive task solely on the mobile CPU, however, can be difficult due to limited computing power, thermal constraints, and energy consumption. App developers and research…
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusterscs.LG
Deep learning models can take weeks to train on a single GPU-equipped machine, necessitating scaling out DL training to a GPU-cluster. However, current distributed DL implementations can scale poorly due to substantial parameter synchronization over the network, because the high throughput of GPUs allows more data batc…
cuRegOT accelerates GPU-based entropic OT solving.
problem Slow convergence and high computational cost of optimal transport on GPUs.
method High-performance GPU solver with algorithmic and architectural optimizations.
result Significant speedups over state-of-the-art solvers.
LoRA fine-tuning on CPUs without GPUs achieves comparable performance to GPU-based methods.
problem Limited computational resources for fine-tuning LLMs.
method Developed a CPU-efficient method to learn meta-operators for LoRA weights.
result LoRA adapters trained on CPUs outperform base models on downstream tasks.
Latent Dirichlet Allocation (LDA) is a popular tool for analyzing discrete count data such as text and images. Applications require LDA to handle both large datasets and a large number of topics. Though distributed CPU systems have been used, GPU-based systems have emerged as a promising alternative because of the high…
Polylab is a MATLAB toolbox for multivariate polynomial modeling.
problem Efficiently modeling and manipulating multivariate polynomials across CPU and GPU.
method Unified symbolic-numeric interface, three aligned classes (MPOLY, MPOLY_GPU, MPOLY_HP), polynomial operations, differentiation, matrix computations.
result Advantages of MPOLY-HP for reduction-heavy simplification and large-scale computations, and the stochastic log-determinant variant for sparse regimes.
The Long-Short-Term-Memory Recurrent Neural Networks (LSTM RNNs) are a popular class of machine learning models for analyzing sequential data. Their training on modern GPUs, however, is limited by the GPU memory capacity. Our profiling results of the LSTM RNN-based Neural Machine Translation (NMT) model reveal that fea…
GP3 framework efficiently analyzes Gaussian processes on GPUs.
problem Certifiable safety in machine learning applications.
method GP3 framework using interval analysis and multi-resolution sampling on GPUs.
result Efficient analysis of Gaussian processes with certifiable safety.
Differentiable voxelization for 3D meshes with GPU acceleration.
problem Efficient and accurate voxelization of 3D meshes.
method Differentiable voxelization using winding number and solid angles, with GPU acceleration and neural network deformation.
result State-of-the-art performance in accuracy and efficiency on the ShapeNet dataset.
Performance-aware channel pruning improves CNN on embedded GPUs.
problem Inefficient channel pruning on embedded GPUs leads to performance slowdowns.
method Evaluate higher-level libraries that analyze input characteristics for optimized code generation.
result Performance-aware pruning can achieve significant performance speedups, up to 10x.
Deep Neural Networks(DNNs) require huge GPU memory when training on modern image/video databases. Unfortunately, the GPU memory is physically finite, which limits the image resolutions and batch sizes that could be used in training for better DNN performance. Unlike solutions that require physically upgrade GPUs, the G…
Neural architecture search (NAS) has a great impact by automatically designing effective neural network architectures. However, the prohibitive computational demand of conventional NAS algorithms (e.g. GPU hours) makes it difficult to \emph{directly} search the architectures on large-scale tasks (e.g. ImageNet).…
DistShap parallelizes GNN explanation for large graphs.
problem Computational expense in attributing GNN predictions to specific edges or features.
method Distributed Shapley values across multiple GPUs for scalable GNN explanations.
result DistShap outperforms existing methods and scales to models with millions of features.
Paper explores reducing precision in SVM for faster text classification.
problem Efficiency in multi-class text classification training.
method Comparison of SVM trained with reduced precision (16-bit, half) vs original.
result Reduced precision training maintains text classification accuracy.
When deep learning models on GPU can be accelerated by taking advantage of unstructured sparsitycs.LG
This paper accelerates sparse CNN layers on GPUs by using unstructured sparsity.
problem Efficiency of sparse CNN layers on GPUs.
method Direct sparse operation and reduced precision.
result Achieving up to 90% sparsity in deep CNN models improves efficiency.
GPU-accelerates multiuser detection for 5G URLLC systems.
problem Efficiently detecting payloads in OFDM frames with ultra-low latency.
method Implemented partially linear multiuser detection in RKHSs on a GPU-accelerated platform.
result Sub-millisecond latency detection in 5G URLLC systems.
New GPU algorithm boosts machine learning with larger datasets.
problem Limited GPU memory restricts training data size.
method Out-of-core GPU gradient boosting algorithm.
result Training larger datasets on GPUs without accuracy loss.
Paper improves financial trading models using GPU parallelism.
problem Challenges in policy instability and sampling bottlenecks in reinforcement learning for financial tasks.
method Revisits ensemble methods with massively parallel simulations on GPUs.
result Significantly improved computational efficiency and robustness of financial decision-making strategies.
A new parallel algorithm speeds up Hawkes process estimation.
problem Slow maximum likelihood estimation for Hawkes processes.
method Parallel prefix scan for sparse transition matrices.
result Massive speedup with complexity.
DoRA improves adaptation efficiency for large models by factoring norms and fusing kernels.
problem High-rank DoRA is computationally expensive and infeasible on common GPUs.
method Factored norms and fused Triton kernels to reduce memory and speed up computation.
result Fused implementation is up to 2.0x faster for inference and 1.9x faster for gradient computation.
Convolutional neural networks have recently achieved significant breakthroughs in various image classification tasks. However, they are computationally expensive,which can make their feasible mplementation on embedded and low-power devices difficult. In this paper convolutional neural network binarization is implemente…
Alternatives to recurrent neural networks, in particular, architectures based on attention or convolutions, have been gaining momentum for processing input sequences. In spite of their relevance, the computational properties of these alternatives have not yet been fully explored. We study the computational power of two…
Synchronized stochastic gradient descent (SGD) optimizers with data parallelism are widely used in training large-scale deep neural networks. Although using larger mini-batch sizes can improve the system scalability by reducing the communication-to-computation ratio, it may hurt the generalization ability of the models…