We describe the multi-GPU gradient boosting algorithm implemented in the XGBoost library (https://github.com/dmlc/xgboost). Our algorithm allows fast, scalable training on multi-GPU systems with all of the features of the XGBoost library. We employ data compression techniques to minimise the usage of scarce GPU memory …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “GPU scalability”
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training. Previous GPU based tree building algorithms are based on parallel multi-…
Synchronized stochastic gradient descent (SGD) optimizers with data parallelism are widely used in training large-scale deep neural networks. Although using larger mini-batch sizes can improve the system scalability by reducing the communication-to-computation ratio, it may hurt the generalization ability of the models…
DistShap parallelizes GNN explanation for large graphs.
problem Computational expense in attributing GNN predictions to specific edges or features.
method Distributed Shapley values across multiple GPUs for scalable GNN explanations.
result DistShap outperforms existing methods and scales to models with millions of features.
Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware. We present an efficient and general approach to GP inference based on Blackbox Matrix-Matrix multiplication (BBMM). BBMM inference uses a modified batched versio…
A Python package for GPU-accelerated signature kernel computation.
problem Efficient computation of signature kernels for sequential data.
method GPU-accelerated algorithms and tensor sketches.
result New algorithm outperforms existing methods.
FIRAL is a scalable active learning algorithm for multiclass classification.
problem Scalability issues with FIRAL in large datasets.
method Proposed an approximate algorithm with reduced storage and computational complexity.
result Demonstrated strong scalability and accuracy on large datasets.
Exact GPs trained on over a million points in under 2 hours.
problem Computational limitations of exact Gaussian processes for large datasets.
method Multi-GPU parallelization and linear conjugate gradients for kernel matrix multiplication.
result Exact GPs can be trained on over a million points in less than 2 hours.
New software package for scalable DPMM inference on large datasets.
problem Scalability and practical adoption of Dirichlet Process Mixture Models.
method Efficient distributed sampling-based inference on CPUs and GPUs.
result Significant speedups and fitting of larger datasets.
A new model decomposes market variability into interpretable components.
problem Understanding the factors driving market variability and predicting future movements.
method H-SGDLM framework with HAR-RV model for GPU-scalable multivariate volatility estimation.
result Superior performance in predicting large moves and longer-term market variability.
GraphGP: Scalable Gaussian Processes with Vecchia's Approximation
problem Naive Gaussian Process computation limits practical use
method GPU algorithm for Vecchia's approximation
result Linear time and memory requirements for nearly a billion parameters
GPUPoly verifies large neural networks robustly on GPUs.
problem Proving robustness of deep neural networks is crucial but challenging.
method Custom polyhedra algorithms on GPUs.
result GPUPoly can verify 1M neuron networks in 34.5 ms.
Block based Singular Value Decomposition approach to matrix factorization for recommender systemscs.LG
This paper improves SVD for recommender systems using block-based matrix factorization.
problem Scalability and performance issues in recommender systems.
method Block-based Singular Value Decomposition (BMF) for matrix factorization.
result BMF paired with SVD enhances performance and scalability.
Paper proposes a GPU-based system for training massive deep learning models in ads systems.
problem Training massive deep learning models with terabyte-scale parameters in ads systems.
method Hierarchical GPU parameter server with 3-layer storage (GPU High-Bandwidth Memory, CPU main memory, SSD).
result 4-node hierarchical GPU parameter server trains a model 2X faster than a 150-node in-memory system.
PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAXcs.LG
PGMax automates PGM inference on GPUs, improving quality and speed.
problem Efficient inference in complex discrete PGMs.
method Factor graph specification and loopy belief propagation in JAX.
result Higher-quality inference with up to 3x speedups.
FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Financeq-fin.CP
FinRL-Podracer accelerates DRL trading strategies in finance with high performance and scalability.
problem Challenges in applying deep reinforcement learning to finance trading models.
method Proposes an RLOps framework and high-performance cloud solution for DRL trading.
result FinRL-Podracer outperforms existing DRL libraries by 12-35% in annual return, 0.1-0.6 in Sharpe ratio, and 3-7 times in training time.
torchsom simplifies SOMs in PyTorch with GPU acceleration and scikit-learn API.
problem Efficient implementation and usability of SOMs in PyTorch.
method PyTorch backend, GPU acceleration, scikit-learn API, 90% test coverage.
result Ease of use and scalability for SOMs in PyTorch.
New theory maps neural network weights to optimize faster and scale.
problem Optimizing neural networks for speed and scalability.
method Constructing a duality map using layer-wise operator norms.
result Derived GPU-friendly algorithms for various layers.
Nested Slice Sampling accelerates Nested Sampling for GPU acceleration.
problem Challenging inference for complex, multimodal targets.
method Vectorized Nested Slice Sampling using Hit-and-Run Slice Sampling.
result NSS maintains accurate evidence estimates and high-quality posterior samples, robust on multimodal problems.
High-speed detection of emergent market clustering via an unsupervised parallel genetic algorithmq-fin.CP
We implement a master-slave parallel genetic algorithm (PGA) with a bespoke log-likelihood fitness function to identify emergent clusters within price evolutions. We use graphics processing units (GPUs) to implement a PGA and visualise the results using disjoint minimal spanning trees (MSTs). We demonstrate that our GP…
BoXHED2.0 boosts survival analysis for complex data.
problem Survival analysis with time-dependent covariates.
method Tree-boosted hazard estimator, fully nonparametric, scalable.
result Scalable to parametric boosted survival models in speed.
BLISS detects and separates astronomical sources quickly and accurately.
problem Detecting and separating overlapping astronomical sources in large images.
method Bayesian Light Source Separator (BLISS) using deep generative models and variational inference.
result BLISS can process megapixel images in seconds and produce highly accurate catalogs.
New methods speed up fitting for large datasets with noisy observations.
problem Fitting large datasets with Gaussian noise and known covariance.
method Two minibatch variants of extreme deconvolution, online EM algorithm, and gradient-based optimisation.
result Methods can scale to larger models and fit larger datasets faster.
Unified framework for integrating linear constraints in time series forecasting.
problem Challenges in traditional time series forecasting algorithms.
method Unified framework combining linear constraints in time series forecasting.
result Exact minimizer of the constrained empirical risk can be computed efficiently using linear algebra.
It is time-consuming and error-prone to implement inference procedures for each new probabilistic model. Probabilistic programming addresses this problem by allowing a user to specify the model and having a compiler automatically generate an inference procedure for it. For this approach to be practical, it is important…
New GPU-based algorithm for fast optimal transport on brain tractograms.
problem Efficiently comparing and transferring labels in brain tractograms.
method Multiscale algorithm using Sinkhorn divergences on GPU.
result Smooth assignments for label transfer in tractograms.
STACI uses neural nets to estimate spatio-temporal fields with valid uncertainty quantification.
problem Scalable spatio-temporal deep learning models fail to capture underlying correlation structure.
method Variational Bayesian neural network approximation of non-stationary spatio-temporal Gaussian Process (GP) with conformal inference.
result STACI provides accurate prediction intervals for spatio-temporal processes, outperforming competing methods.
New neural KB representation speeds up reasoning with large symbolic knowledge bases.
problem Efficiently reasoning with large symbolic knowledge bases.
method Sparse-matrix reified knowledge base, enabling fully differentiable, scalable neural modules.
result Competitive performance on KB completion and semantic parsing benchmarks.
aweSOM accelerates SOM clustering for large datasets.
problem Scalability issues in existing SOM implementations for large, multidimensional data.
method CPU/GPU-accelerated Self-organizing Maps (SOM) with ensemble stacking.
result 10-100x speed up and improved memory efficiency for large datasets.
BlosSOM improves data visualization for large datasets.
problem Insufficient performance of dimensionality reduction methods for large datasets.
method GPU-accelerated semi-supervised EmbedSOM algorithm.
result Produces high-quality visualizations with user control.
Paper speeds up large foundation models for time series data.
problem Resource-intensive foundation models limit accessibility.
method Dimensionality reduction techniques, including PCA and neural network adapters.
result Up to 10x speedup and 4.5x more datasets fit on a single GPU.
SpinSVAR estimates SVAR models with sparse input, improving accuracy and scalability.
problem Estimating SVAR models with sparse input assumptions.
method SpinSVAR models input as independent Laplacian variables, enforcing sparsity and using least absolute error regression.
result SpinSVAR outperforms state-of-the-art methods in accuracy and runtime, identifying significant structural shocks.
Motivation: In this paper we present the latest release of EBIC, a next-generation biclustering algorithm for mining genetic data. The major contribution of this paper is adding support for big data, making it possible to efficiently run large genomic data mining analyses. Additional enhancements include integration wi…
74.7 seconds to train ResNet-50 on ImageNet with 2,048 GPUs.
problem Training deep learning models on large mini-batches efficiently.
method Optimization methods applied to distributed deep learning.
result Top-1 validation accuracy of 75.08% with training time of 74.7 seconds.
In recent years, analyzing task-based fMRI (tfMRI) data has become an essential tool for understanding brain function and networks. However, due to the sheer size of tfMRI data, its intrinsic complex structure, and lack of ground truth of underlying neural activities, modeling tfMRI data is hard and challenging. Previo…
New framework quantifies uncertainty in flexible density-based clustering.
problem Uncertainty quantification in clustering with non-parametric density estimation.
method Martingale posterior distributions and density-based clustering.
result Efficient GPU-compatible inference on clustering structures with uncertainty.
LDTA expands LDA's topic modeling capacity with tree-structured priors.
problem Limited expressiveness of Dirichlet priors in LDA for complex topic relationships.
method Introduces Latent Dirichlet-Tree Allocation (LDTA) with Dirichlet-Tree (DT) priors, and develops universal mean-field variational inference and Expectation Propagation.
result LDTA enables expressive, tree-structured priors over topic proportions, expanding modeling capacity of LDA.
New method improves Kalman filtering and smoothing for large state spaces.
problem High computational cost and uncertainty in large-scale Kalman filtering.
method Probabilistic numerical method leveraging GPU acceleration and tunable trade-off.
result Mitigates scaling issues and provides more accurate uncertainty estimates.
Integrating Multi-Armed Bandit, Active Learning, and Distributed Computing for Scalable Optimizationstat.CO
Unified framework for scalable black-box optimization.
problem Expensive black-box evaluations in scientific and engineering domains.
method Integrates active learning, multi-armed bandits, and distributed computing.
result Consistently outperforms state-of-the-art black-box optimizers.
BSA-TNP improves NP scalability and accuracy for spatiotemporal data.
problem Scalability and accuracy trade-off in Neural Processes.
method Introduces KRBlocks, group-invariant attention biases, and BSA for scalable spatiotemporal inference.
result BSA-TNP matches or exceeds accuracy of best models while training faster.
Novel framework identifies pump-specific deterioration rates using Bayesian hierarchical hazard modeling and causal discovery.
problem Challenges in asset management due to heterogeneous deterioration rates in pump equipment.
method Bayesian hierarchical hazard modeling with causal discovery, GPU-accelerated No-U-Turn Sampling (NUTS), and DirectLiNGAM.
result Identified striking heterogeneity in deterioration rates, with negative effects 400 times larger than positive effects.
Physics-informed GANs model subsurface flow at the Hanford Site.
problem Uncertainty quantification for subsurface flow modeling at the Hanford Site.
method Physics-informed GANs, hierarchical domain parallelism, multiple GPUs, efficient communication.
result Highly scalable physics-informed GANs model subsurface flow on Summit supercomputer.
Gossip-based actor-learner architectures improve deep reinforcement learning efficiency and scalability.
problem Stabilizing and scaling deep reinforcement learning through multi-simulator training.
method Gossip-based communication among actor-learners in a peer-to-peer topology, reducing synchronization.
result Gossip-based architectures outperform A2C in terms of robustness and sample efficiency.
Scalable kernel methods for large datasets using Fourier representations and NUFFT.
problem Cubic complexity in kernel methods limits their use on large-scale datasets.
method Fourier representation of kernels combined with NUFFT for O(n log n) complexity.
result Achieves minimax convergence rates and processes up to tens of billions of samples.
New GPU algorithm boosts machine learning with larger datasets.
problem Limited GPU memory restricts training data size.
method Out-of-core GPU gradient boosting algorithm.
result Training larger datasets on GPUs without accuracy loss.
Massively parallel architectures such as the GPU are becoming increasingly important due to the recent proliferation of data. In this paper, we propose a key class of hybrid parallel graphlet algorithms that leverages multiple CPUs and GPUs simultaneously for computing k-vertex induced subgraph statistics (called graph…
SketchBoost accelerates GBDT for multioutput problems up to 40x.
problem Efficiently training GBDT for multioutput problems with high-dimensional outputs.
method Approximate computation of scoring function for faster decision tree splitting.
result SketchBoost speeds up GBDT training by up to 40 times.
The Earth Mover's Distance (EMD) is a state-of-the art metric for comparing discrete probability distributions, but its high distinguishability comes at a high cost in computational complexity. Even though linear-complexity approximation algorithms have been proposed to improve its scalability, these algorithms are eit…