This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in W 2 W_2 W 2 distance for log-strongly-concave targets. Nested Slice Sampling accelerates Nested Sampling for GPU acceleration.
problem Challenging inference for complex, multimodal targets.
method Vectorized Nested Slice Sampling using Hit-and-Run Slice Sampling.
result NSS maintains accurate evidence estimates and high-quality posterior samples, robust on multimodal problems.
We study first-order optimization methods obtained by discretizing ordinary differential equations (ODEs) corresponding to Nesterov's accelerated gradient methods (NAGs) and Polyak's heavy-ball method. We consider three discretization schemes: an explicit Euler scheme, an implicit Euler scheme, and a symplectic scheme.…
SUOD accelerates OD for large, diverse models.
problem Training and scoring new samples with many unsupervised, heterogeneous OD models.
method Data reduction, model approximation, and taskload optimization.
result SUOD accelerates OD for over 20 benchmark datasets and a real-world case.
Study accelerates optimization methods in non-convex problems, but doesn't improve the algorithm's performance.
problem Understanding the behavior of momentum-based acceleration methods in non-convex, high-dimensional landscapes.
method Used dynamical mean field theory to describe the average dynamics of heavy-ball momentum and Nesterov acceleration in a non-convex model.
result Accelerated dynamics but did not improve the algorithm's performance with respect to gradient descent.
VegasFlow accelerates complex simulations across various hardware platforms.
problem Complex calculations and simulations requiring high-dimensional integrals.
method Monte Carlo integration techniques using Vegas algorithm and TensorFlow.
result Significantly faster performance on various hardware platforms.
Accelerates MMLE using SVGD with Nesterov acceleration.
problem Maximum Marginal Likelihood Estimation optimization.
method Stein variational gradient descent with Nesterov acceleration.
result Consistently accelerates convergence across various tasks.
This paper introduces an acceleration structure for hyperbolic embeddings.
problem Efficiently embedding and visualizing high-dimensional data in hyperbolic spaces.
method Building upon a polar quadtree, the paper introduces a new acceleration structure for hyperbolic embeddings.
result The new method computes embeddings in significantly less time compared to existing methods.
Surrogate models speed up RL training in dynamic systems.
problem High computational cost of high-fidelity simulations.
method Developed and tested surrogate models for RL training.
result Surrogate models can significantly accelerate RL training.
Purpose: To introduce a combined machine learning (ML) and physics-based image reconstruction framework that enables navigator-free, highly accelerated multishot echo planar imaging (msEPI), and demonstrate its application in high-resolution structural and diffusion imaging. Methods: Singleshot EPI is an efficient enco…
Recently algorithms incorporating second order curvature information have become popular in training neural networks. The Nesterov's Accelerated Quasi-Newton (NAQ) method has shown to effectively accelerate the BFGS quasi-Newton method by incorporating the momentum term and Nesterov's accelerated gradient vector. A sto…
Gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distinguish between two fundamentally different algorithms---Nesterov's accelerated gradient method for strongly convex functions (NAG-SC) and Po…
GPU acceleration speeds up financial machine learning training time.
problem Time-intensive classifier training in financial machine learning.
method Deployed NVIDIA GPUs for parallel high-speed arithmetic operations.
result Significantly faster training time achieved.
Robot learns to juggle two balls from 56 minutes of experience.
problem Learning high-acceleration tasks in the real-world with binary rewards.
method Designs a learning system that incorporates safety and sample efficiency.
result High-speed manipulator learns to juggle for up to 33 minutes.
A deep-learning inference accelerator is synthesized from a C-language software program parallelized with Pthreads. The software implementation uses the well-known producer/consumer model with parallel threads interconnected by FIFO queues. The LegUp high-level synthesis (HLS) tool synthesizes threads into parallel FPG…
Unpaired deep learning reconstructs MRI images from accelerated data.
problem Difficulty in acquiring matched fully sampled k-space data for supervised deep learning.
method Optimal transport driven cycleGAN architecture.
result Reconstructs high resolution MR images from accelerated k-space data.
FMCIT accelerates CI tests for causal discovery, maintaining power and efficiency.
problem High computational complexity in CI tests limits practical applicability of causal discovery methods.
method Flow Matching-based Conditional Independence Test (FMCIT) that leverages flow matching for fast CI tests.
result FMCIT effectively controls type-I error and maintains high testing power under the alternative hypothesis.
Improved sampling for high-dimensional posteriors with underdamped Langevin.
problem Scalability issues in high-dimensional problems with approximate Thompson sampling.
method Underdamped Langevin Monte Carlo for accelerated posterior concentration.
result Logarithmic regret improvement from i l d e O ( d ) \mathcal{ ilde O}(d) i l d e O ( d ) to i l d e O ( d ) \mathcal{ ilde O}(\sqrt{d}) i l d e O ( d ) . SDM Policy accelerates inference for robotic tasks while maintaining high action quality.
problem Prolonged inference times in diffusion-based policies for high-frequency control tasks.
method Two-stage optimization: score matching and distribution matching; dual-teacher mechanism.
result 6x inference speedup with state-of-the-art action quality.
BlosSOM improves data visualization for large datasets.
problem Insufficient performance of dimensionality reduction methods for large datasets.
method GPU-accelerated semi-supervised EmbedSOM algorithm.
result Produces high-quality visualizations with user control.
JaxMARL-HFT accelerates MARL for HFT with 240x speedup.
problem Heavy computational cost in MARL for HFT.
method GPU-accelerated JAX framework for multi-agent RL.
result Agents learn to outperform benchmarks in HFT.
Accelerates MCMC sampling for large-scale problems using machine learning.
problem Efficiently sampling large-scale Bayesian inference problems with high computational cost.
method Integrates low-fidelity machine learning models into a multilevel MCMC framework.
result Significantly accelerates multilevel sampling by a factor of two with similar accuracy.
Hardware-accelerated RBM solves large combinatorial problems and integer factorization.
problem Solving large combinatorial optimization and integer factorization problems.
method Logically synthesized RBM architecture, hardware acceleration, and efficient training methods.
result Hardware-accelerated RBM factorizes 16-bit numbers with 10000x speed and 32x power improvements.
A new method for generating synthetic data using posterior distribution learning accelerates inference.
problem Generating high-quality synthetic data requires many discretization steps, which is computationally expensive.
method Learning the posterior distribution of clean data samples given noisy versions, using a scoring rule instead of regression loss.
result Consistently outperforms standard diffusion models at few discretization steps.
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
Improved SDE-BNN model reduces NFEs and accelerates convergence.
problem High computational cost and convergence instability in SDE-BNNs.
method Nesterov's Accelerated Gradient (NAG) method integrated into SDE-BNN framework.
result Significantly reduced number of function evaluations (NFEs) and improved predictive accuracy.
ASBART accelerates Soft BART for faster Bayesian regression.
problem Slow computation in Soft BART.
method Proposed ASBART, a variant of Soft BART.
result ASBART is about 10 times faster than Soft BART with similar accuracy.
Computer simulations are invaluable tools for scientific discovery. However, accurate simulations are often slow to execute, which limits their applicability to extensive parameter exploration, large-scale data analysis, and uncertainty quantification. A promising route to accelerate simulations by building fast emulat…
RL accelerates portfolio optimization and option pricing by dynamically adjusting preconditioner sizes.
problem Large linear systems in portfolio optimization and option pricing lead to slow convergence.
method Reinforcement Learning (RL) dynamically adjusts block-preconditioner sizes to accelerate convergence.
result RL-driven solver significantly reduces computational cost and accelerates convergence.
Paper accelerates diffusion models, improving sampling speed.
problem Low sampling speed in score-based diffusion models.
method Design of novel training-free algorithms for deterministic and stochastic samplers.
result Accelerated samplers converge faster with improved rates.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
A new GP inference method using simplices for high-dimensional data.
problem Scalable Gaussian Processes in high dimensions.
method Developed a Simplex-GP method using a sparse simplicial grid to accelerate MVMs.
result Significantly faster GP inference in high dimensions compared to SKI.
New method speeds up diffusion models without requiring complex assumptions.
problem Slow sampling in diffusion models due to high computational cost.
method Training-free acceleration scheme under minimal assumptions.
result Provable acceleration within O ~ ( d 5 / 4 / ε ) \widetilde{O}(d^{5/4}/\sqrt{\varepsilon}) O ( d 5/4 / ε ) iterations. Paper accelerates diffusion models without retraining, reducing evaluations.
problem Approximating target data distributions efficiently.
method Training-free sampling algorithm using high-order Lagrange interpolation.
result Requires fewer score function evaluations than previous methods.
Paper accelerates K-means clustering for large sparse document data.
problem Efficiently clustering large-scale sparse document data.
method Designs an AFM algorithm leveraging UCs and inverted-index structure.
result Significantly improves clustering speed for large-scale documents.
Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to …
Optimized neural networks for Edge TPU achieve high accuracy in real-time image classification.
problem Designing neural networks for hardware accelerators to achieve optimal performance.
method Hardware-aware neural architecture search and model customization for Edge TPU.
result Improved accuracy-latency tradeoff on Pixel 4's Edge TPU compared to existing models.
New GPU kernels boost deep learning speed and memory efficiency.
problem Sparse deep learning matrices are not well-suited for existing sparse kernels.
method Identified favorable properties of sparse matrices from deep learning, developed high-performance GPU kernels for sparse matrix operations.
result 27% of single-precision peak performance on Nvidia V100 GPUs achieved with new kernels.
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Selective-Backprop uses the output of a training example's forward pass to decide whether to use that example to compute gradients and update para…
PipeDream-2BW accelerates large model training by 20x with minimal memory usage.
problem Training large models requires memory beyond single accelerator capacity.
method Pipeline parallelism, weight gradient coalescing, double buffering.
result Accelerates large model training by up to 20x.
Stella Nera accelerates matrix multiplications with a hash-based approach, achieving high energy efficiency and accuracy.
problem High computational and energy demands due to matrix multiplications in AI models.
method Stella Nera uses Maddness, a hash-based product quantization, to eliminate multipliers and achieve lookups and additions.
result Achieved an energy efficiency of 161 TOp/s/W@0.55V, 25x better than conventional MatMul accelerators.
PNDMs accelerate DDPMs by treating them as differential equations on manifolds.
problem Accelerate DDPMs while maintaining sample quality.
method Propose pseudo numerical methods (PNDMs) to solve differential equations on manifolds.
result PNDMs generate higher quality images with only 50 steps compared to 1000-step DDIMs (20x speedup).
Model-free deep reinforcement learning (RL) algorithms have been widely used for a range of complex control tasks. However, slow convergence and sample inefficiency remain challenging problems in RL, especially when handling continuous and high-dimensional state spaces. To tackle this problem, we propose a general acce…
DeepLight accelerates CTR predictions in ad serving by 46X.
problem Significantly increased serving delay and high memory usage for ad serving.
method Explicitly searching feature interactions, pruning layers, promoting sparsity.
result Accelerates model inference by 46X on Criteo dataset.
GPU-accelerated particle methods outperform neural samplers in LFT benchmarks.
problem High-dimensional multimodal sampling problems in lattice field theory.
method GPU-accelerated particle Monte Carlo methods (Sequential Monte Carlo and nested sampling).
result These methods match or outperform neural samplers in sample quality and wall-clock time.
Accelerates Birkhoff projection for manifold-constrained hyper-connections with high accuracy and speed.
problem Inaccurate and slow Birkhoff projection in mHC implementations.
method Dual formulation, Newton's method, implicit differentiation, warp-level CUDA kernel.
result Substantial speedups and accuracy improvements in doubly stochastic projections.
Recent technological advances have proliferated the available computing power, memory, and speed of modern Central Processing Units (CPUs), Graphics Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs). Consequently, the performance and complexity of Artificial Neural Networks (ANNs) is burgeoning. While…
LightOn OPUs accelerate randomized numerical linear algebra, reducing computational costs.
problem Computational bottleneck in randomization step for large-scale linear algebra.
method Near constant-time linear random projections from LightOn OPUs.
result Significant acceleration of RandNLA algorithms with negligible precision loss.