A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
SEED RL accelerates deep RL training on modern accelerators.
Principal component analysis (PCA) is one of the most powerful tools in machine learning. The simplest method for PCA, the power iteration, requires full-data passes to recover the principal component of a matrix with eigen-gap . Lanczos, a significantly more complex method, achieves an accelerated…
Many applications in Bayesian statistics are extremely computationally intensive. However, they are often inherently parallel, making them prime targets for modern massively parallel processors. Multi-core and distributed computing is widely applied in the Bayesian community, however, very little attention has been giv…
Study accelerates gradient methods in machine learning, revealing risk and stability connections.
Selective-Backprop accelerates deep learning by focusing on high-loss examples.
Modern datasets and models are notoriously difficult to explore and analyze due to their inherent high dimensionality and massive numbers of samples. Existing visualization methods which employ dimensionality reduction to two or three dimensions are often inefficient and/or ineffective for these datasets. This paper in…
New method accelerates CNNs for mobile devices by approximating tensors and quantizing weights.
Review of modern computational optimal transport methods for biomedical applications.
We propose a generic algorithmic building block to accelerate training of machine learning models on heterogeneous compute systems. Our scheme allows to efficiently employ compute accelerators such as GPUs and FPGAs for the training of large-scale machine learning models, when the training data exceeds their memory cap…
This paper revisits the classic iterative proportional scaling (IPS) from a modern optimization perspective. In contrast to the criticisms made in the literature, we show that based on a coordinate descent characterization, IPS can be slightly modified to deliver coefficient estimates, and from a majorization-minimizat…
This work optimizes MCMC algorithms for modern accelerators without synchronization overheads.
New methods accelerate NCGP inference by trading computation for uncertainty.
Recent technological advances have proliferated the available computing power, memory, and speed of modern Central Processing Units (CPUs), Graphics Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs). Consequently, the performance and complexity of Artificial Neural Networks (ANNs) is burgeoning. While…
Nesterov SGD is widely used for training modern neural networks and other machine learning models. Yet, its advantages over SGD have not been theoretically clarified. Indeed, as we show in our paper, both theoretically and empirically, Nesterov SGD with any parameter selection does not in general provide acceleration o…
SparseRT accelerates sparse computations on GPUs for deep learning inference.
ES improves training efficiency by dynamically selecting data samples.
Modern distributed training of machine learning models suffers from high communication overhead for synchronizing stochastic gradients and model parameters. In this paper, to reduce the communication complexity, we propose \emph{double quantization}, a general scheme for quantizing both model parameters and gradients. …
The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and the design of accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition…
Unified framework for accelerated Perceptron and related problems.
cuRegOT accelerates GPU-based entropic OT solving.
Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research. To scale to large datasets and high-dimensional models, Pyro uses stochastic variational inference algorithms and probability distributions built on top of PyTorch, a modern GPU-accele…
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without moving data across machines. In this paper, we introduce a novel distributed du…
GrateTile optimizes CNN feature map storage for efficient data access.
This paper optimizes how deep learning models are distributed across different devices.
Paper improves learning rates for SGD and NAG.
This paper accelerates sparse CNN layers on GPUs by using unstructured sparsity.
KaoKore dataset extracts faces from pre-modern Japanese art for machine learning.
Learning rate decay (lrDecay) is a \emph{de facto} technique for training modern neural networks. It starts with a large learning rate and then decays it multiple times. It is empirically observed to help both optimization and generalization. Common beliefs in how lrDecay works come from the optimization analysis of (S…
This paper evaluates ASR models on various co-processors, showing hardware acceleration benefits.
Neural networks have proven to be extremely powerful tools for modern artificial intelligence applications, but computational and storage complexity remain limiting factors. This paper presents two compatible contributions towards reducing the time, energy, computational, and storage complexities associated with multil…
We introduce vine computational graphs for efficient ML integration of vine copulas.
ADSGD method speeds up model identification in sparse optimization.
While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the evaluation environment, places the public in danger, and, due to the rare nature of accidents, will require billions of miles in or…
Modern machine learning focuses on highly expressive models that are able to fit or interpolate the data completely, resulting in zero training loss. For such models, we show that the stochastic gradients of common loss functions satisfy a strong growth condition. Under this condition, we prove that constant step-size …
New PDMP samplers improve BNN inference with accelerated computation.
Speaker embeddings are continuous-value vector representations that allow easy comparison between voices of speakers with simple geometric operations. Among others, i-vector and x-vector have emerged as the mainstream methods for speaker embedding. In this paper, we illustrate the use of modern computation platform to …
ADASAP accelerates GP inference for large datasets.
BoTorch optimizes Bayesian optimization with MC methods and auto-differentiation.
New insights show NAG and FISTA converge linearly without knowing strong convexity modulus.
Modern statistical inference tasks often require iterative optimization methods to compute the solution. Convergence analysis from an optimization viewpoint only informs us how well the solution is approximated numerically but overlooks the sampling nature of the data. In contrast, recognizing the randomness in the dat…
Deep neural networks have yielded superior performance in many applications; however, the gradient computation in a deep model with millions of instances lead to a lengthy training process even with modern GPU/TPU hardware acceleration. In this paper, we propose AutoAssist, a simple framework to accelerate training of …
The paper analyzes contraction rates for GP regression approximations.
This paper improves the convergence rates of bilevel optimization algorithms.
Data augmentation, a technique in which a training set is expanded with class-preserving transformations, is ubiquitous in modern machine learning pipelines. In this paper, we seek to establish a theoretical framework for understanding data augmentation. We approach this from two directions: First, we provide a general…
New method speeds up diffusion models inference to sub-linear time.
OverQ increases model accuracy by handling outliers in neural networks with minimal hardware changes.
IMPACT improves RL training speed without sacrificing sample efficiency.
Machine learning boosts physics research, especially at high energy experiments.