A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Precise computations of Dehn functions for subgroups of free group products.
Mixed-precision CA-SGD for generalized linear models on GPUs
Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a cruc…
Reduced precision computation for deep neural networks is one of the key areas addressing the widening compute gap driven by an exponential growth in model size. In recent years, deep learning training has largely migrated to 16-bit precision, with significant gains in performance and energy efficiency. However, attemp…
Paper explores reducing precision in SVM for faster text classification.
Paper speeds up GP inference by reducing precision matrix computation.
While Recurrent Neural Networks (RNNs) are famously known to be Turing complete, this relies on infinite precision in the states and unbounded computation time. We consider the case of RNNs with finite precision whose computation time is linear in the input length. Under these limitations, we show that different RNN va…
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…
A framework estimates multiple precision matrices with shared structures.
With ever-increasing computational demand for deep learning, it is critical to investigate the implications of the numeric representation and precision of DNN model weights and activations on computational efficiency. In this work, we explore unconventional narrow-precision floating-point representations as it relates …
State of the art deep learning models have made steady progress in the fields of computer vision and natural language processing, at the expense of growing model sizes and computational complexity. Deploying these models on low power and mobile devices poses a challenge due to their limited compute capabilities and str…
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to an…
The study formalizes temporal precision and recall for anomaly detection in sequences.
Low precision weights, activations, and gradients have been proposed as a way to improve the computational efficiency and memory footprint of deep neural networks. Recently, low precision networks have even shown to be more robust to adversarial attacks. However, typical implementations of low precision DNNs use unifor…
The recent advent of `Internet of Things' (IOT) has increased the demand for enabling AI-based edge computing. This has necessitated the search for efficient implementations of neural networks in terms of both computations and storage. Although extreme quantization has proven to be a powerful tool to achieve significan…
The high computational and parameter complexity of neural networks makes their training very slow and difficult to deploy on energy and storage-constrained computing systems. Many network complexity reduction techniques have been proposed including fixed-point implementation. However, a systematic approach for designin…
The state-of-the-art hardware platforms for training Deep Neural Networks (DNNs) are moving from traditional single precision (32-bit) computations towards 16 bits of precision -- in large part due to the high energy efficiency and smaller bit storage associated with using reduced-precision representations. However, un…
EMPIR combines low and full precision DNNs to enhance robustness against adversarial attacks.
The crosscap number of a knot is an invariant describing the non-orientable surface of smallest genus that the knot bounds. Unlike knot genus (its orientable counterpart), crosscap numbers are difficult to compute and no general algorithm is known. We present three methods for computing crosscap number that offer varyi…
Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to empirically evaluate the statistical effects of quantization while avoiding the substantial overhead of building specific hardware. To suppor…
New framework reduces cost of financial option pricing simulations on FPGAs.
In recent years, dynamically growing data and incrementally growing number of classes pose new challenges to large-scale data classification research. Most traditional methods struggle to balance the precision and computational burden when data and its number of classes increased. However, some methods are with weak pr…
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the num…
Post-training quantization method using multiple low-precision points achieves higher precision for critical weights.
Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demonstrate, for the first time, that low precision policy distillation from a high precision network pro…
Two-layer networks struggle with high frequencies due to numerical and computational limitations.
Gradient descent stagnates in low-precision, but unbiased rounding schemes improve convergence.
Efficient Bitwidth Search optimizes neural network quantization for better performance.
High-precision machine learning reduces particle physics simulations by orders of magnitude.
Finite-precision learning of networks is limited by the Monte Carlo rate.
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
Quantized Neural Networks (QNNs) are often used to improve network efficiency during the inference phase, i.e. after the network has been trained. Extensive research in the field suggests many different quantization schemes. Still, the number of bits required, as well as the best quantization scheme, are yet unknown. O…
A new optimization method reduces memory and compute requirements for deep learning.
The paper addresses numerical integration issues in SV models, proposing a fast regime switching algorithm.
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
This paper numerically computes the topological and smooth invariants of Eschenburg spaces with small fourth cohomology group, following Kruggel's determination of the Kreck-Stolz invariants of Eschenburg spaces that satisfy condition C. The GNU GMP arbitrary-precision library is utilised.
Optimizes CM for stochastic convex optimization with progressive precision.
Efficient Winograd convolution for INT8 networks using RNS.
FQ-Conv quantizes CNNs for efficient inference with low-precision weights and activations.
A new algorithm improves GLasso for sparse precision matrix estimation.
Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and F…
We address the task of identifying densely connected subsets of multivariate Gaussian random variables within a graphical model framework. We propose two novel estimators based on the Ordered Weighted (OWL) norm: 1) The Graphical OWL (GOWL) is a penalized likelihood method that applies the OWL norm to the lowe…
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …
We study a large economy in which firms cannot compute exact solutions to the non-linear equations that characterize the equilibrium price at which they can sell future output. Instead, firms use polynomial expansions to approximate prices. The precision with which they can compute prices is endogenous and depends on t…
Boosting methods are highly popular and effective supervised learning methods which combine weak learners into a single accurate model with good statistical performance. In this paper, we analyze two well-known boosting methods, AdaBoost and Incremental Forward Stagewise Regression (FS), by establishing t…
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
We give a precise and concise formulation of the orientifold construction in Type II superstring theory. Our results include anomaly cancellation on the worldsheet and a spacetime computation of the background Ramond-Ramond charge.