A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Quantization-aware training can recover accuracy lost by post-training quantization.
FrostNet improves INT8 quantization efficiency in mobile networks.
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
APQ jointly optimizes neural architecture, pruning, and quantization for efficient inference.
Degree-Quant improves GNN efficiency by quantizing them without losing accuracy.
We extend quantization-aware training to extreme model compression.
Generative model for HW-aware DNN quantization.
This study improves Ernie's accuracy for INT8 inference by modifying its training process.
QuantEase optimizes LLMs with CD-based quantization, achieving state-of-the-art performance.
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization. By training quantization-ready networks, our approach enables storing a single set of weights that can be quantized on-deman…
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
Paper develops a statistical framework for quantized training of deep neural networks.
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quant…
There is a mismatch between the standard theoretical analyses of statistical machine learning and how learning is used in practice. The foundational assumption supporting the theory is that we can represent features and models using real-valued parameters. In practice, however, we do not use real numbers at any point d…
Paper proposes a method to speed up DNNs by quantizing Winograd/Toom-Cook convolutions.
The implementation of Deep Convolutional Neural Networks (ConvNets) on tiny end-nodes with limited non-volatile memory space calls for smart compression strategies capable of shrinking the footprint yet preserving predictive accuracy. There exist a number of strategies for this purpose, from those that play with the to…
The study analyzes convergence of adaptive optimizers under low-precision training.
Low-precision representation of deep neural networks (DNNs) is critical for efficient deployment of deep learning application on embedded platforms, however, converting the network to low precision degrades its performance. Crucially, networks that are designed for embedded applications usually suffer from increased de…
BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.
Lightweight architectural designs of Convolutional Neural Networks (CNNs) together with quantization have paved the way for the deployment of demanding computer vision applications on mobile devices. Parallel to this, alternative formulations to the convolution operation such as FFT, Strassen and Winograd, have been ad…
Two methods reduce BN and DNN complexity, balancing size and accuracy.
New method uses geometric properties for better density estimation.
SOM-VQ tokenizes discrete models with semantic structure and navigable topology.
Study finds optimal learning rate schedules for sub-100M quantization-aware training across bit-widths.
Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks. This paper aims to raise people's awareness about the …
BatchNorm helps train quantized networks by avoiding gradient explosion.
Real-time semantic segmentation for autonomous vehicles on FPGA reduces latency and power consumption.
Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …
New method trains quantized neural networks to global optimality.
Quantized Adam reduces communication cost in deep learning training.
This paper introduces a differentiable, scalable quantization method for neural networks.
Neural network quantization procedure is the necessary step for porting of neural networks to mobile devices. Quantization allows accelerating the inference, reducing memory consumption and model size. It can be performed without fine-tuning using calibration procedure (calculation of parameters necessary for quantizat…
New framework reduces LLM complexity by directly finetuning in Boolean domain.
The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications. In particular, the growing size of deep learning models leads to higher communication overheads that defy the ideal linear training speedup regarding the number of devices. Gradient quantiza…
Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-precision model for efficient inference on such systems. However, training models directly with coarsely quantized weights is a key step towa…
The deep layers of modern neural networks extract a rather rich set of features as an input propagates through the network. This paper sets out to harvest these rich intermediate representations for quantization with minimal accuracy loss while significantly reducing the memory footprint and compute intensity of the DN…
Neural network quantization methods often involve simulating the quantization process during training, making the trained model highly dependent on the target bit-width and precise way quantization is performed. Robust quantization offers an alternative approach with improved tolerance to different classes of data-type…
Deep quantization of neural networks (below eight bits) offers significant promise in reducing their compute and storage cost. Albeit alluring, without special techniques for training and optimization, deep quantization results in significant accuracy loss. To further mitigate this loss, we propose a novel sinusoidal r…
Deep neural network (DNN) quantization converting floating-point (FP) data in the network to integers (INT) is an effective way to shrink the model size for memory saving and simplify the operations for compute acceleration. Recently, researches on DNN quantization develop from inference to training, laying a foundatio…
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary…
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…
AdaRound improves post-training quantization of neural networks.
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
Global-QSGD accelerates distributed training by up to 3.51%.
AdaQuantFL reduces communication in federated learning by adaptively quantizing model updates.