A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This study optimizes quantized neural networks by considering model architecture and quantization types.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
DQA efficiently quantizes deep neural network activations for resource-constrained devices.
DFRot improves LLMs by reducing outlier and massive activation effects.
S2D selectively decays large singular values to improve quantization of neural activations.
Improved EXACT strategy reduces GNN memory consumption and runtime.
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…
BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.
Quantized neural networks can represent all fixed-point functions under certain conditions.
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…
New method corrects quantization errors in LLMs using low-rank matrices.
Reducing the precision of weights and activation functions in neural network training, with minimal impact on performance, is essential for the deployment of these models in resource-constrained environments. We apply mean-field techniques to networks with quantized activations in order to evaluate the degree to which …
ActNN reduces neural network training memory by 2-bit quantization.
Improved SNNs with quantized activations outperform traditional networks.
FQ-Conv quantizes CNNs for efficient inference with low-precision weights and activations.
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
This paper introduces a differentiable, scalable quantization method for neural networks.
APoT quantization improves neural network efficiency and accuracy.
An appropriate choice of the activation function (like ReLU, sigmoid or swish) plays an important role in the performance of (deep) multilayer perceptrons (MLP) for classification and regression learning. Prototype-based classification learning methods like (generalized) learning vector quantization (GLVQ) are powerful…
New method quantizes neural networks for mobile devices.
Efficient Bitwidth Search optimizes neural network quantization for better performance.
Convolutional Neural Networks (CNN) has become more popular choice for various tasks such as computer vision, speech recognition and natural language processing. Thanks to their large computational capability and throughput, GPUs ,which are not power efficient and therefore does not suit low power systems such as mobil…
This paper addresses a challenging problem - how to reduce energy consumption without incurring performance drop when deploying deep neural networks (DNNs) at the inference stage. In order to alleviate the computation and storage burdens, we propose a novel dataflow-based joint quantization approach with the hypothesis…
Study quantization effects on high-dimensional linear regression learning.
Paper optimizes KWS models using NAS and quantization for limited resources.
Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for training such networks, Learned Step Size Quantization, that achieves…
Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
Improved neural quantization reduces accuracy loss to less than 1% with 4-bit weights.
Model quantization is leveraged to reduce the memory consumption and the computation time of deep neural networks. This is achieved by representing weights and activations with a lower bit resolution when compared to their high precision floating point counterparts. The suitable level of quantization is directly relate…
Similar to convolution neural networks, recurrent neural networks (RNNs) typically suffer from over-parameterization. Quantizing bit-widths of weights and activations results in runtime efficiency on hardware, yet it often comes at the cost of reduced accuracy. This paper proposes a quantization approach that increases…
Quantized Neural Networks (QNNs) are often used to improve network efficiency during the inference phase, i.e. after the network has been trained. Extensive research in the field suggests many different quantization schemes. Still, the number of bits required, as well as the best quantization scheme, are yet unknown. O…
OverQ increases model accuracy by handling outliers in neural networks with minimal hardware changes.
New method trains quantized neural networks to global optimality.
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization. By training quantization-ready networks, our approach enables storing a single set of weights that can be quantized on-deman…
Paper develops a statistical framework for quantized training of deep neural networks.
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices. In order to train networks that can be effectively discretized without loss of performance, we introduce a differentiable quantization procedure. Differentiability can…
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
We introduce a data-free quantization method for deep neural networks that does not require fine-tuning or hyperparameter selection. It achieves near-original model performance on common computer vision architectures and tasks. 8-bit fixed-point quantization is essential for efficient inference on modern deep learning …
Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents. The two most frequently discussed benefits of quantization are reduced memory consumption, and a faster forward pass when implemented with efficient bitwise operations. We propose a t…
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
To improve the execution speed and efficiency of neural networks in embedded systems, it is crucial to decrease the model size and computational complexity. In addition to conventional compression techniques, e.g., weight pruning and quantization, removing unimportant activations can reduce the amount of data communica…
Efficient deep neural network (DNN) inference on mobile or embedded devices typically involves quantization of the network parameters and activations. In particular, mixed precision networks achieve better performance than networks with homogeneous bitwidth for the same size constraint. Since choosing the optimal bitwi…
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
Paper proposes BQNs for efficient Bayesian quantized neural networks.
Paper proposes a fast stochastic algorithm for neural network quantization with error bounds.
Recurrent neural networks have achieved excellent performance in many applications. However, on portable devices with limited resources, the models are often too large to deploy. For applications on the server with large scale concurrent requests, the latency during inference can also be very critical for costly comput…