arXiv research

BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.

This study optimizes quantized neural networks by considering model architecture and quantization types.

We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…

With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods hav…

Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, \textit{Inter-Layer Weight Prediction} (ILWP) and quantization method which quantize the predicted residuals between the weights in all convol…

Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage re…

New method corrects quantization errors in LLMs using low-rank matrices.

This paper finds a new way to compress CNN weights, improving on pruning and quantization.

Quantized Adam reduces communication cost in deep learning training.

We propose Additive Powers-of-Two~(APoT) quantization, an efficient non-uniform quantization scheme for the bell-shaped and long-tailed distribution of weights and activations in neural networks. By constraining all quantization levels as the sum of Powers-of-Two terms, APoT quantization enjoys high computational effic…

Paper proposes a fast stochastic algorithm for neural network quantization with error bounds.

We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with $K$ entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…

Designing a deep neural network (DNN) with good generalization capability is a complex process especially when the weights are severely quantized. Model averaging is a promising approach for achieving the good generalization capability of DNNs, especially when the loss surface for training contains many sharp minima. W…

This paper introduces a differentiable, scalable quantization method for neural networks.

We consider the post-training quantization problem, which discretizes the weights of pre-trained deep neural networks without re-training the model. We propose multipoint quantization, a quantization method that approximates a full-precision weight vector using a linear combination of multiple vectors of low-bit number…

BiQGEMM efficiently multiplies quantized DNN weights using lookup tables.

Quantization-aware training can recover accuracy lost by post-training quantization.

Network quantization is one of the most hardware friendly techniques to enable the deployment of convolutional neural networks (CNNs) on low-power mobile devices. Recent network quantization techniques quantize each weight kernel in a convolutional layer independently for higher inference accuracy, since the weight ker…

Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…

Differentiable model compression adds noise to parameters during training.

This study analyzes quantization in deep learning models using statistical physics methods.

StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.

Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantization with the advanced optimization algorithm ADMM (Alternating Direction Methods of Multipliers) achieves one of state-of-art results in we…

New method improves accuracy of quantized neural networks.

FleXOR trains fractional quantization for neural networks, improving accuracy and size.

In order to quantize the gate parameters of the LSTM (Long Short-Term Memory) neural network model with almost no recognition performance degraded, a new quantization method named Quantization Loss Re-Learn Method is proposed in this paper. The method does lossy quantization on gate parameters during training iteration…

Post-training quantization saves resources for neural networks.

Kernel Quantization improves CNN compression without sacrificing performance.

Conformally equivariant quantization is a peculiar map between symbols of real weight $δ$ and differential operators acting on tensor densities, whose real weights are designed by $λ$ and $λ+δ$. The existence and uniqueness of such a map has been proved by Duval, Lecomte and Ovsienko for a generic weight $δ$. Later, Si…

We study a notion of pre-quantization for $b$-symplectic manifolds. We use it to construct a formal geometric quantization of $b$-symplectic manifolds equipped with Hamiltonian torus actions with nonzero modular weight. We show that these quantizations are finite dimensional $T$-modules.

Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…

We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization. By training quantization-ready networks, our approach enables storing a single set of weights that can be quantized on-deman…

Study identifies three quantization regimes for ReLU networks.

Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.

Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for training such networks, Learned Step Size Quantization, that achieves…

Deep neural networks (DNNs) can be made hardware-efficient by reducing the numerical precision of the weights and activations of the network and by improving the network's resilience to noise. However, this gain in efficiency often comes at the cost of significantly reduced accuracy. In this paper, we present a novel a…

We extend quantization-aware training to extreme model compression.

Although deep neural networks are highly effective, their high computational and memory costs severely challenge their applications on portable devices. As a consequence, low-bit quantization, which converts a full-precision neural network into a low-bitwidth integer version, has been an active and promising research t…

Efficient Bitwidth Search optimizes neural network quantization for better performance.

AdaRound improves post-training quantization of neural networks.

The Straight-Through Estimator (STE) is widely used for back-propagating gradients through the quantization function, but the STE technique lacks a complete theoretical understanding. We propose an alternative methodology called alpha-blending (AB), which quantizes neural networks to low-precision using stochastic grad…

Paper improves DNN accelerator robustness against bit errors with energy savings.

This paper proposes a new way to quantize classical mechanical systems. Here we use ALAG - programme to construct moduli space of half weighted Bohr - Sommerfeld lagrangian cycles of fixed volume which is our quantum phase space. "Dynamical correspondence" principle makes possible to prove that this ALAG - quantization…

Quantizes neural networks using frame theory for improved accuracy.

We introduce a data-free quantization method for deep neural networks that does not require fine-tuning or hyperparameter selection. It achieves near-original model performance on common computer vision architectures and tasks. 8-bit fixed-point quantization is essential for efficient inference on modern deep learning …

Paper optimizes KWS models using NAS and quantization for limited resources.

MobileNets family of computer vision neural networks have fueled tremendous progress in the design and organization of resource-efficient architectures in recent years. New applications with stringent real-time requirements on highly constrained devices require further compression of MobileNets-like already compute-eff…

Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations. Despite model size reduction, achieving performance enhancement on devices is, however, still challenging mainly due to the irregular representations of spa…