Quantizes neural networks using frame theory for improved accuracy.
problem Improving neural network efficiency and accuracy through quantization.
method Sigma-Delta (ΣΔ) quantization with finite unit-norm tight frames. result Error bound between original and quantized neural networks derived.
Extends ONNX for quantized neural networks with new formats and operators.
problem Handling arbitrary-precision quantization in neural networks.
method Introduces new formats and operators in ONNX to represent quantized neural networks.
result Enabled representation of uniform quantization in neural networks.
This study optimizes quantized neural networks by considering model architecture and quantization types.
problem Optimizing quantized neural networks for low-power, high-throughput applications.
method Holistic approach including training methods and quantization-friendly architecture design.
result Deeper models are more sensitive to activation quantization, while wider models improve resilience to both weight and activation quantization.
This paper introduces a differentiable, scalable quantization method for neural networks.
problem Previous quantization methods lacked differentiability and scalability.
method The approach is differentiable and scalable, using bit-shifting and logarithmic quantization.
result The method achieves comparable accuracy to state-of-the-art approaches with less training time and lower inference cost.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
Paper develops a statistical framework for quantized training of deep neural networks.
problem Lack of theoretical understanding of gradient quantization in FQT.
method Presented a statistical framework for analyzing FQT algorithms, viewing quantized gradient as a stochastic estimator of QAT gradient.
result Developed two novel gradient quantizers with smaller variance than existing per-tensor quantizer.
Memory-augmented neural networks (MANNs) refer to a class of neural network models equipped with external memory (such as neural Turing machines and memory networks). These neural networks outperform conventional recurrent neural networks (RNNs) in terms of learning long-term dependency, allowing them to solve intrigui…
Although deep neural networks are highly effective, their high computational and memory costs severely challenge their applications on portable devices. As a consequence, low-bit quantization, which converts a full-precision neural network into a low-bitwidth integer version, has been an active and promising research t…
New method trains quantized neural networks to global optimality.
problem Training optimal quantized neural networks is intractable due to combinatorial non-convex optimization.
method Convex optimization strategy using hidden convexity, semidefinite lifting, and Grothendieck's identity.
result Quantized NN problems can be solved to global optimality in polynomial-time.
Proposes a robust neural network quantization method.
problem Training model's dependency on specific quantization methods.
method Intrinsic robustness to various quantization processes.
result Single model capable of operating at various bit-widths and policies.
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
Paper proposes a new binary quantization method for faster DNN inference.
problem Accelerating deep neural network inference on resource-limited devices.
method Quantized Compressed Sensing (QCS) for binary quantization.
result The proposed method preserves benefits of standard methods while reducing quantization error.
We present a novel method for neural network quantization that emulates a non-uniform k-quantile quantizer, which adapts to the distribution of the quantized parameters. Our approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We suggest to compare the results as …
Paper proposes a fast stochastic algorithm for neural network quantization with error bounds.
problem Error analysis for quantized neural networks with non-convex loss functions and nonlinear activations.
method Greedy path-following mechanism combined with stochastic quantizer.
result Established full-network error bounds for quantized neural networks.
BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.
problem Performance degradation of compact neural networks under extreme quantization.
method Task-dependent Aggregated Transformation (BiTAT) method that orthonormalizes weights and progressively quantizes them.
result BiTAT effectively preserves model performance on ImageNet and CIFAR-100 with compact backbones.
This paper evaluates quantization techniques for deep learning inference.
problem Reducing the size and improving inference of deep neural networks.
method Review and empirical evaluation of quantization parameters for various neural network models.
result 8-bit quantization maintains accuracy within 1% of floating-point models.
SQWA improves low-precision DNNs with model averaging and quantization.
problem Designing good generalization DNNs with quantized weights.
method Floating-point model training, direct quantization, multiple low-precision models, weight averaging, re-quantization, fine-tuning, loss visualization.
result State-of-the-art results for 2-bit QDNNs on CIFAR-100 and ImageNet datasets.
DQA efficiently quantizes deep neural network activations for resource-constrained devices.
problem Efficiently quantizing deep neural network activations for resource-constrained devices.
method DQA uses simple shifting-based operations and Huffman coding for sub-6-bit quantization.
result DQA achieves significantly better accuracy than direct quantization and state-of-the-art methods.
Post-training quantization saves resources for neural networks.
problem Implementing neural networks in resource-constrained hardware.
method Generalized post-training quantization method (GPFQ) with modifications for sparsity and error analysis.
result Error decays linearly with over-parametrization, showing minor loss of accuracy.
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
problem Quantization limits to integer bits restricts compression and accuracy.
method Encryption algorithm with XOR gates for fractional bits during inference.
result FleXOR achieves high accuracy with fractional sub-1-bit weights.
BatchNorm helps train quantized networks by avoiding gradient explosion.
problem Training quantized neural networks is difficult due to gradient issues.
method Investigated the impact of BatchNorm on both full-precision and quantized networks.
result BatchNorm avoids gradient explosion in quantized networks, contrary to expectations.
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
problem Optimal quantization parameters selection for deep neural networks with diverse data distributions.
method Statistical error analysis framework for uniform and floating-point quantization, iterative and analytic quantizers designed for arbitrary and Gaussian-like distributions.
result Improved accuracy and stability in training low-precision neural networks.
Neural network quantization procedure is the necessary step for porting of neural networks to mobile devices. Quantization allows accelerating the inference, reducing memory consumption and model size. It can be performed without fine-tuning using calibration procedure (calculation of parameters necessary for quantizat…
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
Deep neural networks are the state-of-the-art methods for many real-world tasks, such as computer vision, natural language processing and speech recognition. For all its popularity, deep neural networks are also criticized for consuming a lot of memory and draining battery life of devices during training and inference.…
Quantized Adam reduces communication cost in deep learning training.
problem Reducing communication cost in distributed deep learning training.
method Gradient and weight quantization with error feedback in Adam.
result Proposed methods converge to first-order stationary points.
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
SMGD trains low-bit neural networks with memory constraints.
problem Training large neural networks with limited memory.
method Stochastic Markov Gradient Descent (SMGD).
result Encouraging numerical results and theoretical guarantees.
DJPQ optimizes neural network pruning and quantization for hardware efficiency.
problem Efficiently compress neural networks for hardware inference.
method Joint gradient-based optimization of pruning and quantization into a differentiable loss function.
result Significant reduction in Bit-Operations (BOPs) with minimal accuracy loss.
We present a theoretical and experimental investigation of the quantization problem for artificial neural networks. We provide a mathematical definition of quantized neural networks and analyze their approximation capabilities, showing in particular that any Lipschitz-continuous map defined on a hypercube can be unifor…
Optimal gradient quantization reduces communication costs in distributed deep learning.
problem High communication costs in distributed training of deep neural networks.
method Deduced optimal gradient quantization conditions for binary and multi-level quantization, developed novel schemes for dynamic quantization levels.
result Demonstrated superior performance of proposed quantization schemes on CIFAR and ImageNet datasets.
To make deep neural networks feasible in resource-constrained environments (such as mobile devices), it is beneficial to quantize models by using low-precision weights. One common technique for quantizing neural networks is the straight-through gradient method, which enables back-propagation through the quantization ma…
New insights into quantized neural networks reveal learning dynamics and generalization errors.
problem Understanding the impact of quantization hyperparameters on learning dynamics in high-dimensional models.
method Theoretical analysis and fixed-point analysis of STE dynamics in quantized models.
result STE training in quantized models converges to a plateau followed by a sharp drop in generalization error, influenced by quantization range.
With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods hav…
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
problem Quantization challenges in Batch Normalization for QNNs.
method Converts BN to fixed-point operation with shared scale, suitable for hardware.
result Maintains same outputs through rigorous analysis and experiments.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.
method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.
result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
problem Improving quantized deep neural networks for edge devices.
method SPEQ combines stochastic precision sharing and cosine similarity loss for knowledge distillation.
result SPEQ outperforms existing methods in various tasks.
Based on the notion of information bottleneck (IB), we formulate a quantization problem called "IB quantization". We show that IB quantization is equivalent to learning based on the IB principle. Under this equivalence, the standard neural network models can be viewed as scalar (single sample) IB quantizers. It is know…
NEMO framework quantizes DNNs for efficient deployment.
problem Efficient deployment of quantized DNNs.
method Formal framework for quantizing DNN layers, focusing on IntegerDeployable representation.
result Quantized DNNs can be deployed using only integers.
Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks. This paper aims to raise people's awareness about the …
AskewSGD optimizes quantized neural networks with interval-constrained optimization.
problem Training deep neural networks with quantized weights.
method Formulates QNN training as smoothed interval-constrained optimization, proposes AskewSGD for solving each subproblem.
result AskewSGD avoids projections and allows infeasible iterates, performs better than state-of-the-art methods.
In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large memory consumption, which may not be affordable for mobile platforms. Deep model q…
AdaRound improves post-training quantization of neural networks.
problem Improving the accuracy of quantized weights in neural networks.
method Adaptive rounding mechanism that adapts to data and task loss.
result AdaRound outperforms rounding-to-nearest and achieves state-of-the-art performance.
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
As the will to deploy neural networks models on embedded systems grows, and considering the related memory footprint and energy consumption issues, finding lighter solutions to store neural networks such as weight quantization and more efficient inference methods become major research topics. Parallel to that, adversar…
New method improves accuracy of quantized neural networks.
problem Accuracy drop in quantized neural networks, especially MobileNet family.
method Weight equalizing shift scaler, binary shifting to recover output range.
result Top-1 accuracy improved from 0.1% to 69.78% ~ 70.96% in MobileNets.
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…
SGQuant reduces GNN memory usage without significant accuracy loss.
problem High memory consumption in GNNs limits their applicability on memory-constrained devices.
method Proposes a specialized GNN quantization scheme (SGQuant) with a quantization algorithm, fine-tuning scheme, and multi-granularity strategy.
result SGQuant reduces GNN memory footprint from 4.25x to 31.9x with minimal accuracy loss.