Unified framework LPCD optimizes quantization of complex submodules.
problem Quantization of complex submodules in neural networks.
method Layer-Projected Coordinate Descent (LPCD) for quantizing arbitrary submodules.
result LPCD enhances both layer-wise PTQ methods and existing submodule approaches.
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
problem Growth of quantization errors across layers degrades performance, especially in low-bit regimes.
method Quantization Error Propagation (QEP) framework that explicitly propagates and compensates for quantization errors.
result QEP-enhanced layer-wise PTQ achieves substantially higher accuracy, especially in low-bit regimes.
Analyzes layer-wise quantization effects in neural networks.
problem Identifying and fixing degradation in quantized neural networks.
method Layer-wise quantization analysis framework.
result Local fixes can significantly reduce quantization degradation.
BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.
problem Performance degradation of compact neural networks under extreme quantization.
method Task-dependent Aggregated Transformation (BiTAT) method that orthonormalizes weights and progressively quantizes them.
result BiTAT effectively preserves model performance on ImageNet and CIFAR-100 with compact backbones.
Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, \textit{Inter-Layer Weight Prediction} (ILWP) and quantization method which quantize the predicted residuals between the weights in all convol…
In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large memory consumption, which may not be affordable for mobile platforms. Deep model q…
Network quantization is one of the most hardware friendly techniques to enable the deployment of convolutional neural networks (CNNs) on low-power mobile devices. Recent network quantization techniques quantize each weight kernel in a convolutional layer independently for higher inference accuracy, since the weight ker…
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
DFS dynamically decides bitwidths for layers to balance accuracy and efficiency.
problem Balancing model accuracy and inference speed for deep networks.
method Dynamic Fractional Skipping (DFS) framework that assigns bitwidths to layers for input-adaptive inference.
result DFS achieves superior tradeoff between computational cost and model accuracy.
Ternary MobileNets improve efficiency and accuracy on constrained devices.
problem Efficiently compressing MobileNets for real-time applications on constrained devices.
method Per-layer hybrid filter banks for ternary quantization of MobileNets.
result 27.98% energy savings and 51.07% reduction in model size with comparable accuracy.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
New method improves accuracy of quantized neural networks.
problem Accuracy drop in quantized neural networks, especially MobileNet family.
method Weight equalizing shift scaler, binary shifting to recover output range.
result Top-1 accuracy improved from 0.1% to 69.78% ~ 70.96% in MobileNets.
Improved neural quantization reduces accuracy loss to less than 1% with 4-bit weights.
problem Reducing accuracy loss in neural quantization below 8-bits.
method Layer-wise calibration and integer programming to optimize bit-width allocation.
result Less than 1% accuracy degradation with 4-bit weights and activations.
QuantEase optimizes LLMs with CD-based quantization, achieving state-of-the-art performance.
problem Efficiently quantize large language models for deployment.
method Layer-wise quantization using CD-based algorithms with matrix and vector operations.
result State-of-the-art performance in perplexity and zero-shot accuracy.
Kernel Quantization improves CNN compression without sacrificing performance.
problem Efficiently compressing CNN models without significant performance loss.
method Quantizes convolution kernels as the unit, learning a codebook for low-bit indexes.
result Significant compression ratio achieved with minimal accuracy loss.
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…
BatchNorm helps train quantized networks by avoiding gradient explosion.
problem Training quantized neural networks is difficult due to gradient issues.
method Investigated the impact of BatchNorm on both full-precision and quantized networks.
result BatchNorm avoids gradient explosion in quantized networks, contrary to expectations.
Paper proposes a fast stochastic algorithm for neural network quantization with error bounds.
problem Error analysis for quantized neural networks with non-convex loss functions and nonlinear activations.
method Greedy path-following mechanism combined with stochastic quantizer.
result Established full-network error bounds for quantized neural networks.
WaveQ uses sinusoidal regularization to optimize deep quantization for neural networks, improving both efficiency and accuracy.
problem Deep quantization reduces bitwidth but can lead to significant accuracy loss due to inter-layer dependencies.
method WaveQ employs sinusoidal regularization to learn multiple quantization parameters during gradient-based training, balancing compute efficiency and accuracy.
result WaveQ achieves accuracy preservation and efficiency gains across various deep networks, outperforming state-of-the-art techniques.
Study integrability of quantized six-vertex model on torus.
problem Integrability of a specific lattice model on a torus.
method Defined layer transfer matrices and tetrahedron equations for admissible graphs.
result Established commutativity of transfer matrices and derived quantum Hamiltonians.
Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantization with the advanced optimization algorithm ADMM (Alternating Direction Methods of Multipliers) achieves one of state-of-art results in we…
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
Generative model for HW-aware DNN quantization.
problem Efficiently tuning DNNs for various hardware platforms.
method Generative model AQGAN for generating quantization configurations based on target accuracy.
result Generative model generates quantization configurations with competitive accuracy and lower search cost.
One-bit quantization improves inference speed for Random Features models.
problem Efficient inference on resource-constrained devices.
method Analysis of one-bit quantization in Random Features model.
result Asymptotically, quantizing weights except the last incurs no loss in generalization error.
This paper proposes an automatic neural network compression method.
problem Reducing resource requirements for deep neural networks on resource-constrained devices.
method Jointly prunes and quantizes neural networks without manual hyper-parameter tuning.
result Significant reduction in model size with minimal accuracy loss.
Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …
Quantization of neural networks has become common practice, driven by the need for efficient implementations of deep neural networks on embedded devices. In this paper, we exploit an oft-overlooked degree of freedom in most networks - for a given layer, individual output channels can be scaled by any factor provided th…
Study shows LLMs can remove half of layers without significant performance drop.
problem Understanding knowledge storage in LLMs' weights.
method Layer pruning and finetuning to identify and remove unnecessary parameters.
result Minimal degradation of performance after removing up to half of layers.
Deep Neural Networks (DNNs) typically require massive amount of computation resource in inference tasks for computer vision applications. Quantization can significantly reduce DNN computation and storage by decreasing the bitwidth of network encodings. Recent research affirms that carefully selecting the quantization l…
Quantized Neural Networks (QNNs) are often used to improve network efficiency during the inference phase, i.e. after the network has been trained. Extensive research in the field suggests many different quantization schemes. Still, the number of bits required, as well as the best quantization scheme, are yet unknown. O…
New method trains quantized neural networks to global optimality.
problem Training optimal quantized neural networks is intractable due to combinatorial non-convex optimization.
method Convex optimization strategy using hidden convexity, semidefinite lifting, and Grothendieck's identity.
result Quantized NN problems can be solved to global optimality in polynomial-time.
This paper compresses neural networks by permuting and quantizing weights.
problem Efficiently compressing large neural networks for resource-constrained platforms.
method Permuting and quantizing weights, connecting to rate-distortion theory, and using annealed quantization.
result Significant compression with minimal accuracy loss, e.g., 40-70% reduction in gap with uncompressed model.
Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings. Despite their effectiveness, the number of parameters in an embedding layer increases linearly with the number of symbols and poses a critical challenge on memory and storage constraints. I…
Winograd convolutions are used to improve quantized neural networks.
problem Improving quantized neural networks using Winograd convolutions.
method Proposed a Winograd-aware formulation of convolution layers to expose numerical inaccuracies to learning.
result Up to 10% higher classification accuracy on CIFAR-10 with Winograd-aware layers.
VQ-GNN scales GNNs to large graphs using vector quantization.
problem Scaling GNNs to large graphs with stable performance and speed.
method VQ-GNN uses vector quantization to preserve all messages passed to a mini-batch of nodes, avoiding the 'neighbor explosion' problem.
result VQ-GNN achieves competitive performance on large-graph node classification and link prediction benchmarks.
ActNN reduces neural network training memory by 2-bit quantization.
problem Limited memory in training neural networks.
method Randomly quantizes activations to 2 bits, proving convergence and proposing mixed-precision strategies.
result Reduces activation memory footprint by 12x with negligible accuracy loss.
Efficient Bitwidth Search optimizes neural network quantization for better performance.
problem Finding optimal bitwidth for weights and activations of each layer efficiently.
method EBS algorithm reusing meta weights and binary decomposition for efficient mixed precision convolution.
result Mixed precision QNN outperforms uniform bitwidth and other techniques on CIFAR10 and ImageNet.
NEMO framework quantizes DNNs for efficient deployment.
problem Efficient deployment of quantized DNNs.
method Formal framework for quantizing DNN layers, focusing on IntegerDeployable representation.
result Quantized DNNs can be deployed using only integers.
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
problem Quantization challenges in Batch Normalization for QNNs.
method Converts BN to fixed-point operation with shared scale, suitable for hardware.
result Maintains same outputs through rigorous analysis and experiments.
Memory-augmented neural networks (MANNs) refer to a class of neural network models equipped with external memory (such as neural Turing machines and memory networks). These neural networks outperform conventional recurrent neural networks (RNNs) in terms of learning long-term dependency, allowing them to solve intrigui…
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quant…
Differentiable model compression adds noise to parameters during training.
problem Model compression for efficient deployment.
method Adding pseudo quantization noise to model parameters during training.
result Compresses model by more than 8x on ImageNet with 0.3% accuracy loss.
This paper finds a new way to compress CNN weights, improving on pruning and quantization.
problem Improving performance and storage efficiency of CNNs.
method Identifying and exploiting repeated patterns in CNN weight tensors, using Huffman coding and block sparse matrix formats.
result Achieved compaction ratios of 1.4x to 3.1x in addition to pruning and quantization.
Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for training such networks, Learned Step Size Quantization, that achieves…
Study shows simple vector quantization measures correlate with deep learning generalization.
problem Understanding and predicting generalization in deep learning models.
method Applying complexity measures from approximation and information theory to deep learning features.
result Simple vector quantization measures correlate well with generalization performance in deep learning.
SGQuant reduces GNN memory usage without significant accuracy loss.
problem High memory consumption in GNNs limits their applicability on memory-constrained devices.
method Proposes a specialized GNN quantization scheme (SGQuant) with a quantization algorithm, fine-tuning scheme, and multi-granularity strategy.
result SGQuant reduces GNN memory footprint from 4.25x to 31.9x with minimal accuracy loss.
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
problem Improving quantized deep neural networks for edge devices.
method SPEQ combines stochastic precision sharing and cosine similarity loss for knowledge distillation.
result SPEQ outperforms existing methods in various tasks.