Paper proposes efficient INT4 quantization for neural networks.
problem Efficient inference on limited hardware resources for large neural networks.
method Formalizes quantization as MMSE problem, optimizes constrained MSE at each layer, partitions parameters, uses multiple quantized tensors.
result 4-bit integer (INT4) quantization yields state-of-the-art results with minimal accuracy loss.
MCQ uses Monte Carlo methods to efficiently quantize neural networks without re-training.
problem Efficiently quantizing neural networks for lower power consumption.
method Importance sampling for quantization of weights and activations.
result Minimal accuracy loss with quantized networks compared to full-precision networks.
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
Kernel Quantization improves CNN compression without sacrificing performance.
problem Efficiently compressing CNN models without significant performance loss.
method Quantizes convolution kernels as the unit, learning a codebook for low-bit indexes.
result Significant compression ratio achieved with minimal accuracy loss.
New method quantizes neural networks for mobile devices.
problem High computational and memory costs of deep neural networks.
method Formulates quantization as a differentiable function.
result Quantization networks outperform state-of-the-art methods.
A new quantization strategy reduces Transformer model size and inference time.
problem Heavy computation load and memory overhead in Transformer models for mobile devices.
method Mixed precision quantization with varying bits per word in embedding blocks.
result 11.8x smaller model size and 3.5x speed up for on-device NMT.
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
problem Dequantization bottleneck in LLM inference on modern AI accelerators.
method MSD decomposes high-precision activations into multiple low-precision components for direct multiplication with quantized weights.
result MSD avoids INT8-to-BF16 weight conversion, reducing dequantization cycles and HBM traffic.
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
SMGD trains low-bit neural networks with memory constraints.
problem Training large neural networks with limited memory.
method Stochastic Markov Gradient Descent (SMGD).
result Encouraging numerical results and theoretical guarantees.
Post-training quantization method using multiple low-precision points achieves higher precision for critical weights.
problem Discretizing pre-trained deep neural networks without re-training.
method Multipoint quantization with efficient greedy selection and adaptive point number.
result Outperforms state-of-the-art methods on ImageNet classification and PASCAL VOC object detection.
Flexible deep learning models for dynamic accuracy and speed trade-offs.
problem Dynamic accuracy and speed trade-offs in real-world applications.
method Training deep neural networks with a new method allowing flexible numerical precision during inference.
result Achieved comparable accuracy to dedicated models trained at the same precision with dynamic precision settings.
Diffusion models improve image compression at low bit-rates.
problem Efficiently compressing images at very low bit-rates.
method Encoding into an embedding, using diffusion models to refine the embedding iteratively.
result Realistic reconstructions can be generated at extremely low bit-rates.
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
Bayesian Bits unifies quantization and pruning through gradient optimization.
problem Joint mixed precision quantization and pruning for efficient neural networks.
method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.
result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.
method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.
result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
New framework reduces LLM complexity by directly finetuning in Boolean domain.
problem Reducing the complexity of large language models (LLMs) while maintaining performance.
method Proposes a novel framework using multi-kernel Boolean parameters for direct finetuning in the Boolean domain.
result Significantly reduces complexity during both finetuning and inference, outperforming recent techniques.
Moniqua improves SGD convergence with quantized communication.
problem Efficiently communicating in decentralized SGD with limited bandwidth.
method Modulo quantized communication in decentralized SGD.
result Moniqua converges at the same rate as full-precision communication with less bits.
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
Deep neural networks are commonly developed and trained in 32-bit floating point format. Significant gains in performance and energy efficiency could be realized by training and inference in numerical formats optimized for deep learning. Despite advances in limited precision inference in recent years, training of neura…
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
problem Quantization challenges in Batch Normalization for QNNs.
method Converts BN to fixed-point operation with shared scale, suitable for hardware.
result Maintains same outputs through rigorous analysis and experiments.
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
problem Growth of quantization errors across layers degrades performance, especially in low-bit regimes.
method Quantization Error Propagation (QEP) framework that explicitly propagates and compensates for quantization errors.
result QEP-enhanced layer-wise PTQ achieves substantially higher accuracy, especially in low-bit regimes.
Quantization-aware training can recover accuracy lost by post-training quantization.
problem Post-training quantization (PTQ) can fail sharply at aggressive bitwidths.
method A unified geometric framework that explains PTQ failure and QAT recovery.
result QAT has a useful bias that steers iterates back into the basin.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
This paper proposes a new method for efficient data compression using Bayesian neural networks.
problem Efficient compression of data represented as functions mapping coordinates to signal values.
method Overfitting variational Bayesian neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding.
result Our method achieves strong performance on image and audio compression while retaining simplicity.
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
This work proposes a complete 8-bit quantization framework for large-scale deep neural networks.
problem Training large-scale deep neural networks with high performance and low memory footprint.
method WAGEUBN framework that quantizes all data paths including weights, activations, gradients, errors, updates, and batch normalization.
result Achieves competitive accuracy on the ImageNet dataset using only 8-bit integers.
Study quantization effects on high-dimensional linear regression learning.
problem Understanding quantization's impact on learning high-dimensional linear regression models.
method Analyzes stochastic gradient descent for high-dimensional linear regression under various quantization targets.
result Establishes precise bounds on excess risk for different quantization schemes.
Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called BLRNet. By embracing stochasticity during tr…
Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable transmission of coded symbols. Traditional encoding and decoding of constrained …
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
New quantization methods improve accuracy of Random Fourier Features.
problem Improving accuracy of Random Fourier Features for machine learning.
method Sigma-Delta and distributed noise-shaping quantization methods for 1-bit and low bit-depth quantization.
result Quantized RFFs allow high accuracy approximation of underlying kernels with polynomial error decay.
Enhances LLM quantization with MDBF, improving perplexity and accuracy.
problem Limited performance of Double Binary Factorization in extreme quantization.
method Introduces Multi-envelope DBF, retaining sign matrices and replacing single envelope with rank-l envelope. result Improves perplexity and zero-shot accuracy over previous binary formats.
New method compresses facial videos using GANs and latent space optimization.
problem Efficiently compressing facial videos at low bit rates.
method Leverages StyleGAN for latent space representation and compression, learns optimal compression through entropy model and perceptual loss.
result Significantly reduces perceptual distortion at low bit rates compared to state-of-the-art codecs.
This work improves DNN weight quantization with ADMM, achieving lossless binarization and reduced search space.
problem Improving DNN model compression and accuracy with low bit quantization.
method Extending ADMM framework for DNN weight quantization with progressive multi-step approach.
result Achieved lossless and fully binarized DNNs with reduced accuracy loss.
Smart Quantization adapts binary and ternary quantization for neural networks.
problem Resource constraints in deploying neural networks on devices with limited resources.
method Adaptive combination of binary and ternary quantization with a regularization function.
result Adapts quantization depth during training to maintain high model accuracy.
Low precision weights, activations, and gradients have been proposed as a way to improve the computational efficiency and memory footprint of deep neural networks. Recently, low precision networks have even shown to be more robust to adversarial attacks. However, typical implementations of low precision DNNs use unifor…
Mixed-precision CA-SGD for generalized linear models on GPUs
problem SGD communication bottleneck
method Mixed-precision CA-SGD
result Matches FP32 SGD loss within 0.5% on various problems
SWALP averages SGD iterates for low-precision training, improving scalability and performance.
problem Improving scalability and performance in low-precision training.
method Averages low-precision SGD iterates with a modified learning rate schedule.
result SWALP matches full-precision SGD performance with 8-bit quantization and converges to optimal solutions.
Paper explores low-precision SGLD for neural networks, reducing costs without sacrificing performance.
problem Infeasibility of low-precision sampling in large-scale scenarios.
method Developed low-precision SGLD with quantization function and full-precision gradient accumulators.
result Low-precision SGLD achieves comparable performance to full-precision SGLD with only 8 bits.
QABBA improves time series storage efficiency while preserving shape information.
problem Efficient storage and shape preservation of time series data.
method Quantized symbolic time series approximation (QABBA) using ABBA technique.
result QABBA achieves a new state-of-the-art on Monash regression dataset.
QPyTorch simplifies low-precision training simulations.
problem Empirical evaluation of low-precision training algorithms.
method Native PyTorch framework with efficient fused-kernel approach.
result Supports various low-precision configurations and rounding options.
This paper improves low-precision sampling using SGHMC for deep learning models.
problem Enhancing training efficiency of deep neural networks with low-precision training.
method Investigates low-precision sampling via Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) for both log-concave and non-log-concave distributions.
result Low-precision SGHMC achieves quadratic improvement in error compared to SGLD for non-log-concave distributions.
New method analyzes accumulation precision in deep learning networks.
problem Lack of precision analysis for accumulation in deep learning training.
method Statistical approach to analyze partial sum accumulations and derive equations for minimum required bits.
result Reduced accumulation precision can lead to loss of information and degraded network quality.
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the num…
Efficient Bitwidth Search optimizes neural network quantization for better performance.
problem Finding optimal bitwidth for weights and activations of each layer efficiently.
method EBS algorithm reusing meta weights and binary decomposition for efficient mixed precision convolution.
result Mixed precision QNN outperforms uniform bitwidth and other techniques on CIFAR10 and ImageNet.
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…
The study examines how class imbalance affects precision-recall curves.
problem Understanding how precision changes with class imbalance ratios.
method Analyzes the relationship between precision, class imbalance ratio, and true/false positive rates.
result Predicts changes in precision-recall curves and other measures with class imbalance ratios.