This paper proposes a method to improve neural network quantization without retraining.
problem Handling outliers in quantized DNN weights and activations.
method Outlier Channel Splitting (OCS) which duplicates channels containing outliers and halves their values.
result OCS outperforms state-of-the-art clipping techniques with minimal overhead.
Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage re…
Paper proposes efficient INT4 quantization for neural networks.
problem Efficient inference on limited hardware resources for large neural networks.
method Formalizes quantization as MMSE problem, optimizes constrained MSE at each layer, partitions parameters, uses multiple quantized tensors.
result 4-bit integer (INT4) quantization yields state-of-the-art results with minimal accuracy loss.
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
problem Growth of quantization errors across layers degrades performance, especially in low-bit regimes.
method Quantization Error Propagation (QEP) framework that explicitly propagates and compensates for quantization errors.
result QEP-enhanced layer-wise PTQ achieves substantially higher accuracy, especially in low-bit regimes.
ESS-Flow guides flow models without retraining, using Bayesian inference in source space.
problem Training flow models on paired data for conditional generation or sample production.
method Gradient-free Bayesian inference in source space using Elliptical Slice Sampling.
result Effective in diverse tasks including material design and protein structure prediction.
New method improves accuracy of quantized neural networks.
problem Accuracy drop in quantized neural networks, especially MobileNet family.
method Weight equalizing shift scaler, binary shifting to recover output range.
result Top-1 accuracy improved from 0.1% to 69.78% ~ 70.96% in MobileNets.
Quantization-aware training can recover accuracy lost by post-training quantization.
problem Post-training quantization (PTQ) can fail sharply at aggressive bitwidths.
method A unified geometric framework that explains PTQ failure and QAT recovery.
result QAT has a useful bias that steers iterates back into the basin.
Model compression has been introduced to reduce the required hardware resources while maintaining the model accuracy. Lots of techniques for model compression, such as pruning, quantization, and low-rank approximation, have been suggested along with different inference implementation characteristics. Adopting model com…
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
SQWA improves low-precision DNNs with model averaging and quantization.
problem Designing good generalization DNNs with quantized weights.
method Floating-point model training, direct quantization, multiple low-precision models, weight averaging, re-quantization, fine-tuning, loss visualization.
result State-of-the-art results for 2-bit QDNNs on CIFAR-100 and ImageNet datasets.
Traditional classification algorithms assume that training and test data come from similar distributions. This assumption is violated in adversarial settings, where malicious actors modify instances to evade detection. A number of custom methods have been developed for both adversarial evasion attacks and robust learni…
Network Implosion reduces ResNet layers without accuracy loss.
problem High computation costs in Residual Networks.
method Static layer pruning and retraining to erase unimportant layers.
result Reduces ResNet layers by 24.00-42.86% without accuracy drop.
The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest in compression. Recent work by Han et al. (2015a) propose a pipeline that involve…
Efficiently removes specific data subsets without retraining for GDPR compliance.
problem Efficiently removing specific data subsets to comply with GDPR regulations.
method Statistical framework for machine unlearning with minimax optimality for squared loss.
result Developed Unlearning Least Squares (ULS) achieving minimax optimality for estimating model parameters.
Simplified neural network quantization speeds up process by 8 epochs.
problem Efficient quantization of neural networks for mobile devices.
method Trained scale factors for quantization thresholds.
result Significant reduction in training time and model size.
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
This paper introduces a differentiable, scalable quantization method for neural networks.
problem Previous quantization methods lacked differentiability and scalability.
method The approach is differentiable and scalable, using bit-shifting and logarithmic quantization.
result The method achieves comparable accuracy to state-of-the-art approaches with less training time and lower inference cost.
This paper enables deep network inference on microcontrollers with improved accuracy and reduced memory usage.
problem Deploying deep networks on resource-constrained edge-devices with low memory and computational constraints.
method Mixed low-bitwidth compression, rule-based iterative procedure for bit precision determination, quantization-aware retraining, and integer-only model conversion.
result Improved Top1 accuracy of 68% on a 2MB FLASH memory STM32H7 microcontroller, 8% higher than 8-bit implementations.
Post-training quantization method using multiple low-precision points achieves higher precision for critical weights.
problem Discretizing pre-trained deep neural networks without re-training.
method Multipoint quantization with efficient greedy selection and adaptive point number.
result Outperforms state-of-the-art methods on ImageNet classification and PASCAL VOC object detection.
Last layer retraining improves robustness to spurious correlations without high computational costs.
problem Neural networks can rely on spurious features like backgrounds for predictions.
method Simple last layer retraining on large models.
result Last layer retraining matches or outperforms state-of-the-art approaches on spurious correlation benchmarks.
GOBO compresses 99.9% of BERT model parameters to 3 bits, improving inference efficiency.
problem Efficient execution of attention-based NLP models, especially in terms of latency and energy consumption.
method GOBO quantizes 32-bit floating-point parameters to 3 bits without fine-tuning, using hardware compression and co-designed architectures.
result GOBO maintains model accuracy while significantly reducing inference latency and energy consumption.
MCQ uses Monte Carlo methods to efficiently quantize neural networks without re-training.
problem Efficiently quantizing neural networks for lower power consumption.
method Importance sampling for quantization of weights and activations.
result Minimal accuracy loss with quantized networks compared to full-precision networks.
FullyQT quantizes Transformer models to 8-bit precision without sacrificing translation quality.
problem Reducing computational costs of neural machine translation methods without performance loss.
method Proposes FullyQT, an all-inclusive quantization strategy for the Transformer.
result Achieves state-of-the-art quantization results, maintaining or improving translation quality.
New method protects neural networks from adversarial attacks without generating adversarial examples.
problem Vulnerability of neural networks to adversarial examples.
method Entropic retraining, inspired by information theory.
result Significant increase in NNs' security and robustness.
Decomposable-Net compresses neural networks without retraining for various sizes.
problem Performance degradation when changing model size after training.
method Decomposes weight matrices via SVD and adjusts ranks for different sizes.
result Maintains and improves performance across multiple model sizes.
Demon aligns diffusion models without retraining or backpropagation.
problem Aligning diffusion models with user preferences.
method Stochastic optimization to control noise distribution.
result Significantly improves aesthetics scores for text-to-image generation.
Algorithm improves model performance on shifted concepts without retraining.
problem Improving model performance on shifted concepts with limited source data.
method Model consolidation of intermediate internal distributions after adaptation.
result Effective improvement in model performance on shifted concepts.
NEMO framework quantizes DNNs for efficient deployment.
problem Efficient deployment of quantized DNNs.
method Formal framework for quantizing DNN layers, focusing on IntegerDeployable representation.
result Quantized DNNs can be deployed using only integers.
The paper proposes a dataflow-based approach to reduce energy consumption in deep neural networks without sacrificing performance.
problem Reducing energy consumption in deep neural networks without performance drop.
method Dataflow-based joint quantization of weights and activations.
result Joint quantization improves performance and reduces energy consumption.
Quantizes functions on Kähler manifolds without formal deformation.
problem Deforming smooth functions on Kähler manifolds to non-formal quantization.
method Using Fedosov connections and prequantum line bundles.
result Quantizable functions form a sheaf of twisted differential operators.
A new method quantizes LSTM gate parameters without performance loss.
problem Quantization loss in LSTM gate parameters without performance degradation.
method Lossy quantization of gate parameters during training, weight parameters adjust to offset quantization loss.
result F1 score decreased by only 0.7% on Named Entity Recognition dataset.
New method uses neural tangent kernel for efficient active learning.
problem Efficiently approximating deep learning's look-ahead selection criteria.
method Approximates retraining with neural tangent kernel for active learning.
result Approximation works asymptotically and enables sequential active learning.
Quantum computing speeds up neural network training and retraining.
problem Inefficient classical training and retraining of neural networks.
method Adiabatic quantum computing to optimize Kolmogorov-Arnold Networks using Bezier curves.
result Quantum optimization achieves 100x faster retraining compared to classical methods.
Gradient ascent method successfully removes specific data points from neural networks without retraining.
problem Addressing privacy and ethical concerns by removing specific data points from trained models.
method Gradient ascent approach to unlearning, leveraging the implicit bias of gradient descent towards margin maximization conditions.
result Gradient ascent method can successfully unlearn specific data points from two-layer ReLU neural networks without retraining.
ICQ improves high-dimensional similarity search without sacrificing precision.
problem High-dimensional similarity search is computationally expensive.
method Interleaved Composite Quantization (ICQ) reduces code length and quantization error.
result ICQ achieves fast similarity search without using shorter codes.
Quantized Stochastic Primal-Dual Methods for Distributed Optimization
problem Distributed optimization with stochastic gradients and finite-bit communication
method q-PDGD, a quantized stochastic primal-dual method
result Linear contraction to an explicit neighborhood under RSI, O(1/k) convergence under PL inequality
DSNAS optimizes neural architecture and parameters in one step.
problem Poor correlation between architecture performance in two stages of NAS methods.
method Task-specific end-to-end approach with DSNAS framework.
result DSNAS discovers comparable accuracy networks in less time.
A new method quantizes neural networks to low-precision without STE, improving accuracy.
problem Quantization of neural networks to low-precision without a complete theoretical understanding.
method Alpha-blending (AB) using stochastic gradient descent (SGD) to quantize weights and gradually increase the coefficient α. result Improves top-1 accuracy by 0.9% on 1-bit BinaryNet, 0.82% on 8-bit MobileNet v1, and 2.93% on 4-bit ResNet_50 v1/2 compared to STE.
In this paper, using blow-up analysis, we prove a quantization result for an elliptic equation with critical exponential growth on compact Riemannian surface without boundary. Similar results for Euclidean space were obtained by Adimurthi-Struwe \cite{Adi-Stru}, Druet \cite{Druet}, Lamm-Robert-Struwe \cite{L-R-S}, Mart…
Kernel Quantization improves CNN compression without sacrificing performance.
problem Efficiently compressing CNN models without significant performance loss.
method Quantizes convolution kernels as the unit, learning a codebook for low-bit indexes.
result Significant compression ratio achieved with minimal accuracy loss.
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
This paper proposes an automatic neural network compression method.
problem Reducing resource requirements for deep neural networks on resource-constrained devices.
method Jointly prunes and quantizes neural networks without manual hyper-parameter tuning.
result Significant reduction in model size with minimal accuracy loss.
APoT quantization improves neural network efficiency and accuracy.
problem Efficiently quantizing weights and activations in neural networks.
method Constraining quantization levels as sums of Powers-of-Two terms, applying reparameterization, and weight normalization.
result 4-bit quantized ResNet-50 achieves 76.6% top-1 accuracy, 22% computational cost reduction.
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
Quantized Epoch-SGD reduces communication in distributed learning.
problem High communication cost in SGD-based distributed learning.
method Quantizes parameters with variance reduction for efficient distributed learning.
result QESGD achieves better performance with less communication compared to other methods.
A new method defends neural networks from adversarial attacks.
problem Adversarial attacks on quantized neural networks.
method Defensive Quantization (DQ) method to control Lipschitz constant.
result DQ method defends neural networks from adversarial attacks and improves accuracy.
IAL uses interactive learning to improve model performance with minimal human feedback.
problem Overfitting and high human interaction cost in training neural networks.
method IAL framework with NAP attention generator and reranking algorithm.
result IAL significantly outperforms baselines with less retraining and human interaction.
Dynamic model pruning improves performance on deep neural networks without retraining.
problem High memory and latency requirements for deep neural networks on low-end devices.
method Dynamic allocation of sparsity pattern and feedback signal to reactivate pruned weights.
result Sparse models achieve state-of-the-art performance with no additional retraining.