Similar to convolution neural networks, recurrent neural networks (RNNs) typically suffer from over-parameterization. Quantizing bit-widths of weights and activations results in runtime efficiency on hardware, yet it often comes at the cost of reduced accuracy. This paper proposes a quantization approach that increases…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “bit width reduction”
Bayesian Bits unifies quantization and pruning through gradient optimization.
problem Joint mixed precision quantization and pruning for efficient neural networks.
method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.
result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.
BIL allows binary input data in CNNs, improving performance on multimodal datasets.
problem Efficient execution of CNNs on edge devices with reduced bit width.
method BIL concept that learns bit-specific binary weights for binary input data.
result BIL outperforms full precision weights by 1.92% on multimodal datasets.
Improves matrix multiplication throughput for asymmetric bit-width operands.
problem Matrix multiplications between asymmetric bit-width operands, especially 8- and 4-bit, are not efficiently handled by existing SIMD instructions.
method Proposes a new SIMD matrix multiplication instruction that uses mixed precision on inputs (8- and 4-bit) and accumulates into 16-bit output, improving throughput.
result Offers 2x improvement in throughput compared to existing symmetric-operand-size instructions, with negligible overflow.
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
This work optimizes neural network bit-width and layer-width for efficiency.
problem Efficient optimization of deep neural networks for reduced size and computational demands.
method Cluster-based tree-structured Parzen estimator for surrogate modeling, Hessian-based pruning for parameter reduction.
result 20% decrease in model size with 12x reduction in search time compared to existing methods.
Paper explores reducing precision in SVM for faster text classification.
problem Efficiency in multi-class text classification training.
method Comparison of SVM trained with reduced precision (16-bit, half) vs original.
result Reduced precision training maintains text classification accuracy.
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
Proposes a robust neural network quantization method.
problem Training model's dependency on specific quantization methods.
method Intrinsic robustness to various quantization processes.
result Single model capable of operating at various bit-widths and policies.
New techniques save bits in image compression with upsampling.
problem Lack of context dependence in current image compression methods with upsampling.
method Simple, inexpensive techniques exploiting context to predict Laplace distribution parameters.
result Average savings of 0.645 bits per difference, up to 1.489 bits.
MCQ uses Monte Carlo methods to efficiently quantize neural networks without re-training.
problem Efficiently quantizing neural networks for lower power consumption.
method Importance sampling for quantization of weights and activations.
result Minimal accuracy loss with quantized networks compared to full-precision networks.
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization. By training quantization-ready networks, our approach enables storing a single set of weights that can be quantized on-deman…
Study finds optimal learning rate schedules for sub-100M quantization-aware training across bit-widths.
problem Optimal learning rate schedules for quantization-aware training depend on bit-width.
method Factorial grid testing over bit-width, warmdown fraction, LR magnitude, model size, and seed.
result INT6 QAT requires a different schedule than higher-precision training, falsifying the primary hypothesis.
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
Topological data analysis classifies encrypted bits with success.
problem Classifying encrypted data with traditional machine learning methods.
method Persistent homology for generating topological features, machine learning pipeline.
result Successfully classifies encrypted data, outperforming classical models.
Improved neural quantization reduces accuracy loss to less than 1% with 4-bit weights.
problem Reducing accuracy loss in neural quantization below 8-bits.
method Layer-wise calibration and integer programming to optimize bit-width allocation.
result Less than 1% accuracy degradation with 4-bit weights and activations.
AdaptivFloat improves deep learning inference accuracy at low precision.
problem Low precision quantization issues in deep learning inference.
method Dynamic floating-point representation with adaptive clipping.
result Consistently higher inference accuracy at low precision compared to other methods.
New method analyzes accumulation precision in deep learning networks.
problem Lack of precision analysis for accumulation in deep learning training.
method Statistical approach to analyze partial sum accumulations and derive equations for minimum required bits.
result Reduced accumulation precision can lead to loss of information and degraded network quality.
Deep neural networks are commonly developed and trained in 32-bit floating point format. Significant gains in performance and energy efficiency could be realized by training and inference in numerical formats optimized for deep learning. Despite advances in limited precision inference in recent years, training of neura…
In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large memory consumption, which may not be affordable for mobile platforms. Deep model q…
We present a theoretical analysis and empirical evaluations of a novel set of techniques for computational cost reduction of classifiers that are based on learned transform and soft-threshold. By modifying optimization procedures for dictionary and classifier training, as well as the resulting dictionary entries, our t…
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
problem Estimating sparse signals from binary measurements.
method Non-convex sparsity-constrained program with one-shot hard thresholding.
result Simple algorithm produces accurate signal approximation with high probability.
Bit-slice sparsity improves ReRAM-based DNN acceleration.
problem Limited ADC power and area constraints in ReRAM-based DNN accelerators.
method Proposed bit-slice L1 algorithm to induce sparsity during training.
result 2x sparsity improvement compared to previous methods.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.
method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.
result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.
This study analyzes quantization in deep learning models using statistical physics methods.
problem The computational resource requirements for large-scale data analysis models.
method Typical case analysis from statistical physics, specifically the replica method.
result Optimal quantization width minimizes error and delays overfitting.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
This work proposes a complete 8-bit quantization framework for large-scale deep neural networks.
problem Training large-scale deep neural networks with high performance and low memory footprint.
method WAGEUBN framework that quantizes all data paths including weights, activations, gradients, errors, updates, and batch normalization.
result Achieves competitive accuracy on the ImageNet dataset using only 8-bit integers.
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
A new quantization strategy reduces Transformer model size and inference time.
problem Heavy computation load and memory overhead in Transformer models for mobile devices.
method Mixed precision quantization with varying bits per word in embedding blocks.
result 11.8x smaller model size and 3.5x speed up for on-device NMT.
This work reduces model size by 86.11% for recommender systems using 4-bit quantization.
problem Large memory consumption in embedding vectors for recommender systems.
method Post-training 4-bit quantization on embedding tables, including row-wise uniform quantization and codebook-based quantization.
result Consistently reduces accuracy degradation while significantly reducing model size.
4-bit quantization reduces U-Net memory by 8x with minimal accuracy loss.
problem Reducing memory and computation time in deep learning models.
method Fixed-point quantization of U-Net architecture.
result 8x reduction in memory usage with minimal accuracy loss.
QABBA improves time series storage efficiency while preserving shape information.
problem Efficient storage and shape preservation of time series data.
method Quantized symbolic time series approximation (QABBA) using ABBA technique.
result QABBA achieves a new state-of-the-art on Monash regression dataset.
Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatchcs.LG
New method ensures consistent inference across different tensor parallel sizes for large language models.
problem Non-deterministic inference in large language models due to inconsistent reduction orders across GPUs.
method Tree-Based Invariant Kernels (TBIK) that align intra- and inter-GPU reduction orders through a unified hierarchical binary tree structure.
result Bit-wise identical results across different tensor parallel sizes for RL training.
Paper proposes a new detector for SSDs with ICI, reducing bit error rates significantly.
problem Reducing bit error rate in SSDs with inter-cell interference.
method Proposes a novel channel model and uses sum-product algorithm for accurate posterior distributions, then passes them to a soft LDPC decoder. Introduces an iterative scheme to improve performance in quantized cases.
result Significant reduction in bit error rate, reaching 99.99% in simulations with strong diagonal coupling.
DJPQ optimizes neural network pruning and quantization for hardware efficiency.
problem Efficiently compress neural networks for hardware inference.
method Joint gradient-based optimization of pruning and quantization into a differentiable loss function.
result Significant reduction in Bit-Operations (BOPs) with minimal accuracy loss.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
CoDeQ simplifies joint model compression by integrating pruning and quantization.
problem Joint pruning and quantization methods are complex and require additional procedures.
method CoDeQ uses a dead-zone quantizer to directly induce sparsity and learn quantization parameters.
result CoDeQ achieves high sparsity and low-precision accuracy with minimal bit operations.
New method compresses neural networks using random code, improving efficiency.
problem Large memory footprint of deep neural networks.
method Training a variational distribution over weights, encoding using Kullback-Leibler divergence.
result Achieves state-of-the-art compression rates and test performance.
Flexible deep learning models for dynamic accuracy and speed trade-offs.
problem Dynamic accuracy and speed trade-offs in real-world applications.
method Training deep neural networks with a new method allowing flexible numerical precision during inference.
result Achieved comparable accuracy to dedicated models trained at the same precision with dynamic precision settings.
Quantized Neural Networks (QNNs) are often used to improve network efficiency during the inference phase, i.e. after the network has been trained. Extensive research in the field suggests many different quantization schemes. Still, the number of bits required, as well as the best quantization scheme, are yet unknown. O…
Practically Perfectcs.AI
The property of perfectness plays an important role in the theory of Bayesian networks. First, the existence of perfect distributions for arbitrary sets of variables and directed acyclic graphs implies that various methods for reading independence from the structure of the graph (e.g., Pearl, 1988; Lauritzen, Dawid, La…
Data-free quantization method improves model performance without fine-tuning.
problem Efficient quantization of deep neural networks without performance loss.
method Equalizing weight ranges and correcting biases to achieve 8-bit quantization.
result Achieves state-of-the-art quantized model performance on various architectures.
Paper proves a new lower bound on calibration error for binary prediction.
problem Proving a strong lower bound on calibration error for binary prediction.
method Developed two new techniques: early stopping and sidestepping.
result Proves an lower bound on calibration error.
New techniques improve 16-bit training accuracy without 32-bit units.
problem Training deep learning models with only 16-bit floating-point units.
method Studied BFloat16 units and applied stochastic rounding and Kahan summation techniques.
result Up to 7% absolute validation accuracy gain in 16-bit-FPU training.
This paper automates the creation of low precision deep learning operators for mobile devices.
problem Deploying large deep learning models on low power devices is challenging due to limited compute capabilities and energy budgets.
method Introduces a workflow to generate high-performance low precision deep learning operators for multiple CPU architectures, including optimizations like memory tiling and vectorization.
result 1-bit and 2-bit convolutions achieve up to 16x and 2.3x speedups over 16-bit integer baselines on ARM Cortex-A53 CPU.
A new method reduces communication in distributed learning by 32x.
problem High communication costs in distributed machine learning.
method Distributed compressed SGD with Nesterov's momentum and blockwise compression.
result Achieves 32x reduction in communication cost and converges as fast as full-precision SGD.
Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called BLRNet. By embracing stochasticity during tr…