Paper proposes quantizing RNNs to save space and power.
problem Over-parameterization in RNNs leads to inefficiency.
method Increases bit-width reduction for accuracy preservation.
result RNNs can maintain accuracy with reduced precision.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “bit-width reduction”
BIL allows binary input data in CNNs, improving performance on multimodal datasets.
problem Efficient execution of CNNs on edge devices with reduced bit width.
method BIL concept that learns bit-specific binary weights for binary input data.
result BIL outperforms full precision weights by 1.92% on multimodal datasets.
Bayesian Bits unifies quantization and pruning through gradient optimization.
problem Joint mixed precision quantization and pruning for efficient neural networks.
method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.
result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.
Improves matrix multiplication throughput for asymmetric bit-width operands.
problem Matrix multiplications between asymmetric bit-width operands, especially 8- and 4-bit, are not efficiently handled by existing SIMD instructions.
method Proposes a new SIMD matrix multiplication instruction that uses mixed precision on inputs (8- and 4-bit) and accumulates into 16-bit output, improving throughput.
result Offers 2x improvement in throughput compared to existing symmetric-operand-size instructions, with negligible overflow.
Proposes a robust neural network quantization method.
problem Training model's dependency on specific quantization methods.
method Intrinsic robustness to various quantization processes.
result Single model capable of operating at various bit-widths and policies.
MCQ uses Monte Carlo methods to efficiently quantize neural networks without re-training.
problem Efficiently quantizing neural networks for lower power consumption.
method Importance sampling for quantization of weights and activations.
result Minimal accuracy loss with quantized networks compared to full-precision networks.
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization. By training quantization-ready networks, our approach enables storing a single set of weights that can be quantized on-deman…
Study finds optimal learning rate schedules for sub-100M quantization-aware training across bit-widths.
problem Optimal learning rate schedules for quantization-aware training depend on bit-width.
method Factorial grid testing over bit-width, warmdown fraction, LR magnitude, model size, and seed.
result INT6 QAT requires a different schedule than higher-precision training, falsifying the primary hypothesis.
Paper explores reducing precision in SVM for faster text classification.
problem Efficiency in multi-class text classification training.
method Comparison of SVM trained with reduced precision (16-bit, half) vs original.
result Reduced precision training maintains text classification accuracy.
This work optimizes neural network bit-width and layer-width for efficiency.
problem Efficient optimization of deep neural networks for reduced size and computational demands.
method Cluster-based tree-structured Parzen estimator for surrogate modeling, Hessian-based pruning for parameter reduction.
result 20% decrease in model size with 12x reduction in search time compared to existing methods.
New algorithms improve scalar quantization by optimizing sparse least squares.
problem Improving efficiency and accuracy of scalar quantization for neural networks.
method Sparse least square optimization, iterative and clustering-based methods.
result Proposed algorithms outperform existing methods, especially in bit-width reduction scenarios.
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
BCGD algorithm improves training of quantized neural networks.
problem Training quantized deep neural networks at low bit-widths.
method Introduces coarse gradient descent and blended correction for training.
result BCGD achieves high accuracy in quantized neural networks.
Optimizes deep neural network quantization for mobile devices.
problem High computational costs and memory consumption in deep neural networks.
method Proposes a measurement to estimate quantization errors and an optimization process for finding optimal bit-widths.
result Achieves 20-40% higher compression rate compared to previous methods at the same accuracy.
New method analyzes accumulation precision in deep learning networks.
problem Lack of precision analysis for accumulation in deep learning training.
method Statistical approach to analyze partial sum accumulations and derive equations for minimum required bits.
result Reduced accumulation precision can lead to loss of information and degraded network quality.
AdaptivFloat improves deep learning inference accuracy at low precision.
problem Low precision quantization issues in deep learning inference.
method Dynamic floating-point representation with adaptive clipping.
result Consistently higher inference accuracy at low precision compared to other methods.
Improved neural quantization reduces accuracy loss to less than 1% with 4-bit weights.
problem Reducing accuracy loss in neural quantization below 8-bits.
method Layer-wise calibration and integer programming to optimize bit-width allocation.
result Less than 1% accuracy degradation with 4-bit weights and activations.
QABBA improves time series storage efficiency while preserving shape information.
problem Efficient storage and shape preservation of time series data.
method Quantized symbolic time series approximation (QABBA) using ABBA technique.
result QABBA achieves a new state-of-the-art on Monash regression dataset.
Unified framework LPCD optimizes quantization of complex submodules.
problem Quantization of complex submodules in neural networks.
method Layer-Projected Coordinate Descent (LPCD) for quantizing arbitrary submodules.
result LPCD enhances both layer-wise PTQ methods and existing submodule approaches.
Probabilistic BLRNet uses binary weights and activations for efficient neural networks.
problem Efficiently training and deploying deep neural networks with limited memory and compute.
method Probabilistic training method for binary weights and activations, introducing stochastic operations.
result BLRNet achieves performance comparable to full-precision networks while using fewer bits.
New insights into quantized neural networks reveal learning dynamics and generalization errors.
problem Understanding the impact of quantization hyperparameters on learning dynamics in high-dimensional models.
method Theoretical analysis and fixed-point analysis of STE dynamics in quantized models.
result STE training in quantized models converges to a plateau followed by a sharp drop in generalization error, influenced by quantization range.
Investigates how reduced precision affects deep neural networks.
problem Predicting sensitivity of DNNs to reduced numerical precision.
method Emulates arbitrary bit-width using truncation method after each batch.
result Shows impact of model parameters on training accuracy.
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
New framework reduces cost of financial option pricing simulations on FPGAs.
problem Efficiently simulate financial option pricing with reduced computational cost.
method Nested MLMC framework with low precision calculations on FPGAs.
result Higher computational savings compared to existing mixed-precision MLMC frameworks.
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
problem Quantization challenges in Batch Normalization for QNNs.
method Converts BN to fixed-point operation with shared scale, suitable for hardware.
result Maintains same outputs through rigorous analysis and experiments.
Flexpoint improves deep learning training efficiency by using adaptive 16-bit format.
problem Training deep neural networks in low bit-width formats is challenging.
method Flexpoint uses a shared exponent dynamically adjusted to minimize overflows and maximize dynamic range.
result 16-bit Flexpoint tensors closely match 32-bit floating point in training deep networks without tuning.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
Flexible deep learning models for dynamic accuracy and speed trade-offs.
problem Dynamic accuracy and speed trade-offs in real-world applications.
method Training deep neural networks with a new method allowing flexible numerical precision during inference.
result Achieved comparable accuracy to dedicated models trained at the same precision with dynamic precision settings.
CoDeQ simplifies joint model compression by integrating pruning and quantization.
problem Joint pruning and quantization methods are complex and require additional procedures.
method CoDeQ uses a dead-zone quantizer to directly induce sparsity and learn quantization parameters.
result CoDeQ achieves high sparsity and low-precision accuracy with minimal bit operations.
Role-wise data augmentation improves knowledge distillation effectiveness.
problem Existing knowledge distillation methods fail to utilize the full potential of teacher-student data interaction.
method Design and implement data augmentation agents with distinct roles for teacher and student.
result Specially tailored data points enhance the demonstration of teacher's knowledge to the student.
This paper identifies knot projections with reductivity two.
problem Determining knot projections with a specific reductivity level.
method Examined four types of reductivity (Seifert type splice, non-Seifert type splice, recursively, simultaneously) and their combinations.
result Identified all knot projections with reductivity two for the four definitions.
Completes reduction scheme in Lagrange-Poincaré category.
problem Lagrangian reduction by stages in the whole category.
method Analyzes Noether theorem, Hamiltonian reduction, geometric aspects.
result Affirmative answer to open question of Lagrangian reduction.
This paper classifies instantons with closed reductions and provides examples of non-closed reductions.
problem Understanding the geometry of toric Kähler instantons with and without closed reductions.
method Sharp geometric criteria and examples of instantons with different reduction types.
result Established geometric criteria for closed reductions and classified asymptotic geometries.
We consider locally conformal Kaehler geometry as an equivariant (homothetic) Kaehler geometry: a locally conformal Kaehler manifold is, up to equivalence, a pair (K,Γ) where K is a Kaehler manifold and Γa discrete Lie group of biholomorphic homotheties acting freely and properly discontinuously. We define a new invari…
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.
method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.
result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.
Classifies 7- and 8-dimensional naturally reductive spaces.
problem Classifying naturally reductive spaces in 7 and 8 dimensions.
method Combines structure theory and new construction methods.
result Complete classification of 7- and 8-dimensional naturally reductive spaces.
Abstract revisits Kähler reduction using GIT, generalizing results.
problem Generalizing Kähler reduction results to the generalized setting.
method Geometric invariant theory approach to generalized Kähler reduction.
result Many well-known Kähler reduction results can be generalized.
In this paper we describe Routhian reduction as a special case of standard symplectic reduction, also called Marsden-Weinstein reduction. We use this correspondence to present a generalization of Routhian reduction for quasi-invariant Lagrangians, i.e. Lagrangians that are invariant up to a total time derivative. We sh…
Two reduction schemes for symplectic manifolds are shown equivalent.
problem Reduction of Hamiltonian systems on exact symplectic manifolds.
method Modified Marsden-Meyer-Weinstein reduction theorem for exact symplectic manifolds and contact manifolds.
result Reduction schemes are equivalent for exact symplectic manifolds and energy hypersurfaces.
Study extends Kobayashi's method to non-reductive subgroups for homogeneous spaces.
problem Existence of compact Clifford-Klein forms in homogeneous spaces.
method Extend Kobayashi's method to non-reductive subgroups and compare Cartan projections and non-compact dimensions.
result Examples of homogeneous spaces without compact Clifford-Klein forms.
Dirac optimal reductionmath.DG
The purpose of this paper is to generalize the regular Optimal Reduction Theorem to general proper Dirac actions, formulated both in terms of point and orbit reduction. A comparison to general standard singular Dirac reduction is given emphasizing the desingularization role played by optimal reduction.
We show that the contact reduction can be specialized to Sasakian manifolds. We link this Sasakian reduction to Kähler reduction by considering the Kähler cone over a Sasakian manifold. We present examples of Sasakian manifolds obtained by reduction of standard Sasakian spheres.
Naturally reductive metricsmath.DG
Study characterizes naturally reductive metrics on homogeneous manifolds.
problem Characterizing naturally reductive metrics on homogeneous manifolds.
method Characterization through local -products and equivalence of properties.
result Explicit flag curvature formula for naturally reductive metrics.
Abstract: Generalized reduction methods for symmetries in graded geometry.
problem Generalized reduction of symmetries in graded geometry.
method Graded symplectic reduction for Courant, Dirac, and generalized complex structures.
result Systematic recovery of reduction schemes for exact cases.
In this note we give conditions which ensure the reduction of a symplectic connection in the process of a Marsden-Weinstein reduction and of the reduction of a presymplectic manifold.
Introduces symplectic reduction in nonrational toric geometry.
problem Symplectic reduction in nonrational toric geometry.
method Specializes to nonrational toric geometry and rational case for symplectic reduction.
result Symplectic reduction for nonrational Lie subgroups.
This work introduces a unified approach to the reduction of Poisson manifolds using their description by graded symplectic manifolds. This yields a generalization of the classical Poisson reduction by distributions (Marsden-Ratiu reduction). Further it allows one to construct actions of strict Lie 2-groups and to descr…