A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Low-bit training framework reduces energy consumption in CNNs.
SMGD trains low-bit neural networks with memory constraints.
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…
Kernel Quantization improves CNN compression without sacrificing performance.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with the best possible perceptual quality. In this work we demonstrate that a neural network architecture based on VQ-VAE with a WaveNet decoder …
Although deep neural networks are highly effective, their high computational and memory costs severely challenge their applications on portable devices. As a consequence, low-bit quantization, which converts a full-precision neural network into a low-bitwidth integer version, has been an active and promising research t…
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
Diffusion models improve image compression at low bit-rates.
Meta learning optimizes neural network quantization for efficient inference.
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …
Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called BLRNet. By embracing stochasticity during tr…
New framework reduces LLM complexity by directly finetuning in Boolean domain.
Deep neural networks are commonly developed and trained in 32-bit floating point format. Significant gains in performance and energy efficiency could be realized by training and inference in numerical formats optimized for deep learning. Despite advances in limited precision inference in recent years, training of neura…
A new quantization strategy reduces Transformer model size and inference time.
Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary…
Post-training quantization method using multiple low-precision points achieves higher precision for critical weights.
We present any-precision deep neural networks (DNNs), which are trained with a new method that allows the learned DNNs to be flexible in numerical precision during inference. The same model in runtime can be flexibly and directly set to different bit-widths, by truncating the least significant bits, to support dynamic …
Moniqua improves SGD convergence with quantized communication.
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
New method compresses facial videos using GANs and latent space optimization.
Quantization-aware training can recover accuracy lost by post-training quantization.
Deep neural network (DNN) quantization converting floating-point (FP) data in the network to integers (INT) is an effective way to shrink the model size for memory saving and simplify the operations for compute acceleration. Recently, researches on DNN quantization develop from inference to training, laying a foundatio…
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable transmission of coded symbols. Traditional encoding and decoding of constrained …
Study quantization effects on high-dimensional linear regression learning.
Paper introduces adversarial lossy compression for video artifacts reduction.
Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantization with the advanced optimization algorithm ADMM (Alternating Direction Methods of Multipliers) achieves one of state-of-art results in we…
New quantization methods improve accuracy of Random Fourier Features.
Enhances LLM quantization with MDBF, improving perplexity and accuracy.
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
QABBA improves time series storage efficiency while preserving shape information.
This paper proposes a new method for efficient data compression using Bayesian neural networks.
Bayesian Bits unifies quantization and pruning through gradient optimization.
Self-training outperforms pre-training on COCO object detection and segmentation datasets.
Un-trained neural networks outperform trained methods in MRI reconstruction.
We propose \emph{Taylorized training} as an initiative towards better understanding neural network training at finite width. Taylorized training involves training the -th order Taylor expansion of the neural network at initialization, and is a principled extension of linearized training---a recently proposed theory …
Meta-learning performance depends on train-validation split type.
New method reveals how training data influence diffusion model outputs.
Paper shows adversarial training can be fooled by new type of noise.
New MIP methods improve training of integer-valued neural networks.
Free adversarial training reduces the generalization gap compared to vanilla method.
Making neural networks robust against adversarial inputs has resulted in an arms race between new defenses and attacks. The most promising defenses, adversarially robust training and verifiably robust training, have limitations that restrict their practical applications. The adversarially robust training only makes the…
learn2mix trains neural nets faster by adjusting class proportions dynamically.
Loss-guided training accelerates node embedding methods on graphs.