A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper proposes a 1-bit quantization scheme for high-dimensional statistical estimation.
New protocols show 1-bit mean estimation can be order-optimal without interaction.
Moniqua improves SGD convergence with quantized communication.
Paper offers robust recovery for 1-bit sensing with partial Gaussian circulant matrices.
This paper introduces a differentiable, scalable quantization method for neural networks.
Binary Iterative Hard Thresholding converges with optimal number of 1-bit measurements.
We present DeepFPC, a novel deep neural network designed by unfolding the iterations of the fixed-point continuation algorithm with one-sided l1-norm (FPC-l1), which has been proposed for solving the 1-bit compressed sensing problem. The network architecture resembles that of deep residual learning and incorporates pri…
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.
Paper develops an efficient mean estimator for 1-bit communication constraints.
Unified framework for uniform signal recovery in nonlinear GCS with 1-bit/quantized measurements.
New quantization methods improve accuracy of Random Fourier Features.
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage re…
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
Enhances LLM quantization with MDBF, improving perplexity and accuracy.
The recent framework of compressive statistical learning aims at designing tractable learning algorithms that use only a heavily compressed representation-or sketch-of massive datasets. Compressive K-Means (CKM) is such a method: it estimates the centroids of data clusters from pooled, non-linear, random signatures of …
The Straight-Through Estimator (STE) is widely used for back-propagating gradients through the quantization function, but the STE technique lacks a complete theoretical understanding. We propose an alternative methodology called alpha-blending (AB), which quantizes neural networks to low-precision using stochastic grad…
Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations. Despite model size reduction, achieving performance enhancement on devices is, however, still challenging mainly due to the irregular representations of spa…
Study on recovering sparse linear classifiers from mixed binary responses.
Study 1-bit compressive sensing with generative models, improving recovery accuracy.
Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributed neural network training. Using 1-bit quantization, signSGD with majority vote …
A new method for 1-bit matrix completion that is faster and more accurate.
Matrix completion has a long-time history of usage as the core technique of recommender systems. In particular, 1-bit matrix completion, which considers the prediction as a ``Recommended'' or ``Not Recommended'' question, has proved its significance and validity in the field. However, while customers and products aggre…
New method predicts binary matrix entries using empirical Bayes and low-rank structure.
The paper develops a robust signal estimation method for noisy measurements from generative models.
Study quantile reward identification with 1-bit feedback constraints.
Study improves fractional posterior for 1-bit matrix completion.
Low-bit training framework reduces energy consumption in CNNs.
Paper proposes a 1-bit mean estimation method with near-optimal sample complexity.
We consider the problem of noisy 1-bit matrix completion under an exact rank constraint on the true underlying matrix . Instead of observing a subset of the noisy continuous-valued entries of a matrix , we observe a subset of noisy 1-bit (or binary) measurements generated according to a probabilistic model. W…
For fast and energy-efficient deployment of trained deep neural networks on resource-constrained embedded hardware, each learned weight parameter should ideally be represented and stored using a single bit. Error-rates usually increase when this requirement is imposed. Here, we report large improvements in error rates …
Social trust prediction addresses the significant problem of exploring interactions among users in social networks. Naturally, this problem can be formulated in the matrix completion framework, with each entry indicating the trustness or distrustness. However, there are two challenges for the social trust problem: 1) t…
Paper studies signal detection in noisy environments with limited communication.
We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…
Binary Neural Networks (BNNs) show promising progress in reducing computational and memory costs but suffer from substantial accuracy degradation compared to their real-valued counterparts on large-scale datasets, e.g., ImageNet. Previous work mainly focused on reducing quantization errors of weights and activations, w…
Autoencoders fail to capture sparse structure in 1-bit data compression.
Study on recovering supports of multiple sparse vectors from mixed linear measurements.
State of the art deep learning models have made steady progress in the fields of computer vision and natural language processing, at the expense of growing model sizes and computational complexity. Deploying these models on low power and mobile devices poses a challenge due to their limited compute capabilities and str…
New model analyzes customer churn with tensor completion and binary data.
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
Due to challenging applications such as collaborative filtering, the matrix completion problem has been widely studied in the past few years. Different approaches rely on different structure assumptions on the matrix in hand. Here, we focus on the completion of a (possibly) low-rank matrix with binary entries, the so-c…
We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the netwo…
Survey on quantization methods on Kähler manifolds.
The paper classifies quantizable functions and explores symmetry in quantization methods.
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.