A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper offers robust recovery for 1-bit sensing with partial Gaussian circulant matrices.
The goal of standard 1-bit compressive sensing is to accurately recover an unknown sparse vector from binary-valued measurements, each indicating the sign of a linear function of the vector. Motivated by recent advances in compressive sensing with generative models, where a generative modeling assumption replaces the u…
Paper proposes a 1-bit quantization scheme for high-dimensional statistical estimation.
Binary Iterative Hard Thresholding converges with optimal number of 1-bit measurements.
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
Autoencoders fail to capture sparse structure in 1-bit data compression.
Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributed neural network training. Using 1-bit quantization, signSGD with majority vote …
Study on recovering supports of multiple sparse vectors from mixed linear measurements.
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
Study on recovering sparse linear classifiers from mixed binary responses.
We present DeepFPC, a novel deep neural network designed by unfolding the iterations of the fixed-point continuation algorithm with one-sided l1-norm (FPC-l1), which has been proposed for solving the 1-bit compressed sensing problem. The network architecture resembles that of deep residual learning and incorporates pri…
The recent framework of compressive statistical learning aims at designing tractable learning algorithms that use only a heavily compressed representation-or sketch-of massive datasets. Compressive K-Means (CKM) is such a method: it estimates the centroids of data clusters from pooled, non-linear, random signatures of …
Unified framework for uniform signal recovery in nonlinear GCS with 1-bit/quantized measurements.
New protocols show 1-bit mean estimation can be order-optimal without interaction.
Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations. Despite model size reduction, achieving performance enhancement on devices is, however, still challenging mainly due to the irregular representations of spa…
Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage re…
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
A new method for 1-bit matrix completion that is faster and more accurate.
Paper develops an efficient mean estimator for 1-bit communication constraints.
Matrix completion has a long-time history of usage as the core technique of recommender systems. In particular, 1-bit matrix completion, which considers the prediction as a ``Recommended'' or ``Not Recommended'' question, has proved its significance and validity in the field. However, while customers and products aggre…
New method predicts binary matrix entries using empirical Bayes and low-rank structure.
Study quantile reward identification with 1-bit feedback constraints.
Study improves fractional posterior for 1-bit matrix completion.
This paper resolves BIHT convergence, showing normalization is not necessary in noiseless settings but crucial for robustness.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
Paper proposes a 1-bit mean estimation method with near-optimal sample complexity.
We consider the problem of noisy 1-bit matrix completion under an exact rank constraint on the true underlying matrix . Instead of observing a subset of the noisy continuous-valued entries of a matrix , we observe a subset of noisy 1-bit (or binary) measurements generated according to a probabilistic model. W…
For fast and energy-efficient deployment of trained deep neural networks on resource-constrained embedded hardware, each learned weight parameter should ideally be represented and stored using a single bit. Error-rates usually increase when this requirement is imposed. Here, we report large improvements in error rates …
Sparse random networks reduce communication in federated learning.
Social trust prediction addresses the significant problem of exploring interactions among users in social networks. Naturally, this problem can be formulated in the matrix completion framework, with each entry indicating the trustness or distrustness. However, there are two challenges for the social trust problem: 1) t…
Paper studies signal detection in noisy environments with limited communication.
We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…
New quantization methods improve accuracy of Random Fourier Features.
Running Stochastic Gradient Descent (SGD) in a decentralized fashion has shown promising results. In this paper we propose Moniqua, a technique that allows decentralized SGD to use quantized communication. We prove in theory that Moniqua communicates a provably bounded number of bits per iteration, while converging at …
Recently, there is a growing interest in the study of median-based algorithms for distributed non-convex optimization. Two prominent such algorithms include signSGD with majority vote, an effective approach for communication reduction via 1-bit compression on the local gradients, and medianSGD, an algorithm recently pr…
The support recovery problem consists of determining a sparse subset of a set of variables that is relevant in generating a set of observations, and arises in a diverse range of settings such as compressive sensing, and subset selection in regression, and group testing. In this paper, we take a unified approach to supp…
New model analyzes customer churn with tensor completion and binary data.
New method approximates M-estimator and predictions without solving fixed-point equations.
Due to challenging applications such as collaborative filtering, the matrix completion problem has been widely studied in the past few years. Different approaches rely on different structure assumptions on the matrix in hand. Here, we focus on the completion of a (possibly) low-rank matrix with binary entries, the so-c…
We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the netwo…
The paper develops a robust signal estimation method for noisy measurements from generative models.
This paper introduces a differentiable, scalable quantization method for neural networks.
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
BiTAT improves neural network quantization for edge devices by focusing on weight dependencies and disentangling them.
Flexible framework compresses models using LC algorithm.
Reduces multiclass and regression compression schemes to binary ones.
In this paper, we propose a test, called Flagged-1-Bit (F1B) test, to study the intrinsic capability of recurrent neural networks in sequence learning. Four different recurrent network models are studied both analytically and experimentally using this test. Our results suggest that in general there exists a conflict be…