arXiv research

Reduces multiclass and regression compression schemes to binary ones.

Study 1-bit compressive sensing with generative models, improving recovery accuracy.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

Learnable multiclass hypothesis classes don't always have a sample compression scheme of fixed size.

Paper ranks stocks by compression risk, not volatility.

Compressive Sensing (CS) theory asserts that sparse signal reconstruction is possible from a small number of linear measurements. Although CS enables low-cost linear sampling, it requires non-linear and costly reconstruction. Recent literature works show that compressive image classification is possible in CS domain wi…

Paper proposes a new binary quantization method for faster DNN inference.

Lossy compression of statistical data using quantum annealing.

Pruning is an efficient model compression technique to remove redundancy in the connectivity of deep neural networks (DNNs). Computations using sparse matrices obtained by pruning parameters, however, exhibit vastly different parallelism depending on the index representation scheme. As a result, fine-grained pruning ha…

New guarantees for asymmetric sketching in compressive learning.

Paper proposes IIQ for compressing embedding vectors.

We consider the problem of robust compressed sensing whose objective is to recover a high-dimensional sparse signal from compressed measurements corrupted by outliers. A new sparse Bayesian learning method is developed for robust compressed sensing. The basic idea of the proposed method is to identify and remove the ou…

We show through case studies that it is easier to estimate the fundamental limits of data processing than to construct explicit algorithms to achieve those limits. Focusing on binary classification, data compression, and prediction under logarithmic loss, we show that in the finite space setting, when it is possible to…

AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.

In this paper, we study the problem of compressed sensing using binary measurement matrices and $\ell_1$-norm minimization (basis pursuit) as the recovery algorithm. We derive new upper and lower bounds on the number of measurements to achieve robust sparse recovery with binary matrices. We establish sufficient conditi…

Distributed Lion optimizes large model training by reducing communication costs.

This paper proposes to perform authorship analysis using the Fast Compression Distance (FCD), a similarity measure based on compression with dictionaries directly extracted from the written texts. The FCD computes a similarity between two documents through an effective binary search on the intersection set between the …

A new method prunes deep networks in one go without specifying pruning levels.

Midicoth compresses online probability estimates by correcting prior smoothing biases.

Binary Iterative Hard Thresholding converges with optimal number of 1-bit measurements.

Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.

In this paper, we investigate the degree to which the encoding of a $β$-VAE captures label information across multiple architectures on Binary Static MNIST and Omniglot. Even though they are trained in a completely unsupervised manner, we demonstrate that a $β$-VAE can retain a large amount of label information, even w…

This paper analyzes the training dynamics of binary neural networks using information bottleneck.

This paper deals with two related problems, namely distance-preserving binary embeddings and quantization for compressed sensing . First, we propose fast methods to replace points from a subset $\mathcal{X} \subset \mathbb{R}^n$, associated with the Euclidean metric, with points in the cube $\{\pm 1\}^m$ and we associa…

It was proved in 1998 by Ben-David and Litman that a concept space has a sample compression scheme of size d if and only if every finite subspace has a sample compression scheme of size d. In the compactness theorem, measurability of the hypotheses of the created sample compression scheme is not guaranteed; at the same…

FleXOR trains fractional quantization for neural networks, improving accuracy and size.

Compressing convolutional neural networks (CNNs) is essential for transferring the success of CNNs to a wide variety of applications to mobile devices. In contrast to directly recognizing subtle weights or filters as redundant in a given CNN, this paper presents an evolutionary method to automatically eliminate redunda…

The study explores the compressive power of Boolean threshold autoencoders, finding that seven layers are necessary but three are not.

The use of deep neural networks in edge computing devices hinges on the balance between accuracy and complexity of computations. Ternary Connect (TC) \cite{lin2015neural} addresses this issue by restricting the parameters to three levels $-1, 0$, and $+1$, thus eliminating multiplications in the forward pass of the net…

In this paper, we propose majority voting neural networks for sparse signal recovery in binary compressed sensing. The majority voting neural network is composed of several independently trained feedforward neural networks employing the sigmoid function as an activation function. Our empirical study shows that a choice…

Study on recovering sparse linear classifiers from mixed binary responses.

Adaptive sampling method optimizes DNN compression for resource-constrained platforms.

We obtain the first positive results for bounded sample compression in the agnostic regression setting with the $\ell_p$ loss, where $p\in [1,\infty]$. We construct a generic approximate sample compression scheme for real-valued function classes exhibiting exponential size in the fat-shattering dimension but independen…

MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.

This paper improves support recovery in universal one-bit compressed sensing with fewer measurements.

New method compresses large sample data for faster discriminant analysis.

This paper resolves BIHT convergence, showing normalization is not necessary in noiseless settings but crucial for robustness.

We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size-$k$ agnostic sample compression schemes are of…

Optimized sampling scheme for compressed sensing combining randomness and determinism.

Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes are becoming increasingly relevant. A major issue in distributed training is the limited communication bandwidth between contributing nodes o…

Conventional approaches of sampling signals follow the celebrated theorem of Nyquist and Shannon. Compressive sampling, introduced by Donoho, Romberg and Tao, is a new paradigm that goes against the conventional methods in data acquisition and provides a way of recovering signals using fewer samples than the traditiona…

In today's data driven world, storing, processing, and gleaning insights from large-scale data are major challenges. Data compression is often required in order to store large amounts of high-dimensional data, and thus, efficient inference methods for analyzing compressed data are necessary. Building on a recently desi…

Investigates principles of generalization in list learning, refutes sample compression conjecture.

Kolmogorov-Arnold network improves GW catalog posterior construction.

CTT compresses samples to test distributions near-linearly, outperforming existing methods.

The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.

Fetal ECG (FECG) telemonitoring is an important branch in telemedicine. The design of a telemonitoring system via a wireless body-area network with low energy consumption for ambulatory use is highly desirable. As an emerging technique, compressed sensing (CS) shows great promise in compressing/reconstructing data with…

Clapping reduces memory usage in distributed optimization by reusing data samples.