arXiv research

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

Distributed sensors compress and send features to a fusion center for linear regression.

This paper proposes a simple adaptive sensing and group testing algorithm for sparse signal recovery. The algorithm, termed Compressive Adaptive Sense and Search (CASS), is shown to be near-optimal in that it succeeds at the lowest possible signal-to-noise-ratio (SNR) levels, improving on previous work in adaptive comp…

Adaptive sampling method optimizes DNN compression for resource-constrained platforms.

Adaptive step-size method improves compressed SGD performance in machine learning.

NAC-FL optimizes model updates in FL systems by adapting compression to network congestion.

COMP-AMS optimizes distributed training with compressed gradients, achieving similar accuracy with less communication.

Algorithm estimates principal eigenvector with adaptive sensing, improving over non-adaptive methods.

Adaptive Quantization Modules enable online continual compression of non-i.i.d data streams.

To improve how neural networks function it is crucial to understand their learning process. The information bottleneck theory of deep learning proposes that neural networks achieve good generalization by compressing their representations to disregard information that is not relevant to the task. However, empirical evid…

Improves compression of neural networks for embedded systems.

Network agents solve adaptive regression problems with compressed signals.

IntSGD compresses SGD gradients without floats, converging as SGD.

This work proposes ACTC for adaptive distributed learning under communication constraints.

We consider the problems of detection and localization of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements. This is closely related to the problem of compressed sensing, where the task is to estimate a sparse vector using a small…

New method reduces communication costs in distributed nonconvex optimization.

We compress large neural networks for quick adaptation to specific contexts.

Paper proposes Nyström sketches for better adaptive compressive learning.

Theoretical framework for neural network compression using sparsity norms.

This paper improves compression of large NLP models using doped Kronecker Products.

Highly distributed training of Deep Neural Networks (DNNs) on future compute platforms (offering 100 of TeraOps/s of computational capacity) is expected to be severely communication constrained. To overcome this limitation, new gradient compression techniques are needed that are computationally friendly, applicable to …

New adapted renormalized volume for hyperbolic 3-manifolds with compressible boundary.

In this paper we present a a deep generative model for lossy video compression. We employ a model that consists of a 3D autoencoder with a discrete latent space and an autoregressive prior used for entropy coding. Both autoencoder and prior are trained jointly to minimize a rate-distortion loss, which is closely relate…

Prototype for adaptive electron microscopy scans reduces dose and time.

We present an information-theoretic framework for sequential adaptive compressed sensing, Info-Greedy Sensing, where measurements are chosen to maximize the extracted information conditioned on the previous measurements. We show that the widely used bisection approach is Info-Greedy for a family of $k$-sparse signals b…

Recent breakthrough results in compressed sensing (CS) have established that many high dimensional objects can be accurately recovered from a relatively small number of non- adaptive linear projection observations, provided that the objects possess a sparse representation in some basis. Subsequent efforts have shown th…

This paper proposes a method to compress and adapt CNNs for real-world applications.

Basis adaptation in Homogeneous Chaos spaces rely on a suitable rotation of the underlying Gaussian germ. Several rotations have been proposed in the literature resulting in adaptations with different convergence properties. In this paper we present a new adaptation mechanism that builds on compressive sensing algorith…

Flexible framework improves communication efficiency across various systems.

Adaptive quantization improves SGD accuracy in data-parallel settings.

This paper presents a novel signal compression algorithm based on the Blaschke unwinding adaptive Fourier decomposition (AFD). The Blaschke unwinding AFD is a newly developed signal decomposition theory. It utilizes the Nevanlinna factorization and the maximal selection principle in each decomposition step, and achieve…

DZip improves general-purpose lossless compression using neural networks.

End-to-end meta-learned system for image compression.

New method compresses neural networks up to 14x with minimal performance loss.

ARDMs are a new model class for autoregressive diffusion that generalize existing models and can compress data efficiently.

New methods reduce private federated learning communication automatically.

Structured pruning reduces large language model size and latency.

We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all …

TOCO framework compresses neural networks based on tolerance analysis.

VQ-DRAW compresses images and generates realistic samples.

Compressed Monte Carlo improves efficiency in Bayesian inference.

A new federated learning method reduces communication costs and improves adaptivity.

Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery from undersampled measurements. In the context of inverse problems in dynamic imaging, recent research has de…

Deep neural networks (DNNs) have been proven to have many redundancies. Hence, many efforts have been made to compress DNNs. However, the existing model compression methods treat all the input samples equally while ignoring the fact that the difficulties of various input samples being correctly classified are different…

Midicoth compresses online probability estimates by correcting prior smoothing biases.

Unified framework for distributed compressed SGD under $(L_0, L_1)$-smoothness.

This paper proposes a new method to compress CNNs for medical image analysis, improving efficiency and accuracy.

The paper explores tensor decompositions in deep learning models.