A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
We propose and analyze a new type of stochastic first order method: gradient descent with compressed iterates (GDCI). GDCI in each iteration first compresses the current iterate using a lossy randomized compression technique, and subsequently takes a gradient step. This method is a distillation of a key ingredient in t…
Lossy compression of statistical data using quantum annealing.
NAC-FL optimizes model updates in FL systems by adapting compression to network congestion.
New method optimizes lossy compression models more effectively.
We study the limits and methods of training two-layer autoencoders.
New method improves image compression using bits-back coding.
This paper uses diffusion models for lossy image compression, improving perceptual metrics and practicality.
DiffC compresses images by diffusing Gaussian noise, outperforming state-of-the-art methods.
Paper proposes LC-Checkpoint for efficient deep learning model checkpoints.
We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to ad…
Paper designs a lossy compression method for lossless prediction.
Paper introduces adversarial lossy compression for video artifacts reduction.
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder and parallel multi-scale lossless coder. The multi-scale lossy autoencoder extracts the multi-scale image features to quantized variables a…
The large memory requirements of deep neural networks limit their deployment and adoption on many devices. Model compression methods effectively reduce the memory requirements of these models, usually through applying transformations such as weight pruning or quantization. In this paper, we present a novel scheme for l…
We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed sample…
A new method, REC, compresses images by encoding their latent representations efficiently.
Multiple gossip steps improve decentralized optimization convergence.
Estimates rate-distortion function for large datasets using neural networks.
New coding theorem shows achievable rate matches theoretical limit.
New method estimates rate-distortion function using optimal transport.
Autoencoders fail to capture sparse structure in 1-bit data compression.
New methods improve distributed optimization on non-iid data.
Ensemble methods are among the state-of-the-art predictive modeling approaches. Applied to modern big data, these methods often require a large number of sub-learners, where the complexity of each learner typically grows with the size of the dataset. This phenomenon results in an increasing demand for storage space, wh…
We study a new class of codes for lossy compression with the squared-error distortion criterion, designed using the statistical framework of high-dimensional linear regression. Codewords are linear combinations of subsets of columns of a design matrix. Called a Sparse Superposition or Sparse Regression codebook, this s…
New techniques save bits in image compression with upsampling.
New method preserves privacy while improving machine learning accuracy.
As a lossy compression framework, compressed sensing has drawn much attention in wireless telemonitoring of biosignals due to its ability to reduce energy consumption and make possible the design of low-power devices. However, the non-sparseness of biosignals presents a major challenge to compressed sensing. This study…
This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.
We consider the distributional connection between the lossy compressed representation of a high-dimensional signal using a random spherical code and the observation of under an additive white Gaussian noise (AWGN). We show that the Wasserstein distance between a bitrate- compressed version of and its obs…
This paper investigates compression techniques for deep neural networks to reduce their size without sacrificing performance.
Despite progress in training neural networks for lossy image compression, current approaches fail to maintain both perceptual quality and abstract features at very low bitrates. Encouraged by recent success in learning discrete representations with Vector Quantized Variational Autoencoders (VQ-VAEs), we motivate the us…
Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However, in recent years, it has become increasingly accepted that "low distortion" is no…
This work evaluates deep generative models using RD curves, providing a more comprehensive quality assessment.
We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all …
We propose computationally efficient encoders and decoders for lossy compression using a Sparse Regression Code. The codebook is defined by a design matrix and codewords are structured linear combinations of columns of this matrix. The proposed encoding algorithm sequentially chooses columns of the design matrix to suc…
Paper establishes generalization bounds for representation learning using Minimum Description Length.
Deep learning models have become state of the art for natural language processing (NLP) tasks, however deploying these models in production system poses significant memory constraints. Existing compression methods are either lossy or introduce significant latency. We propose a compression method that leverages low rank…
Paper shows how to integrate quantization into neural compression models.
Proposes a link between randomness and compression in deep learning.
Diffusion models improve image compression at low bit-rates.
Paper tightens CMI bounds for learning algorithms.
In this paper we present a a deep generative model for lossy video compression. We employ a model that consists of a 3D autoencoder with a discrete latent space and an autoregressive prior used for entropy coding. Both autoencoder and prior are trained jointly to minimize a rate-distortion loss, which is closely relate…
Compression is at the heart of effective representation learning. However, lossy compression is typically achieved through simple parametric models like Gaussian noise to preserve analytic tractability, and the limitations this imposes on learning are largely unexplored. Further, the Gaussian prior assumptions in model…
Existing techniques to compress point cloud attributes leverage either geometric or video-based compression tools. We explore a radically different approach inspired by recent advances in point cloud representation learning. Point clouds can be interpreted as 2D manifolds in 3D space. Specifically, we fold a 2D grid on…
Modern deep learning models are often trained in parallel over a collection of distributed machines to reduce training time. In such settings, communication of model updates among machines becomes a significant performance bottleneck and various lossy update compression techniques have been proposed to alleviate this p…
Method reconstructs hidden Markov chains from insurance data.
Efficient point cloud compression is fundamental to enable the deployment of virtual and mixed reality applications, since the number of points to code can range in the order of millions. In this paper, we present a novel data-driven geometry compression method for static point clouds based on learned convolutional tra…