arXiv research

Proposes a method to train neural networks directly on compressed text data.

Neural model extracts tokens as latent variables for text compression.

Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification…

Neural networks compress uninformative input directions, improving test error.

Compressed LLM embeddings improve noisy regression tasks without overfitting.

Improved image compression with diffusion models outperforming state-of-the-art methods.

New bounds for large language models using token properties.

This paper introduces a new Lagrangian for the Information Bottleneck problem to simplify optimization.

This work tackles extractive compression by formulating it as tree transduction.

Differentiable mask prunes deep networks for vision and text.

This paper proposes to perform authorship analysis using the Fast Compression Distance (FCD), a similarity measure based on compression with dictionaries directly extracted from the written texts. The FCD computes a similarity between two documents through an effective binary search on the intersection set between the …

Data compression is a popular technique for improving the efficiency of data processing workloads such as SQL queries and more recently, machine learning (ML) with classical batch gradient methods. But the efficacy of such ideas for mini-batch stochastic gradient descent (MGD), arguably the workhorse algorithm of moder…

LLMs compress financial texts, but distort decision-making.

Compression method reduces word embedding size for NLP models.

We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our interest is in describing their convergence with sequence length, assuming no limit…

Discrete Flow Maps bypass sequential prediction limits for parallel text generation.

New compression methods handle biased input sequences for more accurate posterior summaries.

Weight Squeezing transfers knowledge from large models to smaller ones, improving performance and speed.

Generative AI tasks analyzed for text, images, audio, video, code, and molecules.

PAQ8 is an open source lossless data compression algorithm that currently achieves the best compression rates on many benchmarks. This report presents a detailed description of PAQ8 from a statistical machine learning perspective. It shows that it is possible to understand some of the modules of PAQ8 and use this under…

ANT learns sparse embeddings for large vocabularies efficiently.

This book presents a methodology and philosophy of empirical science based on large scale lossless data compression. In this view a theory is scientific if it can be used to build a data compression program, and it is valuable if it can compress a standard benchmark database to a small size, taking into account the len…

Algorithm compresses large matrices by approximating them as low rank and low precision factors.

We sparsify gated RNNs by simplifying their structure.

This work compresses BERT into simple neural networks using unlabeled data.

Deep networks learn to compress high-dimensional data into lower dimensions.

PCA reduces language model embeddings to improve speed and memory efficiency.

Funnel-Transformer reduces computation by compressing sequence data.

We consider an online version of the robust Principle Component Analysis (PCA), which arises naturally in time-varying source separations such as video foreground-background separation. This paper proposes a compressive online robust PCA with prior information for recursively separating a sequences of frames into spars…

PaGoDA reduces diffusion model training costs by 64x.

GENs model generates sparse graphs from text-based inputs, achieving high validity.

We develop embeddings for nonlinear subspaces preserving vector norms.

Improves document summarization by combining word embeddings and n-grams.

This paper explores how NLP enhances insurance data analysis.

Study shows nonextendibility of warped spacelike singularities in specific spacetimes.

Flexible framework compresses models using LC algorithm.

Reduces multiclass and regression compression schemes to binary ones.

Proposes a link between randomness and compression in deep learning.

A new method classifies images using a compressed summary.

Unified framework for non-compressed networks from compression bounds.

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

Galen algorithm compresses neural networks for specific hardware with reduced latency.

The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…

Paper compresses neural network weight-updates for image artifacts removal.

HiLLoC compresses large images losslessly using VAEs.

Neural NCD reveals LLMs don't compress well for classification.

Randomly chosen support makes sparse linear regression easy.

BDC compresses both sample size and dimensionality of large datasets.