A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Develops a method for lossless compression using latent variable models.
We make the following striking observation: fully convolutional VAE models trained on 32x32 ImageNet can generalize well, not just to 64x64 but also to far larger photographs, with no changes to the model. We use this property, applying fully convolutional models to lossless compression, demonstrating a method to scale…
IDF++ improves integer discrete flows for lossless compression.
The bits-back argument suggests that latent variable models can be turned into lossless compression schemes. Translating the bits-back argument into efficient and practical lossless compression schemes for general latent variable models, however, is still an open problem. Bits-Back with Asymmetric Numeral Systems (BB-A…
Lossless compression methods shorten the expected representation size of data without loss of information, using a statistical model. Flow-based models are attractive in this setting because they admit exact likelihood optimization, which is equivalent to minimizing the expected number of bits per message. However, con…
Lossless compression of deep neural networks using NTK and RMT.
Deep latent variable models have seen recent success in many data domains. Lossless compression is an application of these models which, despite having the potential to be highly useful, has yet to be implemented in a practical manner. We present `Bits Back with ANS' (BB-ANS), a scheme to perform lossless compression w…
NeLLoC improves image compression with parallel decoding.
Paper designs a lossy compression method for lossless prediction.
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder and parallel multi-scale lossless coder. The multi-scale lossy autoencoder extracts the multi-scale image features to quantized variables a…
A tutorial on VAEs explaining its derivation and applications.
Deep neural networks have been successful in many predictive modeling tasks, such as image and language recognition, where large neural networks are often used to obtain good accuracy. Consequently, it is challenging to deploy these networks under limited computational resources, such as in mobile devices. In this work…
New method improves generalization of VAEs by reducing overfitting.
Study reduces memory needs for active learning with enriched queries.
ARDMs are a new model class for autoregressive diffusion that generalize existing models and can compress data efficiently.
We consider lossless compression based on statistical data modeling followed by prediction-based encoding, where an accurate statistical model for the input data leads to substantial improvements in compression. We propose DZip, a general-purpose compressor for sequential data that exploits the well-known modeling capa…
Ensemble methods are among the state-of-the-art predictive modeling approaches. Applied to modern big data, these methods often require a large number of sub-learners, where the complexity of each learner typically grows with the size of the dataset. This phenomenon results in an increasing demand for storage space, wh…
The ultimate goal of a supervised learning algorithm is to produce models constructed on the training data that can generalize well to new examples. In classification, functional margin maximization -- correctly classifying as many training examples as possible with maximal confidence --has been known to construct mode…
Generalizes bits back coding for time-series models with latent Markov structures.
This book presents a methodology and philosophy of empirical science based on large scale lossless data compression. In this view a theory is scientific if it can be used to build a data compression program, and it is valuable if it can compress a standard benchmark database to a small size, taking into account the len…
We present a new similarity measure based on information theoretic measures which is superior than Normalized Compression Distance for clustering problems and inherits the useful properties of conditional Kolmogorov complexity. We show that Normalized Compression Dictionary Size and Normalized Compression Dictionary En…
Neural NCD reveals LLMs don't compress well for classification.
Proposes a new linearity-based neural network compression method.
This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.
Physics-inspired methods optimize SVD compression of LLMs.
Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…
This paper investigates compression techniques for deep neural networks to reduce their size without sacrificing performance.
A new method, REC, compresses images by encoding their latent representations efficiently.
Data compression is a popular technique for improving the efficiency of data processing workloads such as SQL queries and more recently, machine learning (ML) with classical batch gradient methods. But the efficacy of such ideas for mini-batch stochastic gradient descent (MGD), arguably the workhorse algorithm of moder…
SHVC improves image compression with fewer parameters.
Likelihood-based generative models are the backbones of lossless compression due to the guaranteed existence of codes with lengths close to negative log likelihood. However, there is no guaranteed existence of computationally efficient codes that achieve these lengths, and coding algorithms must be hand-tailored to spe…
NeuZip compresses neural network weights to save memory during training and inference.
Paper proposes a compression principle for neural networks using Bayesian optimization.
Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantization with the advanced optimization algorithm ADMM (Alternating Direction Methods of Multipliers) achieves one of state-of-art results in we…
A new method learns graph compression from data.
PAQ8 is an open source lossless data compression algorithm that currently achieves the best compression rates on many benchmarks. This report presents a detailed description of PAQ8 from a statistical machine learning perspective. It shows that it is possible to understand some of the modules of PAQ8 and use this under…
ResRep prunes CNNs without losing accuracy by separating remembering and forgetting.
Paper compresses deep neural networks by eliminating redundant neurons.
Unified bounds linking compressibility, fractal dimensions, and mutual information.
A framework infers causal direction from symbolic sequences using compression measures.
End-to-end meta-learned system for image compression.
Deep convolutional neural networks (CNNs) are powerful tools for a wide range of vision tasks, but the enormous amount of memory and compute resources required by CNNs pose a challenge in deploying them on constrained devices. Existing compression techniques, while excelling at reducing model sizes, struggle to be comp…
New techniques save bits in image compression with upsampling.
A CAE improves DNN's outlier and adversary defense.
Bayesian Attention Networks compress data by focusing on key training samples.
Paper detects common subtrees with identical labels in trees.
Study excess risk in statistical inference with transformations.
RFX accelerates and compresses Random Forests for large datasets.