The problem of population recovery refers to estimating a distribution based on incomplete or corrupted samples. Consider a random poll of sample size conducted on a population of individuals, where each pollee is asked to answer binary questions. We consider one of the two polling impediments: (a) in lossy pop…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “lossy samples”
The paper proposes a nonparametric test for incomplete samples quantized to B bits.
problem Statistical inference with lossy or incomplete samples.
method Nonparametric testing procedure based on B-bit quantized samples.
result The proposed test achieves the classical minimax rate of testing for spline models when B exceeds a threshold.
We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed sample…
DiffC compresses images by diffusing Gaussian noise, outperforming state-of-the-art methods.
problem Efficient lossy image compression without transform coding.
method Unconditional diffusion generative models to encode and denoise corrupted pixels.
result DiffC outperforms HiFiC on ImageNet 64x64, achieving a 3 dB gain at high bitrates.
Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Codingcs.IT
Estimates rate-distortion function for large datasets using neural networks.
problem Designing lossy data compression schemes and comparing them with theoretical limits.
method Re-formulate rate-distortion objective and solve using neural networks.
result NERD accurately estimates the rate-distortion function for real-world datasets.
New method optimizes lossy compression models more effectively.
problem Difficulty in manually setting a tradeoff parameter for lossy compression models.
method Constrained optimization method of Rezende and Viola, 2018.
result The method enables pointwise model comparisons and outperforms existing methods.
Lossy compression of statistical data using quantum annealing.
problem Efficiently compressing statistical floating-point data.
method Representation learning with binary variables, classical optimization of basis vectors, quantum annealing for coefficients, bias correction.
result Quantum annealing shows promising results with 3.5x better compression than neural-network autoencoders.
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder and parallel multi-scale lossless coder. The multi-scale lossy autoencoder extracts the multi-scale image features to quantized variables a…
NAC-FL optimizes model updates in FL systems by adapting compression to network congestion.
problem Federated Learning systems face congestion and delays in data exchanges.
method NAC-FL dynamically adjusts client compression based on network congestion.
result NAC-FL reduces training time and achieves robust performance improvements.
This paper uses diffusion models for lossy image compression, improving perceptual metrics and practicality.
problem Lossy image compression with improved perceptual metrics and practicality.
method End-to-end optimized lossy image compression using conditional diffusion models.
result The model yields stronger FID scores and competitive performance in distortion metrics.
New method improves image compression using bits-back coding.
problem Lossy image compression with deep latent variable models.
method Iterative inference, stochastic annealing, bits-back coding.
result New state-of-the-art performance on lossy image compression.
Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decodingcs.IT
We propose computationally efficient encoders and decoders for lossy compression using a Sparse Regression Code. The codebook is defined by a design matrix and codewords are structured linear combinations of columns of this matrix. The proposed encoding algorithm sequentially chooses columns of the design matrix to suc…
New coding theorem shows achievable rate matches theoretical limit.
problem Unknown existence of encoders and decoders for RDPF.
method Used stochastic, variable-length codes to prove RDPF achievable.
result Achievable rate matches theoretical rate-distortion-perception function.
New study shows tradeoffs between compression quality, distortion, and perception.
problem Optimizing compression for low distortion often sacrifices perceptual quality.
method Adopted Blau & Michaeli's perceptual quality definition and studied the rate-distortion-perception tradeoff.
result Restricting perceptual quality to high generally requires a trade-off between rate and distortion.
Paper designs a lossy compression method for lossless prediction.
problem Ensuring high performance on predictive tasks with minimal data.
method Characterizes bit-rate requirements for invariant transformations, designs unsupervised objectives for neural compressors.
result Achieves substantial rate savings on ImageNet compared to JPEG without compromising classification performance.
The large memory requirements of deep neural networks limit their deployment and adoption on many devices. Model compression methods effectively reduce the memory requirements of these models, usually through applying transformations such as weight pruning or quantization. In this paper, we present a novel scheme for l…
Study shows data heterogeneity affects distributed learning's generalization error.
problem Effect of data heterogeneity on distributed learning performance.
method Established bounds on generalization error using information-theoretic rate-distortion theory.
result Data heterogeneity improves generalization error for distributed learning.
We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to ad…
A new method quantizes LSTM gate parameters without performance loss.
problem Quantization loss in LSTM gate parameters without performance degradation.
method Lossy quantization of gate parameters during training, weight parameters adjust to offset quantization loss.
result F1 score decreased by only 0.7% on Named Entity Recognition dataset.
Study shows how information loss and operation loss are related in feature representations.
problem Understanding the relationship between information loss and operation loss in feature representations.
method Analyzes the interplay between weak information loss and operation loss in continuous representations.
result Specific forms of vanishing information loss imply vanishing MPE loss in classification.
In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly so-called pooling. Maximation and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convoluti…
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
Representation learning seeks to expose certain aspects of observed data in a learned representation that's amenable to downstream tasks like classification. For instance, a good representation for 2D images might be one that describes only global structure and discards information about detailed texture. In this paper…
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
problem Communication bottleneck in decentralized deep learning models.
method Low-rank linear compressors applied on model differences using power iteration steps.
result Asymptotically independent of network and compression, faster convergence, and comparable performance to tuned compression algorithms.
This paper generalizes the Maurer--Pontil framework of finite-dimensional lossy coding schemes to the setting where a high-dimensional random vector is mapped to an element of a compact set of latent representations in a lower-dimensional Euclidean space, and the reconstruction map belongs to a given class of nonlinear…
A new method compresses point clouds efficiently, outperforming existing techniques.
problem Efficiently compressing large point cloud datasets for VR applications.
method Learned convolutional transforms and uniform quantization for joint rate and distortion optimization.
result Significant rate-distortion improvement (51.5% BDBR savings) on Microsoft Voxelized Upper Bodies dataset.
High-quality image synthesis with diffusion models, achieving state-of-the-art FID score.
problem Generating high-quality images from latent variables.
method Training diffusion probabilistic models with a weighted variational bound, inspired by denoising score matching and Langevin dynamics.
result State-of-the-art FID score of 3.17 on CIFAR10 dataset.
Automates feature extraction from JSON data for machine learning.
problem Manual feature engineering for JSON data is laborious, lossy, and prone to bias.
method Automates feature extraction using Mill.jl and JsonGrinder.jl.
result Creates a differentiable machine learning model from raw JSON samples.
A new method, REC, compresses images by encoding their latent representations efficiently.
problem Efficiently compressing single images with latent representations.
method Relative Entropy Coding (REC) that directly encodes latent representations with codelength close to relative entropy.
result REC is more efficient for single image compression compared to previous methods and is competitive for lossy compression.
This work evaluates deep generative models using RD curves, providing a more comprehensive quality assessment.
problem Quantitative evaluation of deep generative models is challenging, especially for implicit models.
method Proposes using rate distortion (RD) curves to evaluate and compare deep generative models, approximating the entire curve with similar computations to log-likelihood estimation.
result Approximating the entire RD curve provides a more comprehensive quality assessment than scalar-valued metrics.
Paper establishes generalization bounds for representation learning using Minimum Description Length.
problem Designing efficient statistical supervised learning algorithms that generalize well to unseen data.
method Developed a compressibility framework using Minimum Description Length (MDL) to derive upper bounds on generalization error.
result Established the first theoretical generalization bounds for Information Bottleneck type encoders and representation learning.
We study a new class of codes for lossy compression with the squared-error distortion criterion, designed using the statistical framework of high-dimensional linear regression. Codewords are linear combinations of subsets of columns of a design matrix. Called a Sparse Superposition or Sparse Regression codebook, this s…
We study the limits and methods of training two-layer autoencoders.
problem Understanding the limits and methods of training two-layer autoencoders.
method Focus on non-linear two-layer autoencoders trained in the proportional regime, using gradient methods.
result Gradient methods achieve the minimizers of the population risk and reveal the structure of the features.
Paper tightens CMI bounds for learning algorithms.
problem Generalization error in statistical learning.
method Stochastic projection and lossy compression.
result New CMI bounds yield generalization guarantees of .
Ensemble methods are among the state-of-the-art predictive modeling approaches. Applied to modern big data, these methods often require a large number of sub-learners, where the complexity of each learner typically grows with the size of the dataset. This phenomenon results in an increasing demand for storage space, wh…
Data coarse graining can improve model performancecond-mat.stat-mech
Data coarse graining improves model performance by filtering out less relevant features.
problem Lossy data transformations lose information but can improve model generalization.
method Data coarse graining schemes that systematically discard features based on relevance to the learning task.
result A 'high-pass' scheme helps models generalize better by filtering out less relevant features.
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
New techniques save bits in image compression with upsampling.
problem Lack of context dependence in current image compression methods with upsampling.
method Simple, inexpensive techniques exploiting context to predict Laplace distribution parameters.
result Average savings of 0.645 bits per difference, up to 1.489 bits.
New method estimates rate-distortion function using optimal transport.
problem Estimating the fundamental performance limit of data compression.
method Wasserstein gradient descent to learn optimal reproduction distribution.
result Local convergence and sample complexity analysis of R-D estimator.
The relevance of data quantifies learning efficiency.
problem Understanding the statistical nature of high-dimensional, sparse data.
method Defining relevance as information content, and using it to define ideal limits of samples and learning machines.
result Maximally informative samples and optimal learning machines exhibit critical features like power-law frequency distributions and anomalously large susceptibility.
New method preserves privacy while improving machine learning accuracy.
problem Privacy-preserving machine learning for daily data.
method Compressive Privacy and multi-kernel method.
result Improved utility classification accuracy with privacy preservation.
Modern scientific instruments produce vast amounts of data, which can overwhelm the processing ability of computer systems. Lossy compression of data is an intriguing solution, but comes with its own drawbacks, such as potential signal loss, and the need for careful optimization of the compression ratio. In this work, …
This paper improves multi-class calibration methods using mutual information maximization-based binning.
problem Calibration of deep neural network predictions, especially for small prior classes.
method I-Max concept for binning, shared class-wise calibration strategy.
result Improves multi-class ranking and calibration performance using a small calibration set.
Gradient descent with compressed iterates reduces model size for federated learning.
problem Efficiently compressing model sizes in federated learning.
method Gradient descent with iterates compressed using lossy randomized techniques.
result Improves model compression and aggregation in federated learning.
Method reconstructs hidden Markov chains from insurance data.
problem Recovering hidden Markov chains from incomplete insurance data.
method Neural architecture to explicitly provide transition probabilities.
result Neural model successfully validates decompression of insurance information.
The Information Sievestat.ML
We introduce a new framework for unsupervised learning of representations based on a novel hierarchical decomposition of information. Intuitively, data is passed through a series of progressively fine-grained sieves. Each layer of the sieve recovers a single latent factor that is maximally informative about multivariat…
We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all …
As a lossy compression framework, compressed sensing has drawn much attention in wireless telemonitoring of biosignals due to its ability to reduce energy consumption and make possible the design of low-power devices. However, the non-sparseness of biosignals presents a major challenge to compressed sensing. This study…