It was proved in 1998 by Ben-David and Litman that a concept space has a sample compression scheme of size d if and only if every finite subspace has a sample compression scheme of size d. In the compactness theorem, measurability of the hypotheses of the created sample compression scheme is not guaranteed; at the same…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,878 papers · 148 categories
Trend · papers per month
48 results for “compressed measurements”
New measure helps identify better word embedding compression methods.
problem Challenges in evaluating compressed word embeddings for downstream tasks.
method Proposed eigenspace overlap score and developed generalization bounds.
result Eigenspace overlap score correlates with better downstream performance.
We present a new similarity measure based on information theoretic measures which is superior than Normalized Compression Distance for clustering problems and inherits the useful properties of conditional Kolmogorov complexity. We show that Normalized Compression Dictionary Size and Normalized Compression Dictionary En…
Study 1-bit compressive sensing with generative models, improving recovery accuracy.
problem Accurately recover sparse vectors from binary measurements with generative models.
method Analyzes noiseless and noisy 1-bit measurements with i.i.d.~Gaussian and Lipschitz continuous generative priors, proving sample complexity bounds and stability properties.
result Proves sample complexity bounds and stability properties for 1-bit compressive sensing with generative models.
This work considers an estimation task in compressive sensing, where the goal is to estimate an unknown signal from compressive measurements that are corrupted by additive pre-measurement noise (interference, or clutter) as well as post-measurement noise, in the specific setting where some (perhaps limited) prior knowl…
RandNet learns from compressed image data, improving efficiency and accuracy.
problem Efficiency and accuracy in training neural networks with large datasets.
method RandNet uses compressed random measurements of images to train neural networks efficiently.
result RandNet achieves comparable accuracy to full data training with minimal loss.
New coherence parameter for GNNs with Fourier measurements improves signal recovery.
problem Characterizing generative compressed sensing with Fourier measurements.
method Subspace counting arguments and high-dimensional probability theory.
result First known restricted isometry guarantee for generative compressed sensing with subsampled isometries.
A fast compression-based similarity measure with applications to content-based image retrievalstat.ML
Compression-based similarity measures are effectively employed in applications on diverse data types with a basically parameter-free approach. Nevertheless, there are problems in applying these techniques to medium-to-large datasets which have been seldom addressed. This paper proposes a similarity measure based on com…
A field known as Compressive Sensing (CS) has recently emerged to help address the growing challenges of capturing and processing high-dimensional signals and data sets. CS exploits the surprising fact that the information contained in a sparse signal can be preserved in a small number of compressive (or random) linear…
We solve the compressive sensing problem via convolutional factor analysis, where the convolutional dictionaries are learned {\em in situ} from the compressed measurements. An alternating direction method of multipliers (ADMM) paradigm for compressive sensing inversion based on convolutional factor analysis is develope…
Binary Iterative Hard Thresholding converges with optimal number of 1-bit measurements.
problem Recovering sparse signals from 1-bit compressed measurements.
method Binary Iterative Hard Thresholding (BIHT) algorithm.
result BIHT converges with only O(k/ε) measurements, optimal for recovery.
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the ratio, for compressed CNNs.
Profile entropy measures learnability and compressibility of discrete distributions.
problem Understanding the learnability and compressibility of discrete distributions.
method Investigates profile entropy, showing its role in estimation, inference, and compression.
result Profile entropy is a fundamental measure unifying estimation, inference, and compression.
The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.
problem Theoretical and practical improvements in compressed sensing with optimized sampling schemes.
method Theoretical analysis and empirical experiments with optimized sampling schemes for subsampled unitary matrices.
result The error caused by measurement noise vanishes with an increasing number of measurements for optimized sampling schemes, assuming Gaussian noise.
Advances understanding of neural network generalization via tensor analysis.
problem Understanding the generalizability of deep neural networks.
method Tensor analysis to measure compressibility and generalizability.
result Proposed generalization bound outperforms previous methods, especially for tensor-based networks.
We consider the problems of detection and localization of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements. This is closely related to the problem of compressed sensing, where the task is to estimate a sparse vector using a small…
A framework infers causal direction from symbolic sequences using compression measures.
problem Inferring causal direction from two observed discrete symbolic sequences.
method Lossless compressors for inferring context-free grammars (CFGs) and quantifying compression extent.
result Grammar inferred from one sequence better compresses the other sequence, indicating causal direction.
Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.
problem Designing a decoder for sparse codes from linear measurements alone.
method Matrix factorization to recover encoder and sparse coding matrices from measurements.
result Decoder-Expander Based Factorisation recovers encoder and sparse coding matrix at optimal measurement rate with high probability.
This letter proposes a dictionary learning algorithm for blind one bit compressed sensing. In the blind one bit compressed sensing framework, the original signal to be reconstructed from one bit linear random measurements is sparse in an unknown domain. In this context, the multiplication of measurement matrix $\Ab$ an…
Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theorystat.ML
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
Improved compressed sensing using a generator that learns from measurements.
problem Signal recovery accuracy in compressed sensing.
method Proposes a framework that uses measurement-conditional generative models to refine signal estimation.
result Uniformly superior performance with up to an order of magnitude reduction in reconstruction error.
Deep learning models have significantly improved the visual quality and accuracy on compressive sensing recovery. In this paper, we propose an algorithm for signal reconstruction from compressed measurements with image priors captured by a generative model. We search and constrain on latent variable space to make the m…
Compressive Sensing (CS) theory asserts that sparse signal reconstruction is possible from a small number of linear measurements. Although CS enables low-cost linear sampling, it requires non-linear and costly reconstruction. Recent literature works show that compressive image classification is possible in CS domain wi…
This paper studies the problem of estimating the covariance of a collection of vectors using only highly compressed measurements of each vector. An estimator based on back-projections of these compressive samples is proposed and analyzed. A distribution-free analysis shows that by observing just a single linear measure…
Algorithm estimates principal eigenvector with adaptive sensing, improving over non-adaptive methods.
problem Estimating principal eigenvector with limited scalar measurements.
method Compressed variant of Oja's algorithm using two adaptive measurements per sample.
result Convergence rate of after iterations, matching information-theoretic lower bound.
Proposes using equivariant generative models for compressed sensing with unknown orientations.
problem Recovering signals with unknown orientations from underdetermined systems of linear measurements.
method Equivariant variational autoencoder as a generative prior for compressed sensing.
result Signals with unknown orientations can be recovered using iterative gradient descent on the latent space of equivariant models.
Paper proposes robust compressed sensing using generative models.
problem Estimating high-dimensional vectors from noisy linear equations with heavy-tailed or outlier data.
method Inspired by Median-of-Means (MOM), proposes an algorithm for robust recovery.
result Guarantees recovery for heavy-tailed data, even in the presence of outliers.
This paper proposes a simple adaptive sensing and group testing algorithm for sparse signal recovery. The algorithm, termed Compressive Adaptive Sense and Search (CASS), is shown to be near-optimal in that it succeeds at the lowest possible signal-to-noise-ratio (SNR) levels, improving on previous work in adaptive comp…
Sparse linear regression with compressed and low-precision data via concave quadratic programmingmath.OC
New method recovers sparse vectors from compressed, noisy data.
problem Recovering sparse vectors from compressed and noisy measurements.
method Non-convex quadratic programming exploiting prior magnitude information.
result More efficient support recovery with sufficient conditions for success.
We study compressing empirical measures in finite RKHSs using convex optimization.
problem Efficiently approximating empirical measures in high-dimensional spaces.
method Convex optimization and lower bounds on ball size.
result High probability lower bounds on ball size under various conditions.
We present an information-theoretic framework for sequential adaptive compressed sensing, Info-Greedy Sensing, where measurements are chosen to maximize the extracted information conditioned on the previous measurements. We show that the widely used bisection approach is Info-Greedy for a family of -sparse signals b…
Neural Normalized Compression Distance and the Disconnect Between Compression and Classificationcs.LG
Neural NCD reveals LLMs don't compress well for classification.
problem The disconnect between compression and classification in neural networks.
method Developed Neural NCD to compare LLMs to classic algorithms, finding classification accuracy not correlated with compression rate.
result Classification accuracy is not predictable by compression rate alone, challenging current understanding.
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
problem Estimating sparse vectors from noisy sign measurements in 1-bit compressed sensing.
method Binary Iterative Hard Thresholding (BIHT) algorithm, using Gaussian matrices and high-dimensional geometry analysis.
result BIHT provides estimates within ε+τ error with τ-fraction of incorrect measurements, maintaining universality of measurements.
We consider a decomposition method for compressive streaming data in the context of online compressive Robust Principle Component Analysis (RPCA). The proposed decomposition solves an - cluster-weighted minimization to decompose a sequence of frames (or vectors), into sparse and low-rank components, from com…
Proposes a new linearity-based neural network compression method.
problem Reduction of neural network model size while maintaining accuracy.
method Integrates linearity-based intuition with ReLU activation functions to merge layers.
result Achieves up to 75% reduction in model size without loss of accuracy.
Quantum TNCS uses machine learning to efficiently transmit data.
problem Efficient quantum communication of large datasets.
method Combining compressed sensing, tensor networks, and machine learning.
result High efficiency and accuracy in transmitting information.
Adaptive-Rate Sparse Signal Reconstruction With Application in Compressive Background Subtractionmath.OC
We propose and analyze an online algorithm for reconstructing a sequence of signals from a limited number of linear measurements. The signals are assumed sparse, with unknown support, and evolve over time according to a generic nonlinear dynamical model. Our algorithm, based on recent theoretical results for -$…
Estimating the level set of a signal from measurements is a task that arises in a variety of fields, including medical imaging, astronomy, and digital elevation mapping. Motivated by scenarios where accurate and complete measurements of the signal may not available, we examine here a simple procedure for estimating the…
Optimal Network Compressionq-fin.RM
Paper tackles optimal network compression for financial systems.
problem Optimal network compression for financial systems under shocks.
method Formulated as an NP-hard problem, studied systemic risk measures, and analyzed specific networks.
result Systemic fragility results no longer hold generally under shocks and heterogeneous networks.
Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descentcs.LG
Faster reconstruction of compressed signals using conditional GAN and NPGD.
problem Recovering compressed signals from measurements.
method Network-based projected gradient descent (NPGD) combined with measurement-conditional generative adversarial networks (GANs/BEGANs).
result Significant speed-up in reconstruction (up to 140-175 times faster).
This paper investigates methods for quantifying similarity between audio signals, specifically for the task of of cover song detection. We consider an information-theoretic approach, where we compute pairwise measures of predictability between time series. We compare discrete-valued approaches operating on quantised au…
This paper proposes to perform authorship analysis using the Fast Compression Distance (FCD), a similarity measure based on compression with dictionaries directly extracted from the written texts. The FCD computes a similarity between two documents through an effective binary search on the intersection set between the …
Compressed sensing (CS) provides an elegant framework for recovering sparse signals from compressed measurements. For example, CS can exploit the structure of natural images and recover an image from only a few random measurements. CS is flexible and data efficient, but its application has been restricted by the strong…
Untrained neural networks can recover natural images from few measurements.
problem Recovering natural images from a small number of measurements.
method Gradient descent on un-trained convolutional neural networks.
result Untrained neural networks can approximate reconstruct signals/images from a near minimal number of random measurements.
We develop a new compressive sensing (CS) inversion algorithm by utilizing the Gaussian mixture model (GMM). While the compressive sensing is performed globally on the entire image as implemented in our lensless camera, a low-rank GMM is imposed on the local image patches. This low-rank GMM is derived via eigenvalue th…
Study on recovering supports of multiple sparse vectors from mixed linear measurements.
problem Recovering supports of multiple sparse vectors from a mixture of linear measurements.
method Developed algorithms to identify the support of all component vectors using polynomial and quasi-polynomial number of measurements.
result Polynomial and quasi-polynomial number of measurements sufficient for recovering the supports of all component vectors.
Proposes a link between randomness and compression in deep learning.
problem Improving efficiency in deep learning training.
method Introduces a novel tomographic compression framework called Dual Tomographic Compression (DTC).
result Demonstrates high correlation between learning performance and Gibbs entropy over compression ratios.
The goal of compressed sensing is to estimate a vector from an underdetermined system of noisy linear measurements, by making use of prior knowledge on the structure of vectors in the relevant domain. For almost all results in this literature, the structure is represented by sparsity in a well-chosen basis. We show how…