This work tackles extractive compression by formulating it as tree transduction.
problem Extractive compression as a challenging natural language processing problem.
method Formulated as a parse tree transduction problem, using a deep neural model with Long Short-Term Memory extended to consider parent-child relationships.
result Achieves state-of-the-art performance on sentence compression benchmarks.
Neural model extracts tokens as latent variables for text compression.
problem Compressing text using neural models.
method Extracting tokens with highest tf-idf scores or highest loss from a bidirectional language model.
result Extracting tokens as latent variables significantly outperforms state-of-the-art methods.
Improved gesture recognition using compressed domain signals.
problem Efficient gesture recognition in compressed domain.
method Direct gesture feature extraction from compressed measurements, improved DTW-based K-NN classifiers.
result Strong support for the proposed algorithm in simulations and hardware.
A method extracts binary features directly from CS measurements for compressive image classification.
problem Efficiently classify images using compressive sensing without reconstruction.
method DCT-based approach for binary feature extraction from CS measurements, feature fusion with CNN features.
result Fused features outperform state-of-the-art methods in image classification.
This paper proposes to perform authorship analysis using the Fast Compression Distance (FCD), a similarity measure based on compression with dictionaries directly extracted from the written texts. The FCD computes a similarity between two documents through an effective binary search on the intersection set between the …
FRAE compresses speech data efficiently with low bitrate.
problem Online compression of sequential data with temporal dependency.
method Feedback Recurrent AutoEncoder architecture with learned prior and entropy coder.
result Achieves high-quality speech waveforms at low fixed bitrate and variable bitrate.
New decoder improves robustness of compressive clustering.
problem Designing robust decoders for compressive clustering.
method Inspired by mean shift, proposes a new decoder.
result Significantly improves recovery of clusters from smaller sketches.
Entangled watermarks improve model defense against extraction attacks.
problem Model extraction attacks risk leaking valuable intellectual property.
method Entangled Watermarking Embeddings (EWE) leverages model capacity to overfit to outlier input-output pairs.
result The approach successfully claims model ownership with high confidence and minimal performance loss.
Extract class-specific subnetworks from neural models for better understanding and improved explanations.
problem Understanding and explaining the complex behavior of deep neural networks.
method For each semantic class, extract a class-specific subnetwork with a compressed structure that maintains comparable performance.
result Extracted subnetworks improve explanation saliency and adversarial example detection.
Improves document summarization by combining word embeddings and n-grams.
problem Exact word matching fails to measure semantic similarity between sentences.
method Uses deep embedding features and tf-idf features to improve sentence similarity measure; builds an improved sentence similarity graph; employs a submodular objective function; develops a Transformer-based compression model.
result Outperforms tf-idf based approach and achieves state-of-the-art performance on DUC04 dataset.
G-CREWE efficiently aligns large networks using node embeddings and compression.
problem Efficiently aligning large networks for various applications.
method Uses node embeddings and compression to align networks at fine and coarse resolutions.
result G-CREWE achieves efficient and accurate network alignment, twice as fast as existing methods.
This work automates CNN model compression for mobile devices.
problem Deploying trained CNNs to mobile devices requires balancing speed, memory, and accuracy.
method Reinforcement learning with architecture search and knowledge distillation.
result An automated model compression algorithm improves the trade-off between speed, memory, and accuracy.
New method compresses deep learning layers using tensor decomposition.
problem Reduction of computation cost and interpretability for tensor data.
method CP-decomposition to compress convolutional layers in deep learning.
result Reduces model complexity and maintains prediction performance.
RNNs are suboptimal at compressing past sensory inputs for future prediction.
problem RNNs do not optimally compress past sensory inputs for future prediction.
method Investigated RNNs trained with maximum likelihood and found they extract unnecessary information. Injected noise into hidden states to improve performance.
result Injecting noise into RNN hidden states improves predictive information, sample quality, likelihood, and classification performance.
Scattering representations simplify SBI for images without extra compression.
problem Efficiently performing simulation-based inference on images with limited data.
method Use scattering representations for compression and learning, combined with spatial averaging and expressive density estimators.
result Scattering representations provide more information than traditional methods, without requiring additional simulations.
We present an information-theoretic framework for sequential adaptive compressed sensing, Info-Greedy Sensing, where measurements are chosen to maximize the extracted information conditioned on the previous measurements. We show that the widely used bisection approach is Info-Greedy for a family of k-sparse signals b…
The paper defines and studies discrete p-density and compression-radius profiles of lattice knots.
problem Understanding geometric properties of lattice knots.
method Develops a framework for discrete p-density and compression-radius profiles of lattice knots, studying them on length-filtered sets and finite move-graph exploration.
result Density and compression-radius values are not monotone, illustrating distinct optimization problems.
Machine vision-guided 3D medical image compression improves segmentation accuracy.
problem High data traffic and computation costs in cloud-based medical image analysis.
method Developed a machine vision-oriented 3D image compression framework for medical segmentation.
result Significantly higher segmentation accuracy at the same compression rate or better compression rate under the same accuracy.
Gaussian Process upsampling boosts OCR accuracy from low-res images.
problem Low-quality and downsampled image data hinders OCR accuracy.
method Gaussian Process upsampling model for improving OCR on low-resolution documents.
result Upsampling improves OCR accuracy on low-resolution images.
New method extracts cosmological information from dark matter halo catalogues using graph neural networks.
problem Quantifying cosmological information from large-scale structure data.
method Implicit likelihood approach with Information Maximising Neural Networks (IMNNs) on graph representations of dark matter halo catalogues.
result Graph neural network summaries can extract information from noisy catalogues and improve parameter constraints.
Maximizing margins leads to lossless compression of training data.
problem Generalization in supervised learning.
method Information-theoretic interpretation of margin maximization.
result Margin maximization is a form of lossless maximal compression.
Deep learning detects pneumonia with 36x compression on low-power devices.
problem High accuracy pneumonia detection on low-power embedded devices.
method Structured weight pruning method for compression and maintaining accuracy.
result Up to 36x compression ratio with no accuracy loss.
Spectral clustering is one of the most widely used techniques for extracting the underlying global structure of a data set. Compressed sensing and matrix completion have emerged as prevailing methods for efficiently recovering sparse and partially observed signals respectively. We combine the distance preserving measur…
A new method for efficient distributed optimization using trajectory-based normalized gradients.
problem Efficient communication in large-scale distributed optimization.
method A bijective mapping between gradient distributions, using normalized gradients and dynamically extracted references.
result Trajectory-based normalized gradients (TNG) improves communication efficiency in distributed optimization.
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
New guarantees for asymmetric sketching in compressive learning.
problem Statistical guarantees for compressive learning with asymmetric feature maps.
method Proves existing guarantees carry over to asymmetric scheme with LPD property, applies to quantized sketches.
result Existing statistical guarantees for compressive learning extend to asymmetric schemes with controlled error.
Paper uses random projection to preserve subspace structure for efficient data analysis.
problem Efficiently analyzing data with low-dimensional structure.
method Compressed Subspace Learning (CSL) framework based on Johnson-Lindenstrauss property.
result Random projection preserves the UoS structure of data, enabling efficient analysis.
Faster and accurate JPEG2000 image classification without reconstruction.
problem Efficiently classify j2k-compressed images without reconstructing them.
method Train a deep CNN using DWT coefficients directly from j2k-compressed images, using different augmentation techniques.
result Achieved faster and more accurate classification of j2k images without additional computation.
CRCCA framework improves non-linear CCA with compressed representations.
problem Non-linear CCA for multi-view data with limited samples.
method Information-theoretic compressed representation framework (CRCCA) based on lattice quantization.
result The CRCCA framework provides theoretical bounds and optimality conditions, offering a flexible and computationally efficient solution.
Quantum computing speeds up analysis of financial stochastic processes.
problem Challenging simulation and analysis of continuous time stochastic processes.
method Established a quantum framework for efficient state preparation and information extraction.
result Extraction of path-dependent and history-sensitive information from stochastic processes efficiently.
Extends MMF to nonsymmetric matrices for hierarchical structure.
problem Capturing hierarchical structure in nonsymmetric matrices.
method Multiresolution Matrix Factorization (MMF) extended to nonsymmetric matrices.
result Effective for matrix compression tasks, outperforming low-rank methods.
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder and parallel multi-scale lossless coder. The multi-scale lossy autoencoder extracts the multi-scale image features to quantized variables a…
The front-end factor analysis (FEFA), an extension of principal component analysis (PPCA) tailored to be used with Gaussian mixture models (GMMs), is currently the prevalent approach to extract compact utterance-level features (i-vectors) for automatic speaker verification (ASV) systems. Little research has been conduc…
This paper focuses on the estimation of the sample covariance matrix from low-dimensional random projections of data known as compressive measurements. In particular, we present an unbiased estimator to extract the covariance structure from compressive measurements obtained by a general class of random projection matri…
New methods improve tree ensemble models by compressing them while maintaining accuracy.
problem Theoretical understanding and practical compression of tree ensembles like random forests and gradient boosting machines.
method Spectral perspective on tree ensembles, deriving minimax rates and developing compression schemes.
result Leading eigenfunctions/singular vectors capture dominant predictive directions, leading to smaller, competitive models.
New method extracts latent variables from process data using autoencoders.
problem Extracting useful information from diverse, noisy, and nonstandard response processes.
method Sequence-to-sequence autoencoder to compress response processes into standard numerical vectors.
result The latent variables extracted from response processes are useful for understanding complex skills.
AIB method improves robustness against adversarial perturbations.
problem Optimizing the IB principle for better robustness and understanding compression effects.
method Proposes adversarial information bottleneck (AIB) method to optimize IB principle without explicit distribution assumptions.
result Demonstrates effectiveness in learning more invariant representations and mitigating adversarial perturbations.
Proposes a new method to selectively access privileged information in reinforcement learning.
problem Selective compression of privileged information in reinforcement learning.
method Formulates a variational bandwidth bottleneck to decide stochastically whether to access privileged information.
result Improves generalization and reduces access to costly information in reinforcement learning experiments.
Deep networks learn to compress high-dimensional data into lower dimensions.
problem Learning to identify and extract task-relevant variables from high-dimensional data.
method Applied state-of-the-art techniques for intrinsic dimensionality estimation.
result Neural networks first expand dimensionality in initial layers, then compress in later layers.
A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations. The world model's extracted features are fed into compact and simple policies trained by evolution, achieving state of the art resul…
This paper presents a method to summarize directed graphs while preserving edge information.
problem Summarizing directed graphs while maintaining edge directionality.
method A model based on minimizing reconstruction error with non-negative constraints, related to Max-Cut criterion, using multiplicative update algorithms.
result The proposed method identifies compressed nodes and directed compressed relations, providing a more accurate representation of directed graphs.
Designs a classifier for malware detection using MDL principle.
problem Static malware detection in PE files.
method Clustering, closed frequent pattern mining, MDL principle for compression.
result Our classifier performs close to deep neural networks.
Paper proposes a new method for training small models on regression problems.
problem Training small models for regression problems with noisy labels.
method Developed a new loss function and a multi-task network approach.
result Improved model accuracy on various datasets, consistent across different levels of annotation errors.
We explore building generative neural network models of popular reinforcement learning environments. Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment. By using features extracted from the world model as inputs to an agent, we c…
New method bypasses global fit for LISA's Galactic binaries, extracting population parameters directly.
problem Disentangling LISA's Galactic binary sources from backgrounds in a computationally intensive process.
method Simulation-based approach using normalizing flow to infer population parameters.
result Direct inference of population parameters from LISA's frequency strain series.
New methods identify local clusters in graphs with few labels.
problem Identifying specific substructures in large graphs without additional structural information.
method Random sampling, diffusion, and overlap analysis of local clusters.
result Proves the correctness of the proposed methods and achieves state-of-the-art results.
The recent developments of basis pursuit and compressed sensing seek to extract information from as few samples as possible. In such applications, since the number of samples is restricted, one should deploy the sampling points wisely. We are motivated to study the optimal distribution of finite sampling points. Formul…
Network Implosion reduces ResNet layers without accuracy loss.
problem High computation costs in Residual Networks.
method Static layer pruning and retraining to erase unimportant layers.
result Reduces ResNet layers by 24.00-42.86% without accuracy drop.