The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…
The paper uses information geometry to analyze model compression techniques, focusing on operator factorization.
problem Efficiently compressing deep learning models for resource-constrained devices.
method Information geometry applied to model compression, focusing on operator factorization.
result Iterative methods are crucial for fine-tuning models, especially when compression ratios are fixed.
A new method compresses point clouds efficiently, outperforming existing techniques.
problem Efficiently compressing large point cloud datasets for VR applications.
method Learned convolutional transforms and uniform quantization for joint rate and distortion optimization.
result Significant rate-distortion improvement (51.5% BDBR savings) on Microsoft Voxelized Upper Bodies dataset.
GeoIB uses information geometry to control compression in deep learning models.
problem The indirect and biased nature of traditional IB implementations in deep learning.
method GeoIB uses Fisher-Rao and Jacobian-Frobenius terms to control information compression directly.
result GeoIB achieves better trade-off between accuracy and compression than traditional IB methods.
Diffusion maps are a commonly used kernel-based method for manifold learning, which can reveal intrinsic structures in data and embed them in low dimensions. However, as with most kernel methods, its implementation requires a heavy computational load, reaching up to cubic complexity in the number of data points. This l…
We show that a pseudo-Anosov map on a boundary component of an irreducible 3-manifold has a power that partially extends to the interior if and only if its (un)stable lamination is a projective limit of meridians. The proof is through 3-dimensional hyperbolic geometry, and involves an investigation of algebraic limits …
Computational Fluid Dynamics (CFD) is a hugely important subject with applications in almost every engineering field, however, fluid simulations are extremely computationally and memory demanding. Towards this end, we present Lat-Net, a method for compressing both the computation time and memory usage of Lattice Boltzm…
A novel method compresses point cloud attributes by folding them onto a 2D grid.
problem Efficiently compressing point cloud attributes for storage and transmission.
method Interpreting point clouds as 2D manifolds, folding onto a grid, and mapping attributes to the grid using optimized methods.
result The proposed folding-based approach achieves performance comparable to state-of-the-art codecs.
Paper models and compresses wideband CSI feedback in FDD MIMO systems.
problem Fundamental limits of channel state information (CSI) feedback in FDD massive MIMO systems.
method Modeling CSI as a Gaussian-mixture source with latent geometry states, proposing Gaussian-mixture transform coding (GMTC).
result Near-optimal CSI compression achieved through state-adaptive transform coding without large neural encoders.
Paper improves deep point cloud compression techniques.
problem Efficiently compressing 3D point cloud data for various applications.
method Integrates scale hyperprior model, deeper transforms, focal loss, and optimal thresholding.
result Achieves significant BD-PSNR gains over existing methods.
For a real or complex semisimple Lie group G G G and two nested parabolic subgroups Q ⊂ P ⊂ G Q\subset P\subset G Q ⊂ P ⊂ G , we study parabolic geometries of type ( G , Q ) (G,Q) ( G , Q ) . Associated to the group P P P , we introduce a class of relative natural bundles and relative tractor bundles and construct some basic invariant differential operators on …
Unified framework for distributed compressed SGD under ( L 0 , L 1 ) (L_0, L_1) ( L 0 , L 1 ) -smoothness.
problem Understanding the joint effect of batch noise, adaptivity, and compression in distributed stochastic optimization.
method Developed a unified theoretical framework using SDEs that incorporate curvature-dependent terms.
result Normalizing updates in DCSGD stabilizes convergence, with normalization degree determined by noise structure and landscape regularity.
Discrete Flow Maps bypass sequential prediction limits for parallel text generation.
problem Sequential autoregressive prediction limits large language model speed.
method Flow Maps compress generative trajectories into single-step mappings.
result Discrete Flow Maps surpass previous state-of-the-art results in discrete flow modeling.
FibQuant improves KV-cache compression for long-context inference.
problem Memory traffic bottleneck in long-context inference due to KV cache growth.
method Introduces FibQuant, a universal vector quantizer that combines Beta-quantile radii, Fibonacci/Roberts-Kronecker directions, and Lloyd-Max refinement.
result FibQuant achieves high compression rates with minimal loss in attention cosine similarity.
This paper improves neural network compression by using robust low-rank approximations.
problem Neural network compression sensitivity to outliers.
method Introduces robust low-rank approximations using ℓ p \ell_p ℓ p norms (for p ∈ [ 1 , 2 ] p\in [1,2] p ∈ [ 1 , 2 ] ) and provides efficient algorithms. result Achieves up to 28% compression with minimal accuracy loss compared to existing methods.
Study compares hyperbolic and extremal lengths for shortest curves.
problem Comparing hyperbolic and extremal lengths for shortest curves.
method Lower bounds for widths of collars and upper bounds for renormalized volume of Schottky manifolds.
result Upper bounds of renormalized volume in terms of hyperbolic length of compressible curves.
We present a novel condition, which we term the net- work nullspace property, which ensures accurate recovery of graph signals representing massive network-structured datasets from few signal values. The network nullspace property couples the cluster structure of the underlying network-structure with the geometry of th…
Geometric framework for Newton's equations on diffeomorphism groups.
problem Modeling fluid dynamics and related systems on geometric spaces.
method Geodesic approach and infinite-dimensional information geometry.
result Unified framework for various fluid dynamics equations.
Flexible framework compresses models using LC algorithm.
problem Efficiently compressing neural networks for resource constraints.
method Decouples learning and compression steps with alternating L and C phases.
result Compressed models maintain performance and accuracy.
Reduces multiclass and regression compression schemes to binary ones.
problem Developing efficient learning algorithms for multiclass and regression problems.
method Reduces sample compression schemes for binary classes to multiclass and regression settings.
result Establishes new compression schemes for multiclass and regression problems.
Proposes a link between randomness and compression in deep learning.
problem Improving efficiency in deep learning training.
method Introduces a novel tomographic compression framework called Dual Tomographic Compression (DTC).
result Demonstrates high correlation between learning performance and Gibbs entropy over compression ratios.
Unified framework for non-compressed networks from compression bounds.
problem Generalization of large deep neural networks with huge model size.
method Unified framework converting compression based bounds to non-compressed networks.
result Unified data-dependent generalization error bound with tighter evaluation.
This paper presents a variational and multisymplectic formulation of both compressible and incompressible models of continuum mechanics on general Riemannian manifolds. A general formalism is developed for non-relativistic first-order multisymplectic field theories with constraints, such as the incompressibility constr…
Paper introduces a new adaptive gradient method with gradient compression for distributed training.
problem Communication overhead in distributed machine learning systems.
method Adaptive gradient method with gradient compression, scalable system BytePS-Compress.
result Convergence rate of O ( 1 / T ) \mathcal{O}(1/\sqrt{T}) O ( 1/ T ) for non-convex problems. Galen algorithm compresses neural networks for specific hardware with reduced latency.
problem Finding optimal compression policies for neural networks on specific hardware.
method Reinforcement learning using pruning and quantization to optimize inference latency.
result Compressed ResNet18 for ARM processor reduced inference latency by 80%.
Paper compresses neural network weight-updates for image artifacts removal.
problem Efficiently compressing neural network weight-updates for image artifacts removal.
method Fine-tuning a pre-trained artifact removal network on target data with a compression objective that encourages sparse and quantized weight-updates.
result Achieves reconstruction quality comparable to traditional codecs at comparable bitrates.
HiLLoC compresses large images losslessly using VAEs.
problem Lossless compression of large color photographs.
method Fully convolutional VAE models trained on ImageNet are applied to lossless compression.
result Achieves state-of-the-art compression for full-size ImageNet images.
Neural NCD reveals LLMs don't compress well for classification.
problem The disconnect between compression and classification in neural networks.
method Developed Neural NCD to compare LLMs to classic algorithms, finding classification accuracy not correlated with compression rate.
result Classification accuracy is not predictable by compression rate alone, challenging current understanding.
Boundary-induced apparent risk aversion in non-ergodic growth models.
problem Risk aversion in multiplicative growth systems with absorbing boundaries.
method Exact lattice propagation and analysis of binary multiplicative processes.
result Optimal exposure is compressed near absorbing boundaries, mimicking risk aversion.
BDC compresses both sample size and dimensionality of large datasets.
problem Large datasets in both sample size and dimensionality.
method Two-stage framework using Decoded MMD, Reconstruction MMD, and Encoded MMD.
result BDC achieves comparable or superior performance with lower cost and higher compression rates.
This paper compresses RNNs for IoT devices by 15-38x using Kronecker products.
problem Resource constraints on IoT devices make RNNs difficult to deploy.
method Kronecker product (KP) for compressing RNN layers by 15-38x with minimal accuracy loss.
result Kronecker product can compress RNNs by 50x when quantized to 8-bits.
DoCoFL compresses model updates for cross-device federated learning.
problem Downlink compression for cross-device federated learning where clients may appear only once.
method Proposes DoCoFL framework for downlink compression in cross-device federated learning.
result Significant bi-directional bandwidth reduction with competitive accuracy.
Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…
Paper presents a new video compression method using autoencoders.
problem Efficient video compression with reduced quality loss.
method 3D autoencoder with discrete latent space and autoregressive prior trained jointly.
result Method outperforms state-of-the-art learned video compression networks.
New taxonomy and evaluation of neural network compression methods.
problem Efficiency of deep neural networks in real-world applications.
method Categorization and evaluation of tensor factorization and probabilistic compression methods.
result SVD and probabilistic compression methods are complementary and give the best results.
Proposes a method to train neural networks directly on compressed text data.
problem Training neural networks on compressed text data without decompression.
method Introduces composer modules to encode symbols from grammar compression rules into vector representations.
result Demonstrates that the proposed method can achieve both memory and computational efficiency while maintaining moderate performance.
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.
problem Reducing communication overhead in Federated Learning with data heterogeneity and partial participation.
method Revisit and simplify stochastic controlled averaging, proposing SCALLION and SCAFCOM for unbiased and biased compression.
result SCALLION and SCAFCOM outperform existing methods in communication and computation complexities.
Gradient descent with compressed iterates reduces model size for federated learning.
problem Efficiently compressing model sizes in federated learning.
method Gradient descent with iterates compressed using lossy randomized techniques.
result Improves model compression and aggregation in federated learning.
New uncertainty principle limits compression in distributed learning, suggesting optimal methods.
problem Minimizing communication cost while maintaining message quality in distributed learning.
method Formalized uncertainty principle for compression operators, introduced Kashin compression.
result Explicit formula for Kashin compression's variance bound, dimension independent.
Proposes a new linearity-based neural network compression method.
problem Reduction of neural network model size while maintaining accuracy.
method Integrates linearity-based intuition with ReLU activation functions to merge layers.
result Achieves up to 75% reduction in model size without loss of accuracy.
Paper introduces compressibility loss for learning sparse neural network weights.
problem Learning highly compressible neural network weights.
method Applying a compressibility loss to minimize the negated sparsity of the signal.
result At critical points, weight vectors are ternary signals with a sparsity directly related to the objective value.
Paper compresses RNNs for resource-constrained devices.
problem Difficulty deploying RNNs on resource-constrained devices.
method Uses Kronecker product (KP) to compress RNN layers.
result KP compresses RNN layers by 16-38x with minimal accuracy loss.
A new method compresses NLP networks by using multiple subspaces instead of a single one.
problem Large errors in compressing NLP networks using a single subspace.
method Projective clustering to find multiple subspaces that minimize squared distances.
result Networks are more accurate and smaller compared to standard matrix factorization.
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…
Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification…
Unified framework compresses GANs up to 47x with minimal quality loss.
problem High parameter complexity of GANs for resource-constrained devices.
method Unified optimization framework combining model distillation, channel pruning, and quantization.
result 47x compression of CartoonGAN with minimal quality degradation.
Dream Distillation compresses models without data, achieving high accuracy.
problem Model compression without real data.
method Data-independent model compression framework.
result Achieves 88.5% accuracy on CIFAR-10 test set.