The paper explores mechanisms of market regime transition via slow and fast bifurcations.
problem Understanding and predicting market regime transitions.
method Analyzes slow and fast bifurcations in market dynamics using a Markovian chain diffusion.
result Efficiency of precursors in predicting market regime transitions.
Compressive Transformer learns long-range sequences by compressing past memories.
problem Learning long-range sequences in language and speech models.
method Compressive Transformer compresses past memories for efficient long-range sequence learning.
result State-of-the-art performance on language and speech benchmarks.
Given Poincare spaces M and X, we study the possibility of compressing embeddings of M x I in X x I down to embeddings of M in X. This results in a new approach to embedding in the metastable range both in the smooth and Poincare duality categories.
COIN++ compresses multiple data types efficiently.
problem Handling diverse data modalities in neural compression.
method Implicit neural representations and modulations quantization.
result Significant compression gains with reduced encoding time.
A new density model using Fourier basis achieves better approximations and compression.
problem Approximating multi-modal 1D densities.
method Constrained Fourier basis model for end-to-end training.
result Lower cross entropy compared to deep factorized models.
New video compression method outperforms traditional approaches.
problem Efficient video compression in low latency mode.
method Feedback Recurrent Autoencoder network architecture.
result State of the art MS-SSIM/rate performance on UVG dataset.
PCRs compress data for deep learning, reducing training time.
problem Efficiently training deep learning models over large datasets.
method Combining progressive compression with an efficient storage layout.
result PCRs can tolerate up to 50% compression without significantly affecting training accuracy.
New coherence parameter for GNNs with Fourier measurements improves signal recovery.
problem Characterizing generative compressed sensing with Fourier measurements.
method Subspace counting arguments and high-dimensional probability theory.
result First known restricted isometry guarantee for generative compressed sensing with subsampled isometries.
New methods show deep networks can compress info without saturating activations.
problem Understanding how neural networks generalize and compress information.
method Adaptive mutual information estimation techniques for neural networks.
result Compression occurs in networks with non-saturating activation functions.
New algorithm reduces communication in distributed learning by sharing compressed beliefs.
problem Efficiently learning from private data in a distributed setting with large hypothesis sets.
method Proposes a belief update rule for distributed cooperative learning with compressed (sparse or quantized) beliefs.
result Beliefs converge almost surely to optimal hypotheses with a linear concentration rate.
A new method compresses point clouds efficiently, outperforming existing techniques.
problem Efficiently compressing large point cloud datasets for VR applications.
method Learned convolutional transforms and uniform quantization for joint rate and distortion optimization.
result Significant rate-distortion improvement (51.5% BDBR savings) on Microsoft Voxelized Upper Bodies dataset.
New method recovers signals from compressed measurements using generative networks with contractive layers.
problem Signal recovery from compressed measurements with generative network priors.
method Developed a new matrix concentration inequality (R2WDC) to relax expansivity conditions for generative networks.
result Signals in the range of a Gaussian generative network can be recovered from few linear measurements with contractive layers.
This paper proposes an automatic neural network compression method.
problem Reducing resource requirements for deep neural networks on resource-constrained devices.
method Jointly prunes and quantizes neural networks without manual hyper-parameter tuning.
result Significant reduction in model size with minimal accuracy loss.
New method connects compression bodies through cone manifolds.
problem Understanding how compression bodies can be transformed.
method Using cone manifold holonomy groups and standard CAT(0) space techniques. result Realized all edges in compression body graph through paths.
A new algorithm for compressing latent representations in deep models.
problem Compressing continuous latent representations in deep models.
method Separates model design and training from quantization; uses adaptive quantization based on posterior uncertainty.
result Image compression with the proposed algorithm outperforms JPEG over a wide range of bit rates.
This work characterizes model compression techniques for deep learning on embedded systems.
problem Efficient deep inference on resource-constrained devices.
method Extensive experiments on 11 neural network architectures using data quantization and pruning.
result Opportunities to achieve fast deep inference on embedded systems exist but require careful compression settings.
CSDM integrates compressed sensing into diffusion models for faster data generation.
problem Efficiently generating synthetic data in high-dimensional spaces.
method Integrating compressed sensing into diffusion models (CSDM) to reduce dimensionality and accelerate inference.
result Achieves provably faster convergence and better latent space dimension selection.
DeepThin compresses deep neural networks, improving performance and reducing resource usage.
problem Efficiently compressing large neural networks for mobile devices.
method Combining rank factorization with a reshaping process to add nonlinearity.
result DeepThin achieves significant improvements in word error rates and test loss compared to existing methods.
The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.
problem Theoretical and practical improvements in compressed sensing with optimized sampling schemes.
method Theoretical analysis and empirical experiments with optimized sampling schemes for subsampled unitary matrices.
result The error caused by measurement noise vanishes with an increasing number of measurements for optimized sampling schemes, assuming Gaussian noise.
Echo noise improves compression in autoencoders with exact rate-distortion.
problem Limitations of Gaussian noise in lossy compression and VAEs.
method Introduces Echo noise, a data-driven noise channel with exact mutual information.
result Echo noise leads to improved bounds on log-likelihood and dominates VAEs.
Paper uses SGLD to recover signals from generative models, proving convergence under mild conditions.
problem Signal recovery from generative priors in compressed sensing.
method Stochastic Gradient Langevin Dynamics (SGLD) for signal recovery.
result SGLD converges to the true signal under mild assumptions on the generative model.
This paper compresses neural networks by permuting and quantizing weights.
problem Efficiently compressing large neural networks for resource-constrained platforms.
method Permuting and quantizing weights, connecting to rate-distortion theory, and using annealed quantization.
result Significant compression with minimal accuracy loss, e.g., 40-70% reduction in gap with uncompressed model.
A new method compresses large datasets for gradient descent algorithms efficiently.
problem Efficiently compress large-scale datasets for gradient descent algorithms.
method Proposes a novel sequential coreset framework based on gradient descent locality.
result Significantly reduces computational complexity and running time.
We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our interest is in describing their convergence with sequence length, assuming no limit…
Improved sampling strategy reduces Fourier measurements for neural network signals.
problem Efficiently sampling signals from neural networks with random Fourier matrices.
method Model-adapted sampling strategy with improved sample complexity.
result Reduced sample complexity from O(kdnα∞²) to O(kdα²₂) measurements.
New algorithm uses untrained neural networks for image recovery, offering better compression.
problem Using untrained neural networks for image recovery and theoretical guarantees.
method Projected gradient descent scheme for solving linear and non-linear inverse problems.
result The method achieves better compression rates for the same image quality compared to hand-crafted priors.
Randomly shuffled kernels can be compressed efficiently.
problem Reducing storage cost of CNN parameters on resource-limited platforms.
method Randomly-shuffled tensor decomposition (RsTD) to embed kernels into random low-rank subspaces.
result CNNs can be significantly compressed even with randomly shuffled kernels, achieving more stable accuracy.
New method reconstructs signals and images from periodic nonlinearities.
problem Reconstructing signals and images from periodic nonlinearities.
method Design of measurement scheme for efficient reconstruction, adaptable to compressive sensing.
result Effective reduction in measurement complexity for HDR imaging with minimal quality loss.
Improves convergence speed in compressive sensing with a new probabilistic approach.
problem Efficiently solving the best subset selection problem in compressive sensing.
method Smooth probabilistic reformulation of ℓ0 regularized regression. result Empirically outperforms existing compressive sensing algorithms across various settings.
Focused quantization reduces CNN models by 18x while maintaining accuracy.
problem Memory and compute resource constraints in deploying CNNs on constrained devices.
method Focused quantization, exploiting weight distributions after pruning, dynamically discovers optimal numerical representations.
result Achieved a 18.08x compression ratio with only 0.24% loss in accuracy in ResNet-50.
New method estimates vectors near generative models without sparsity.
problem Estimating vectors from noisy measurements without sparsity.
method Using generative models for vector estimation with Gaussian measurements.
result Achieves similar recovery guarantees to standard compressed sensing with fewer measurements.
Improved hybrid image compression using deep learning and traditional codecs.
problem Efficiently compressing images with high quality and low bitrates.
method Combining CNN for compact representation, FLIF for base layer, and BPG for enhancement layer.
result Outperforms state-of-the-art schemes in PSNR and MS-SSIM metrics.
The study finds conditions for compressing the hidden dimension of Graph Transformers for transductive learning.
problem The challenge of efficiently analyzing and training Graph Transformers for transductive learning.
method Theoretical bounds on hidden dimension compression for Graph Transformers, considering both sparse and dense variants.
result Theoretical findings on how and under what conditions the hidden dimension of Graph Transformers can be compressed.
Generative model learns compact codes for video recovery.
problem Efficiently represent and reconstruct videos from missing data.
method Generative network trained to map compact latent codes to images, with low-rank and similarity constraints.
result Can recover true video sequences even if not in pretrained network's range.
Understanding how funding and 4H context regulate crypto markets.
problem Analyzing the chaotic appearance of financial markets.
method Observing interactions between market context and capital conditions in the 4H timeframe.
result Ranges in crypto markets are strategic positioning by informed participants, not indecision.
PAQ8 is an open source lossless data compression algorithm that currently achieves the best compression rates on many benchmarks. This report presents a detailed description of PAQ8 from a statistical machine learning perspective. It shows that it is possible to understand some of the modules of PAQ8 and use this under…
Decomposable-Net compresses neural networks without retraining for various sizes.
problem Performance degradation when changing model size after training.
method Decomposes weight matrices via SVD and adjusts ranks for different sizes.
result Maintains and improves performance across multiple model sizes.
Algorithm compresses large matrices by approximating them as low rank and low precision factors.
problem Efficiently storing and processing large matrices with billions of elements.
method Randomized sketching and quantization of matrix columns to achieve low rank and low precision factorization.
result Achieves compression ratios as low as one bit per matrix coordinate while maintaining or improving performance.
New method reduces summary points for datasets while maintaining quality.
problem Thinning datasets to reduce summary points while maintaining quality.
method Low-rank analysis of sub-Gaussian thinning.
result Guarantees high-quality compression for any distribution and kernel.
Novel method decomposes EDA signals to reveal user responses.
problem Superposition of noise components obscures EDA signal information.
method Simple pre-processing followed by compressed sensing decomposition.
result Provably accurate recovery of user responses with reduced noise.
We consider the reconstruction problem in compressed sensing in which the observations are recorded in a finite number of bits. They may thus contain quantization errors (from being rounded to the nearest representable value) and saturation errors (from being outside the range of representable values). Our formulation …
New video compression algorithm outperforms existing codecs.
problem Efficient video compression with low latency.
method End-to-end learned video coding architecture.
result Significantly outperforms existing codecs across bitrate ranges.
Neural codec for high-fidelity audio compression.
problem Efficiently compress audio while maintaining high quality.
method End-to-end neural network architecture with quantized latent space, single multiscale spectrogram adversary, loss balancer mechanism, and lightweight Transformer compression.
result 40% compression with no loss in quality, faster than real-time.
Memory-efficient algorithms reduce kernel matrix size for machine learning.
problem Efficiently handling large kernel matrices in machine learning.
method Hierarchical matrix approximations and clustering techniques.
result Compression improves efficiency without sacrificing prediction accuracy.
Midicoth compresses online probability estimates by correcting prior smoothing biases.
problem Compression inefficiency due to prior smoothing in adaptive models.
method Micro-diffusion denoising applied in a bitwise tree hierarchy.
result Significant compression improvement with reliable calibration.
New method for real-time reconstruction of sparse STEM images from non-rectangular scans.
problem Sparse sampling and non-rectangular scanning in STEM.
method General method for real-time reconstruction of sparsely sampled images from high-speed, non-invasive and diverse scanning pathways.
result Demonstrated on synthetic and experimental STEM data, achieving real-time reconstruction.
Efficient distributed learning with Byzantine-resilient thresholding and error feedback.
problem Byzantine-resilient distributed learning with communication efficiency.
method Simple thresholding for Byzantine mitigation, compressed gradients and norms for aggregation, error feedback.
result Statistical error rate matches Yin et al.~\cite{dong} but with simpler schemes, and improved convergence with error feedback.
SR3 framework improves sparse regression solutions.
problem Sparse regression problems in various fields.
method SR3 framework solves relaxed regularized regression problems.
result SR3 provides superior solutions with faster algorithms.