A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new framework compresses neural networks using sparse optimization.
Paper tackles optimal network compression for financial systems.
Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…
LEAD algorithm speeds up decentralized optimization with compression.
This work explores the non-convex optimization in compressive learning and the performance of heuristics.
Unified framework compresses GANs up to 47x with minimal quality loss.
Paper proposes a compression principle for neural networks using Bayesian optimization.
Paper improves communication in distributed optimization, reducing worker-to-server data exchanges.
Physics-inspired methods optimize SVD compression of LLMs.
Wavelets help compress neural networks efficiently.
DP-Net uses dynamic programming for efficient deep neural network compression.
This paper introduces matrix product state (MPS) decomposition as a new and systematic method to compress multidimensional data represented by higher-order tensors. It solves two major bottlenecks in tensor compression: computation and compression quality. Regardless of tensor order, MPS compresses tensors to matrices …
Asynchronous computation and gradient compression have emerged as two key techniques for achieving scalability in distributed optimization for large-scale machine learning. This paper presents a unified analysis framework for distributed gradient methods operating with staled and compressed gradients. Non-asymptotic bo…
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization. Despite the significant attention received, current compression schemes either do not scale well or fail to achieve the target test accuracy. We propose a new low-rank gradient compressor based on …
NAC-FL optimizes model updates in FL systems by adapting compression to network congestion.
Deep model compression has been extensively studied, and state-of-the-art methods can now achieve high compression ratios with minimal accuracy loss. This paper studies model compression through a different lens: could we compress models without hurting their robustness to adversarial attacks, in addition to maintainin…
New method compresses facial videos using GANs and latent space optimization.
The paper analyzes geometric densities and compression radii for knot types.
We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size- agnostic sample compression schemes are of…
The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.
GeneCAI optimizes DNN compression hyper-parameters for mobile devices.
COMP-AMS optimizes distributed training with compressed gradients, achieving similar accuracy with less communication.
The paper uses information geometry to analyze model compression techniques, focusing on operator factorization.
This paper proposes a simple adaptive sensing and group testing algorithm for sparse signal recovery. The algorithm, termed Compressive Adaptive Sense and Search (CASS), is shown to be near-optimal in that it succeeds at the lowest possible signal-to-noise-ratio (SNR) levels, improving on previous work in adaptive comp…
Goal: This paper deals with the problems that some EEG signals have no good sparse representation and single channel processing is not computationally efficient in compressed sensing of multi-channel EEG signals. Methods: An optimization model with L0 norm and Schatten-0 norm is proposed to enforce cosparsity and low r…
We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to ad…
Paper introduces MoTEF for faster decentralized optimization with compressed communication.
Multiple gossip steps improve decentralized optimization convergence.
Real-world data typically contain repeated and periodic patterns. This suggests that they can be effectively represented and compressed using only a few coefficients of an appropriate basis (e.g., Fourier, Wavelets, etc.). However, distance estimation when the data are represented using different sets of coefficients i…
In this work, we propose an end-to-end block-based auto-encoder system for image compression. We introduce novel contributions to neural-network based image compression, mainly in achieving binarization simulation, variable bit rates with multiple networks, entropy-friendly representations, inference-stage code optimiz…
In order to mitigate the high communication cost in distributed and federated learning, various vector compression schemes, such as quantization, sparsification and dithering, have become very popular. In designing a compression method, one aims to communicate as few bits as possible, which minimizes the cost per commu…
New method reduces communication costs in distributed nonconvex optimization.
Network agents solve adaptive regression problems with compressed signals.
Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.
Optimizes MCMC output compression by selecting a subset of states.
New algorithm reduces communication in distributed learning by sharing compressed beliefs.
Deep Neural Networks (DNNs) are applied in a wide range of usecases. There is an increased demand for deploying DNNs on devices that do not have abundant resources such as memory and computation units. Recently, network compression through a variety of techniques such as pruning and quantization have been proposed to r…
New method optimizes lossy compression models more effectively.
This paper analyzes error feedback in compressed federated learning for non-convex optimization problems.
Optimizes tensor rank selection for neural network compression.
The online problem of computing the top eigenvector is fundamental to machine learning. In both adversarial and stochastic settings, previous results (such as matrix multiplicative weight update, follow the regularized leader, follow the compressed leader, block power method) either achieve optimal regret but run slow,…
Designing a data sharing mechanism without sacrificing too much privacy can be considered as a game between data holders and malicious attackers. This paper describes a compressive adversarial privacy framework that captures the trade-off between the data privacy and utility. We characterize the optimal data releasing …
End-to-end meta-learned system for image compression.
Clapping reduces memory usage in distributed optimization by reusing data samples.
Proposes a new linearity-based neural network compression method.
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…
We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed sample…
We propose a general framework for neural network compression that is motivated by the Minimum Description Length (MDL) principle. For that we first derive an expression for the entropy of a neural network, which measures its complexity explicitly in terms of its bit-size. Then, we formalize the problem of neural netwo…