New method improves accuracy of compressed least-squares solutions.
problem Error and noise introduced by compression in least-squares problems.
method Proposes robust partially-compressed least-squares models and algorithms.
result Robust solutions more accurate than classical compressed variants.
Artemis framework improves distributed learning with bidirectional compression and partial participation.
problem Learning in distributed or federated settings with communication constraints and device partial participation.
method Artemis framework using bidirectional compression, memory mechanism, and Polyak-Ruppert averaging.
result Fast rates of convergence (linear up to a threshold) under weak assumptions on stochastic gradients.
Paper offers robust recovery for 1-bit sensing with partial Gaussian circulant matrices.
problem Accurately recovering vectors from 1-bit measurements using structured matrices.
method Correlation-based optimization with randomly signed partial Gaussian circulant matrices and generative models.
result Recovery guarantees match those for i.i.d. Gaussian matrices but with faster computation.
Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.
problem Reducing communication overhead in Federated Learning with data heterogeneity and partial participation.
method Revisit and simplify stochastic controlled averaging, proposing SCALLION and SCAFCOM for unbiased and biased compression.
result SCALLION and SCAFCOM outperform existing methods in communication and computation complexities.
This paper analyzes error feedback in compressed federated learning for non-convex optimization problems.
problem Reducing communication cost in federated learning with biased gradient compression.
method Proposes Fed-EF, a compressed federated learning scheme with error feedback, and analyzes its convergence rate and performance under partial client participation.
result Fed-EF can match the convergence rate of full-precision FL under data heterogeneity with a linear speedup and no extra slow-down factor due to stale error compensation.
This paper improves network compression techniques using partial regularization.
problem Efficient compression of deep neural networks for reduced computation and memory usage.
method Group lasso regularization and its variants, with an improving framework of partial regularization.
result Partial regularization methods improve classification accuracy on multiple datasets.
This is the second of three papers about the Compression Theorem. We give proofs of Gromov's theorem on directed embeddings [M Gromov, Partial differential relations, Springer--Verlag (1986); 2.4.5 C'] and of the Normal Deformation Theorem [The compression theorem I; 4.7], arxiv:math.GT/9712235.
A new compression method reduces model complexity and improves performance.
problem Efficiently compressing deep neural networks for constrained platforms.
method Partial regularization instead of full regularization.
result Improved performance and generalization ability of pruned models.
FedSGM tackles constrained federated learning with unified framework.
problem Functional constraints, communication bottlenecks, local updates, and partial client participation in federated learning.
method Unified framework based on switching gradient method, incorporating bi-directional error feedback, and soft switching for stability.
result Achieves O ( 1 / T ) \boldsymbol{\mathcal{O}}(1/\sqrt{T}) O ( 1/ T ) convergence rate with high-probability bounds decoupling from sampling noise. New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
New model learns union of transforms for better MRI image reconstructions.
problem Recover images from undersampled data with unknown sparse model.
method Union of sparsifying transforms model, block coordinate descent algorithms.
result Better image reconstructions than single adaptive transform.
We show that a pseudo-Anosov map on a boundary component of an irreducible 3-manifold has a power that partially extends to the interior if and only if its (un)stable lamination is a projective limit of meridians. The proof is through 3-dimensional hyperbolic geometry, and involves an investigation of algebraic limits …
Study ribbon concordance and minimal compressions, proving new results about fibered knots.
problem Understanding ribbon concordance and minimal compressions of surface homeomorphisms.
method Proving monotonicity of simplicial volume and dilatation under ribbon concordance, algorithmic enumeration of minimal compressions.
result Every fibered knot has only finitely many predecessors in the ribbon-concordance partial order.
NESTA accelerates neural networks by compressing Hamming weights.
problem Efficiently computing convolution layers in deep neural networks.
method NESTA reformats convolutions into 3 i m e s 3 3 imes 3 3 im es 3 batches and uses Hamming Weight Compressors to process each batch, approximating partial sums and adding residuals. result Significantly speeds up convolution computations with reduced energy consumption.
Predictive state representations (PSRs) offer an expressive framework for modelling partially observable systems. By compactly representing systems as functions of observable quantities, the PSR learning approach avoids using local-minima prone expectation-maximization and instead employs a globally optimal moment-base…
DiffC compresses images by diffusing Gaussian noise, outperforming state-of-the-art methods.
problem Efficient lossy image compression without transform coding.
method Unconditional diffusion generative models to encode and denoise corrupted pixels.
result DiffC outperforms HiFiC on ImageNet 64x64, achieving a 3 dB gain at high bitrates.
Natural signals and images are well-known to be approximately sparse in transform domains such as Wavelets and DCT. This property has been heavily exploited in various applications in image processing and medical imaging. Compressed sensing exploits the sparsity of images or image patches in a transform domain or synth…
Prototype for adaptive electron microscopy scans reduces dose and time.
problem Reduce electron microscopy scan time and dose with minimal loss.
method Adaptive partial scanning with reinforcement learning.
result Reinforcement learning trained neural network optimizes scan paths.
Links on surfaces have at most two intersecting points.
problem Understanding intersections of links on surfaces.
method Analyzing compressing disks and isotopies of surfaces.
result Surfaces can be isotoped to have a standard tube at crossings.
New method finds compact strong lottery tickets in partially frozen networks.
problem Finding compact strong lottery tickets in randomly pruned networks.
method Freezing a random subset of initial weights in a partially frozen network.
result Better accuracy and compression achieved with frozen networks.
Optimized sampling scheme for compressed sensing combining randomness and determinism.
problem Improving compressed sensing performance with deterministic sampling.
method Optimized sampling scheme combining random and deterministic selection of rows.
result Measurable improvements in image compressed sensing for generative and sparse priors.
Unified approach for robust low rank matrix estimation with adversaries.
problem Robust low rank matrix estimation in the presence of adversaries.
method Unified approach combining Huber loss and nuclear norm penalization.
result Sharp estimation error bounds for matrix compressed sensing and completion.
This paper improves binary embeddings and quantized compressed sensing methods.
problem Distance-preserving binary embeddings and quantization for compressed sensing.
method Quantization of fast Johnson-Lindenstrauss embeddings and bounded orthonormal systems.
result Quantization methods yield reconstruction errors that decay polynomially and exponentially in the number of measurements.
New method compresses GANs by 1669:1 ratio on MNIST, 58:1 on CIFAR-10, 87:1 on Celeb-A.
problem Expensive GANs for low SWAP hardware and real-time applications.
method Knowledge distillation between a student and teacher GAN.
result Compressed GANs outperform standard training methods with a fixed parameter budget.
SA-PEF improves federated learning efficiency by correcting gradient mismatches.
problem Slow decay of residual error in federated learning under non-IID data.
method Integrates step-ahead correction with partial error feedback.
result Achieves faster convergence and target accuracy compared to standard EF.
New compression methods outperform FEFA in ASV tasks with similar accuracy but significantly faster.
problem Improving ASV system development speed while maintaining accuracy.
method Compared and evaluated several supervector compression methods including PPCA, FA, SPPCA, and PPLS.
result Supervector compression approaches are as effective as FEFA in ASV tasks but offer significant speed improvements.
Extends PAC learning theory to handle partial concepts with special properties.
problem Traditional PAC learning theory cannot handle tasks with special data properties.
method Introduces partial concepts and new PAC learning framework.
result Partial concept classes cannot be captured by traditional PAC theory.
New approach quantizes neural nets with guaranteed convergence to loss-optimal states.
problem Optimizing deep neural net compression by quantizing weights.
method Model compression as constrained optimization framework, alternating learning and quantization.
result Guaranteed convergence to local optimum of loss for quantized nets, achieving high compression rates.
Paper proposes a faster method for sparse parameter recovery from noisy linear combinations with low-rank matrices.
problem Recovering sparse parameters from noisy linear combinations with partial matrix information.
method Unified four-step problem combining partial matrix completion and sparse vector recovery, ignoring zero elements in the sparse vector.
result The unified approach achieves best performance with less computational requirements.
Spectral clustering is one of the most widely used techniques for extracting the underlying global structure of a data set. Compressed sensing and matrix completion have emerged as prevailing methods for efficiently recovering sparse and partially observed signals respectively. We combine the distance preserving measur…
Reduces policy space complexity for reinforcement learning.
problem Efficiency in exploring vast policy spaces in reinforcement learning.
method Uses Rényi divergence and l 1 l_1 l 1 norm to determine sample size for accurate policy approximation. result Established error bounds for sample size requirements in model-based and model-free settings.
New neural network class reduces VC dimension, leading to better generalization.
problem VC theory struggles with explaining small generalization errors in overparametrized neural networks.
method Developed hyperplane arrangement neural networks (HANNs) and used sample compression analysis.
result HANNs can have significantly smaller VC dimension than the number of weights, yet remain highly expressive.
Optimal privacy and accuracy in distributed mean estimation with compression.
problem Achieving optimal accuracy under privacy and communication constraints.
method Compression to reduce communication while maintaining privacy and accuracy.
result Achieves optimal error with significantly reduced communication.
SANs use sparse activation functions to compress data representations.
problem Learning meaningful features without considering compression.
method Introduce φ metric, define activation functions, and present SANs.
result SANs achieve small description length and interpretable kernels.
Study compares hyperbolic and extremal lengths for shortest curves.
problem Comparing hyperbolic and extremal lengths for shortest curves.
method Lower bounds for widths of collars and upper bounds for renormalized volume of Schottky manifolds.
result Upper bounds of renormalized volume in terms of hyperbolic length of compressible curves.
A new method uses a frozen language model to improve sample efficiency in reinforcement learning.
problem Improving sample efficiency in reinforcement learning with partially observable environments.
method FROZEN Hopfield network and HELM (History Embedding Language Model) method.
result HELM achieves new state-of-the-art results on Minigrid and Procgen environments.
CSER improves SGD efficiency by resetting errors and partial synchronization.
problem Limited scalability of Distributed Stochastic Gradient Descent (SGD) due to communication bottlenecks.
method Introduces 'error reset' technique and partial synchronization for gradients and models.
result Proves convergence for smooth non-convex problems and accelerates distributed training significantly.
Improved survival analysis using square root Cox's models and neural networks.
problem Feature selection in survival analysis.
method Square root Cox's survival analysis by the fittest linear and neural networks model, directly tuning penalty parameter λ.
result Substantially improved over traditional methods, achieving phase transition in feature selection.
This the first of a set of three papers about the Compression Theorem: if M^m is embedded in Q^q X R with a normal vector field and if q-m > 0, then the given vector field can be straightened (ie, made parallel to the given R direction) by an isotopy of M and normal field in Q X R. The theorem can be deduced from Gromo…
New adapted renormalized volume for hyperbolic 3-manifolds with compressible boundary.
problem Analyzing convex co-compact hyperbolic 3-manifolds with compressible boundaries.
method Defining and analyzing a new version of the renormalized volume.
result The adapted renormalized volume is bounded and has properties analogous to the classical renormalized volume.
BNCR-GAN improves GANs to generate clean images from degraded inputs.
problem Generating clean images from blurred, noisy, and compressed degraded inputs.
method Multiple-generator model with image, blur-kernel, noise, and quality-factor generators, using masking architectures and adaptive consistency losses.
result BNCR-GAN effectively learns clean image generators from degraded images without degradation parameters.
Midicoth compresses online probability estimates by correcting prior smoothing biases.
problem Compression inefficiency due to prior smoothing in adaptive models.
method Micro-diffusion denoising applied in a bitwise tree hierarchy.
result Significant compression improvement with reliable calibration.
Quantum TNCS uses machine learning to efficiently transmit data.
problem Efficient quantum communication of large datasets.
method Combining compressed sensing, tensor networks, and machine learning.
result High efficiency and accuracy in transmitting information.
Paper extends neural network method to irregular solutions in PDEs.
problem Solving irregular and data-enriched PDEs.
method Deep neural networks for numerical PDE solutions, extending to irregular and data-enhanced cases.
result Demonstrates ease and integration of large datasets in PDE modeling.
New methods reduce communication in distributed training for variational inequalities.
problem Reducing communication in distributed training for high-dimensional models.
method Distributed methods with compressed communication for solving variational inequalities.
result Theoretical guarantees and practical algorithms for compressed communication.
Paper proposes binary-valued gates for better LSTM training.
problem LSTMs struggle with soft gates, leading to unclear information flow.
method Introduces binary-valued gates to control information flow more clearly.
result Binary-valued gates improve LSTM performance and generalization.
Extends characterization of P D 3 PD_3 P D 3 -pairs with aspherical boundaries to those with spherical boundaries.
problem Characterizing fundamental triples of P D 3 PD_3 P D 3 -pairs with boundary components of different types. method Extends Turaev and Bleile's work by relaxing the π 1 π_1 π 1 -injectivity hypothesis and considering pairs with spherical boundary components. result Characterization of fundamental triples for P D 3 PD_3 P D 3 -pairs with spherical boundary components and c . d . π 1 ( P ) ≤ 2 c.d.π_1(P)\leq2 c . d . π 1 ( P ) ≤ 2 . Paper establishes convergence rates for learning elliptic pseudo-differential operators.
problem Learning elliptic pseudo-differential operators in partial differential equations.
method Wavelet-Galerkin framework, structured infinite-dimensional regression problem, sparse estimator, matrix compression, nested-support strategy.
result Obtained convergence rates for the estimator and efficient Galerkin solver.