Paper introduces MPS for efficient tensor compression and classification.
problem Efficiently compressing and classifying higher-order tensors.
method Matrix Product State (MPS) using successive SVD.
result MPS achieves better classification performance with lower computation cost.
Study on core consistency preservation in compressed tensors.
problem Ensuring low-rank structure is maintained during tensor compression.
method Theoretical analysis and experimental validation of compression schemes.
result Identified sufficient conditions for preserving core consistency.
Tensor regression networks improve neural network compression and regularization.
problem Improving neural network compression and regularization with low-rank tensor approximations.
method Investigating various low-rank tensor approximations in tensor regression networks.
result Tensor regression networks with Global Average Pooling layer outperformed in deep CNNs, while shallow CNNs with tensor regression and dropout achieved lower test error.
New result on tensor recovery without strong assumptions.
problem Recoverability of randomly compressed tensors with low CP rank.
method Deriving restricted isometry property (R.I.P.) via set covering techniques.
result The tensor is recoverable if the number of measurements is proportional to the model parameters.
Efficiently compress SPNs using tensor networks.
problem Efficiently compressing Sum-Product Networks (SPNs).
method Mapping SPNs onto tensor networks and employing novel optimization techniques.
result Remarkable parameter compression with negligible loss in accuracy.
This paper proposes a method to automatically compress neural networks using Bayesian tensor decomposition.
problem Challenges in directly applying tensor compression in neural network training.
method Bayesian tensorized neural network with automatic rank selection.
result Produces significantly more compact neural networks (7.4x to 137x) directly from training.
New method compresses LSTM networks using MPS tensor trains.
problem Challenges in maintaining performance of compressed RNNs.
method Use of MPS tensor trains for LSTM network compression.
result MPS tensor trains outperform MPOs in storage and inference time.
Advances understanding of neural network generalization via tensor analysis.
problem Understanding the generalizability of deep neural networks.
method Tensor analysis to measure compressibility and generalizability.
result Proposed generalization bound outperforms previous methods, especially for tensor-based networks.
T-Basis represents neural network tensors with fewer parameters.
problem Efficiently representing neural network tensors with fewer parameters.
method T-Basis uses Tensor Rings to represent tensors in a neural network, parameterizing them with a small number of coefficients.
result T-Basis achieves high compression rates with minimal performance loss.
Optimizes tensor rank selection for neural network compression.
problem Finding optimal tensor rank for regression models.
method Analyzes population expressions for training-testing discrepancy under Gaussian design.
result Optimal rank minimizes prediction error and aligns with cross-validation.
HOTCAKE compresses CNNs by decomposing kernels into smaller parts.
problem Compressing deep CNNs without significant accuracy loss.
method Input channel decomposition, guided Tucker rank selection, higher order Tucker decomposition, fine-tuning.
result HOTCAKE produces highly compressed CNN models with good accuracy.
TR-Nets compress deep networks by 11x for LeNet-5 and 243x for Wide ResNet.
problem Large neural networks require excessive memory and computation.
method Tensor Ring factorization to compress fully connected and convolutional layers.
result TR-Nets can compress LeNet-5 by 11x and Wide ResNet by 243x with minimal accuracy loss.
Efficiently compress neural networks with MUSCO method.
problem Compression of deep neural networks.
method Iterative approach alternating low-rank factorization with rank selection and fine-tuning.
result Improves compression rate while maintaining accuracy.
Efficiently price high-dimensional Bermudan options using tensor compression.
problem High-dimensional option pricing with computational complexity.
method Hierarchical tensor compression for Monte Carlo and dual martingale methods.
result Tensor compression alleviates the curse of dimensionality for Bermudan option pricing.
OCTen compresses and speeds up online tensor decomposition.
problem Handling dynamic, growing datasets efficiently.
method Compression-based online parallel CP decomposition.
result OCTen achieves similar or better accuracy with up to 40-200% less memory.
This work tackles fast and accurate low-rank factorization of compressed data.
problem Accurately and efficiently computing low-rank matrix or tensor factorizations from compressed data.
method Factorization in the compressed domain followed by reconstruction of original factors.
result Provable recovery of original factors under certain conditions.
Paper compresses RNNs using HT decomposition for better performance.
problem Large model sizes of RNNs in sequence analysis.
method Hierarchical Tucker (HT) tensor decomposition for model compression.
result HT-LSTM achieves better compression and accuracy than state-of-the-art methods.
Adaptive algorithm learns tensor network structures from data.
problem Identifying optimal tensor network structure from data.
method Greedy approach starting from rank one tensor, small rank increments.
result Adaptive algorithm identifies efficient tensor network structures.
The paper explores tensor decompositions in deep learning models.
problem Compressing parameter space and creating richer representations.
method Tensor decompositions applied to deep learning models.
result Tensor methods can yield richer adaptive representations of complex data.
Tensorial Neural Networks improve neural network compression and performance.
problem Efficiently compressing neural networks while maintaining or improving performance.
method Introducing tensor operations on high-order operands to solve hierarchical nonlinear tensor decomposition using stochastic gradient descent.
result TNNs achieve up to 5% test accuracy improvement on CIFAR10 compared to state-of-the-art compression methods.
New taxonomy and evaluation of neural network compression methods.
problem Efficiency of deep neural networks in real-world applications.
method Categorization and evaluation of tensor factorization and probabilistic compression methods.
result SVD and probabilistic compression methods are complementary and give the best results.
Paper compresses ViTs models by 60% with minimal accuracy loss.
problem Memory constraints in ViTs models.
method Activation-aware low-rank tensor approximations.
result 60% reduction in model size with <1% accuracy loss.
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the K/R ratio, for compressed CNNs. New method compresses deep learning layers using tensor decomposition.
problem Reduction of computation cost and interpretability for tensor data.
method CP-decomposition to compress convolutional layers in deep learning.
result Reduces model complexity and maintains prediction performance.
Randomly shuffled kernels can be compressed efficiently.
problem Reducing storage cost of CNN parameters on resource-limited platforms.
method Randomly-shuffled tensor decomposition (RsTD) to embed kernels into random low-rank subspaces.
result CNNs can be significantly compressed even with randomly shuffled kernels, achieving more stable accuracy.
New method uses low-rank tensor factor analysis for better image restoration.
problem Restoring images from limited data.
method Low-rank tensor factor analysis combined with ADMM.
result The method outperforms traditional approaches, especially at low sampling rates.
End-to-end meta-learned system for image compression.
problem Reducing the gap between training and inference conditions in image compression.
method Model-Agnostic Meta-learning approach for latent tensor overfitting and updating encoder and decoder networks.
result Meta-learned system achieves better compression performance compared to traditional methods.
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.
problem Determining optimal decomposition ranks in tensor decompositions.
method MARS uses binary masks to learn optimal tensor structure during training via relaxed MAP estimation.
result MARS achieves better results than previous methods in various tasks.
Tensorized random projections reduce high-dimensional tensor size efficiently.
problem Efficiently reducing the dimension of very high-dimensional tensors.
method Proposes two tensorized random projection maps using TT and CP decompositions.
result TT format offers superior performance in terms of required random projection size.
We introduce a new parameterization method for deep learning layers using spectral tensor train decomposition.
problem Efficiency and stability in deep learning models with weight matrix compression.
method Spectral Tensor Train Parameterization (STTP) of weight matrices.
result Improved compression and training stability in neural networks.
Spatio-temporal data compression method reduces memory usage.
problem Efficiently storing and analyzing large spatio-temporal datasets.
method Adaptive sampling of tensor slices to compress and preserve structure.
result SkeTenSmooth outperforms other sampling methods in retaining patterns.
Quantum TNCS uses machine learning to efficiently transmit data.
problem Efficient quantum communication of large datasets.
method Combining compressed sensing, tensor networks, and machine learning.
result High efficiency and accuracy in transmitting information.
Tensor trains speed up option pricing for multi-asset options.
problem Speeding up option pricing for multi-asset options.
method Tensor train learning algorithms to compress functions with parameter dependence.
result The proposed method outperforms Monte Carlo-based pricing in computational complexity.
New method compresses non-Gaussian distributions exponentially.
problem Efficiently representing and computing non-Gaussian probability distributions.
method Tensor-Network Fourier Methods using QTT representation.
result Exponential compression of non-Gaussian distributions.
Tensor trains simplify solving complex PDEs efficiently.
problem Solving high-dimensional parabolic PDEs using traditional methods is computationally infeasible.
method Reformulate PDEs as backward stochastic differential equations and use tensor train format for compression and efficient computation.
result Tensor train methods achieve a good balance between accuracy and computational efficiency.
A new sketching method reduces tensor memory usage and enables efficient tensor operations.
problem Efficiently compressing and retaining tensor structure in large datasets.
method Higher-order Count Sketch (HCS) using multiple hash functions and tensor products.
result HCS achieves significant memory savings and efficient tensor operations.
BT-Nets simplify DNNs by reducing parameter count.
problem Redundancy in DNN weights limits deployment to resource-constrained environments.
method Replaces FC-layers with BT-layers using block term decomposition.
result BT-layers achieve high compression ratios while maintaining performance.
GrateTile optimizes CNN feature map storage for efficient data access.
problem Efficient storage and access of sparse CNN feature maps.
method Divides feature maps into uneven-sized subtensors, compresses and stores them in a compressed yet accessible format.
result Average 55% DRAM bandwidth reduction with minimal indexing overhead.
This paper finds a new way to compress CNN weights, improving on pruning and quantization.
problem Improving performance and storage efficiency of CNNs.
method Identifying and exploiting repeated patterns in CNN weight tensors, using Huffman coding and block sparse matrix formats.
result Achieved compaction ratios of 1.4x to 3.1x in addition to pruning and quantization.
TEAFormers preserve multi-dimensional time series structures for better forecasting.
problem Traditional Transformers flatten multi-dimensional time series data, losing critical multi-dimensional relationships.
method Tensor-Augmented Transformer (TEAFormer) with Tensor-Augmentation (TEA) module.
result Significant performance enhancements in time series forecasting across benchmarks.
Tensorized GAN reduces model size and complexity for mobile devices.
problem High computational complexity and large model size in GANs for mobile devices.
method Tensorizing each layer and using multilinear operations to reduce model parameters.
result Achieved high compression rate of up to 35 times for MNIST dataset.
This paper offers an overview of neural network compression techniques.
problem Overparameterized neural networks are large and resource-intensive.
method Pruning, quantization, tensor decomposition, knowledge distillation.
result A comprehensive review of compression techniques for deep neural networks.
TensorShield defends images from adversarial attacks using tensor decomposition.
problem Adversarial attacks on images can fool deep neural networks.
method Tensor decomposition to find low-rank approximations of images, reducing high-frequency perturbations.
result TensorShield outperforms existing methods like SLQ by 14% against FGSM attacks.
Explores tensor products in hyperdimensional computing.
problem Understanding tensor products in hyperdimensional computing.
method Generalized results from graph embeddings to vector symbolic architectures and hyperdimensional computing.
result Tensor product is the most general and expressive representation with errorless unbinding and detection.
Paper bounds tensor decomposition's RLCT, aiding Bayesian inference.
problem Unclear mathematical property of tensor decomposition.
method Algebraic geometrical method for upper bound derivation.
result Upper bound of real log canonical threshold (RLCT) derived.
Paper explores compression techniques for LSTM networks in language modeling.
problem High space complexity and inference time of LSTM networks in language modeling.
method Pruning, quantization, low-rank factorization, tensor train decomposition.
result Compared techniques for model size and fast inference suitability.
Unified framework for generalized sparsity and RIP analysis.
problem Analyzing inverse problems with sparsity models.
method Proposed generalized notions of sparsity and a unified RIP framework.
result Extends RIP analysis to broader contexts including tensor products.