Deep learning improves sparse representation for better classification.
problem Improving classification accuracy using sparse representation.
method A transductive deep learning network combining convolutional autoencoder and fully-connected layers.
result The proposed network achieves better classification results than state-of-the-art SRC methods.
D-CSC framework reveals how ReLU activation functions recover activation paths in neural networks.
problem Understanding how ReLU activation functions recover activation paths in neural networks.
method Deep Convolutional Sparse Coding (D-CSC) framework, omitting dictionary learning, to analyze activation paths.
result Uniform guarantees for recovery of true activation paths with high probability for greater activation densities.
Proposes a hierarchical deep generative model for natural images.
problem Analyzing piecewise smooth signals like natural images.
method Hierarchical deep generative model with alternating minimization algorithm.
result Demonstrates the model's representation capabilities and classification performance.
New models explain residual and dilated dense neural networks using sparse coding.
problem Lack of theoretical understanding of residual and dilated dense neural networks.
method Proposed Res-CSC and MSD-CSC models, derived mathematical relationships, implemented ISTA.
result Mathematical understanding of residual and dilated dense neural networks.
New algorithms compare and improve convolutional dictionary learning methods.
problem Challenges in learning convolutional dictionaries.
method Comprehensive comparison and development of new algorithms.
result Identifies most effective methods for convolutional dictionary learning.
It has recently been observed that certain extremely simple feature encoding techniques are able to achieve state of the art performance on several standard image classification benchmarks including deep belief networks, convolutional nets, factored RBMs, mcRBMs, convolutional RBMs, sparse autoencoders and several othe…
A new stochastic solver improves Convolutional Sparse Coding efficiency.
problem Efficiency and sparsity in Convolutional Sparse Coding.
method Randomized subsampling strategy in spatial domain for online learning.
result Improved execution time with no loss in learning quality.
A new greedy method tackles ℓ0,∞ sparse coding for better image processing.
problem Imbalanced sparsity in ℓ0 and ℓ1 norms for image processing. method Greedy matching pursuit for ℓ0,∞ norm optimization. result Efficient method for ℓ0,∞ sparse coding and dictionary learning. A deep network classifies images by scattering and dictionary learning.
problem Classifying images with high accuracy using deep learning.
method Sparse scattering transform followed by ℓ1 dictionary learning in a deep convolutional network. result Higher classification accuracy than AlexNet on ImageNet dataset.
SparseRT accelerates sparse computations on GPUs for deep learning inference.
problem Efficiently handling unstructured sparsity patterns on GPUs for deep learning.
method SparseRT, a code generator that leverages unstructured sparsity for accelerating sparse linear algebra operations.
result Geometric mean speedups of 3.4x at 90% sparsity and 5.4x at 95% sparsity for 1x1 convolutions and fully connected layers.
Deep sparse coding models resist adversarial examples.
problem Adversarial examples can fool deep learning models.
method Used deep sparse coding models to resist adversarial examples.
result Deep sparse coding models are robust to adversarial examples.
New multi-layer algorithm improves CNN performance.
problem Efficiently modeling and processing information with parsimonious representations.
method Generalized Basis Pursuit to multi-layer setting, proposing ML-ISTA and ML-FISTA algorithms.
result Nested first order algorithms converge to solve the multi-layer problem.
New model encodes multivariate signals more efficiently with sparsity and low-rank constraints.
problem Efficiently encoding multivariate signals with sparsity and low-rank constraints.
method Multivariate convolutional sparse coding with tensor algebra, CP decomposition, and alternating optimization.
result Proves model closely related to Kruskal tensor regression problem with theoretical guarantees.
Unsupervised deep learning is one of the most powerful representation learning techniques. Restricted Boltzman machine, sparse coding, regularized auto-encoders, and convolutional neural networks are pioneering building blocks of deep learning. In this paper, we propose a new building block -- distributed random models…
Proposes a new CSC model for handling unknown noise.
problem Existing CSC methods can only model Gaussian noise, which is restrictive.
method Uses Gaussian mixture model for unknown noise and EM algorithm for optimization.
result Effective modeling of complicated unknown noise with high-quality filters and representation.
BioHash improves similarity search performance using sparse high-dimensional hash codes.
problem Improving similarity search performance in high-dimensional data.
method BioHash produces sparse high-dimensional hash codes through a data-driven approach based on synaptic plasticity.
result BioHash outperforms previous hashing methods in various similarity search tasks.
New pursuit algorithm for ML-CSC model with improved stability and dictionary learning.
problem Lack of exact pursuit algorithms and conditions for non-empty model in ML-CSC.
method Projection approach for pursuit algorithm, stability bounds, practical alternatives, online dictionary learning.
result Sound pursuit algorithm and practical dictionary learning for ML-CSC model.
In this paper, we introduce DICOD, a convolutional sparse coding algorithm which builds shift invariant representations for long signals. This algorithm is designed to run in a distributed setting, with local message passing, making it communication efficient. It is based on coordinate descent and uses locally greedy u…
New deep network derived from rate reduction principles, explaining features and efficiency.
problem Understanding and optimizing deep learning architectures.
method Gradient ascent scheme for rate reduction leading to multi-layer deep network.
result Explicitly constructed multi-layer network with precise optimization and interpretation.
This work combines deep learning and sparse coding for CT image reconstruction.
problem Improving image quality in low-dose CT scans.
method Sparse signal representation using learned dictionaries, inspired by variational autoencoders and deep learning techniques.
result Regularization with learned dictionaries achieves competitive performance in CT reconstruction.
New method learns sparse distributions by thresholding samples, improving performance and efficiency.
problem Sparse coding optimization in high-dimensional problems is computationally expensive and inefficient.
method Proposes a new variational sparse coding approach that learns sparse distributions by thresholding samples.
result Shows superior performance, statistical efficiency, and gradient estimation compared to other sparse distributions.
Proposes a method to boost deep reinforcement learning with sparse rewards.
problem Challenges in learning complex behaviors with long horizons and sparse rewards.
method Predictive coding for reward shaping.
result Achieves better learning by providing reward signals that understand environment dynamics and emphasize useful features.
While sparse coding-based clustering methods have shown to be successful, their bottlenecks in both efficiency and scalability limit the practical usage. In recent years, deep learning has been proved to be a highly effective, efficient and scalable feature learning tool. In this paper, we propose to emulate the sparse…
Proposes a non-parametric method for deep discrete latent variable models.
problem Learning sparse discrete latent representations in deep models.
method Iterative algorithm with Beta-Bernoulli process prior and local data scaling.
result Improves sparsity and scalability of deep discrete latent variable models.
Transform learning improves MRI image reconstruction from sparse data.
problem Efficiently reconstruct MRI images from limited data.
method TL-based methods using learned models and transform domains.
result TL-based methods outperform classical CS methods in MRI reconstruction.
New CSC model extracts EEG signals with low noise sensitivity.
problem Analyzing noisy EEG signals during anesthesia.
method Kruskal CSC model using Kruskal decomposition for low-rank tensor activations.
result TC-FISTA efficiently extracts robust, sparse, and interpretable EEG encodings.
Convolutional neural networks (CNN) have led to many state-of-the-art results spanning through various fields. However, a clear and profound theoretical understanding of the forward pass, the core algorithm of CNN, is still lacking. In parallel, within the wide field of sparse approximation, Convolutional Sparse Coding…
Paper introduces efficient algorithm for double-sparse coding with theoretical guarantees.
problem High storage and processing costs in sparse coding for high-dimensional data.
method Simple algorithm for double-sparse coding, leveraging neural architectures for efficiency.
result Theoretical analysis shows asymptotic sample complexity and running time benefits over existing methods.
New defense method reduces adversarial attacks on neural networks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Sparse coding frontend trained on clean images.
result Significantly attenuates adversarial attacks before reaching classifier.
Deep learning using multi-layer neural networks (NNs) architecture manifests superb power in modern machine learning systems. The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable pe…
This paper compresses deep neural networks for efficient learning on embedded systems.
problem Large memory and computation requirements of deep neural networks.
method Sparse coding with proximal point algorithms and debiasing for model compression.
result Minimal learning models suitable for small embedded devices are produced.
CRsAE auto-encoder recovers convolutional dictionary from noisy signals.
problem Recovering a convolutional dictionary from noisy signals.
method Constrained recurrent sparse auto-encoder (CRsAE) architecture.
result CRsAE successfully recovers the underlying dictionary in the presence of noise.
Reduced-gate LSTM improves spatiotemporal prediction with less parameters.
problem Next-frame video prediction in deep learning.
method Predictive coding framework with reduced-gate convolutional LSTM.
result Reduced-gate model achieves equal or better accuracy with fewer parameters.
LSALSA accelerates sparse coding and MCA by learning optimal sparse codes.
problem Efficiently solving sparse coding and MCA problems.
method Deep learning architecture based on SALSA and ADMM.
result LSALSA achieves significant improvements in running time and code quality.
New method learns complex brain signal patterns from EEG/MEG data.
problem Complex waveforms in brain signals not captured by linear filters.
method Multivariate convolutional sparse coding (CSC) algorithm.
result Reveals non-sinusoidal mu-shaped patterns in brain signals.
New deep learning methods improve CT image quality from few projections.
problem Sparse-view CT images suffer from streaking artifacts due to limited projections.
method Inspired by deep convolutional framelets, propose new U-Net variants that satisfy the frame condition.
result New U-Net variants provide better reconstruction performance for sparse-view CT.
Deep learning improves decoding of constrained sequence codes, reducing errors and increasing throughput.
problem Errors during transmission of constrained sequence codes.
method Deep learning, specifically MLP and CNN networks.
result Achieved low bit error rates close to MAP decoding and improved system throughput.
Sparse linear models improve neural network debuggability.
problem Improving neural network interpretability and debugging.
method Using sparse linear models over learned deep feature representations.
result The approach leads to more debuggable and accurate neural networks.
This paper accelerates sparse CNN layers on GPUs by using unstructured sparsity.
problem Efficiency of sparse CNN layers on GPUs.
method Direct sparse operation and reduced precision.
result Achieving up to 90% sparsity in deep CNN models improves efficiency.
Deep convolutional networks provide state of the art classifications and regressions results over many high-dimensional problems. We review their architecture, which scatters data with a cascade of linear filter weights and non-linearities. A mathematical framework is introduced to analyze their properties. Computation…
Deep GCNII tackles over-smoothing problem in graph convolutional networks.
problem Over-smoothing problem in shallow graph convolutional networks.
method Proposes GCNII with initial residual and identity mapping techniques.
result Deep GCNII outperforms state-of-the-art methods on various tasks.
Direct feedback alignment reduces data movement in neural networks.
problem Efficiency and energy-efficiency in training large neural networks.
method Sparse feedback matrix for local learning, reducing data movement and compute.
result Orders of magnitude improvement in data movement and 2x improvement in multiply-and-accumulate operations.
CRsAE autoencoder learns dictionary for image denoising and brain spike detection.
problem Sparse dictionary learning for image and neural signal processing.
method CRsAE architecture, alternating E-step and M-step, FISTA for sparse coding, EM-inspired regularization.
result CRsAE outperforms conventional methods in image denoising and spike detection.
New method trains sparse deep networks efficiently.
problem Training deep neural networks with limited parameters.
method Dynamic sparse reparameterization.
result Our method outperforms previous techniques in accuracy for a fixed parameter budget.
Sparse molecular representations improve interpretability in graph neural networks.
problem Difficulty in understanding which molecular graph aspects drive deep learning predictions.
method Constrain weights in a graph convolutional neural network using the Gini index to maximize representation inequality.
result The Gini-constrained approach does not degrade evaluation metrics and allows for interpretable representation combination.
Paper solves complex signal processing problem efficiently.
problem Learning an unknown filter from multiple sparse convolutions.
method Nonconvex optimization over the sphere manifold using manifold gradient descent.
result Manifold gradient descent provably recovers the filter under random data model.
Deep neural network reconstructs traffic speeds from sparse vehicle data.
problem Reconstructing traffic speeds from limited probe vehicle data.
method Convolutional neural network architecture for spatio-temporal learning.
result The method can reconstruct traffic speeds with low probe vehicle penetration.
PUDLE method analyzes and improves unrolled sparse coding networks for dictionary learning.
problem Dictionary learning problem, representing data as a combination of few atoms.
method PUDLE method addresses challenges in unrolled sparse coding networks through theoretical analysis and practical strategies.
result PUDLE method provides conditions for recovering and preserving the support of the latent code, and resolves bias and instability issues.