Random dictionaries help solve complex inverse problems without strict assumptions.
problem Solving ill-posed linear inverse problems with overcomplete dictionaries.
method Apply random dictionaries to regression problems and study their performance.
result Random dictionaries can solve ill-posed linear inverse problems without stringent compatibility conditions.
Optimal dictionaries minimize the average squared error in representing random vectors.
problem Finding optimal dictionaries for minimizing ℓ2-norm of coefficients in random vector representations. method Using rank-1 decompositions of symmetric positive semidefinite matrices, explicit descriptions and polynomial-time algorithms for ℓ2-optimal dictionaries are provided. result Explicit descriptions and polynomial-time algorithms for ℓ2-optimal dictionaries are provided. Researchers find optimal dictionaries for minimizing average squared coefficients in random vector representations.
problem Finding optimal dictionaries for minimizing the average squared coefficients in random vector representations.
method Using rank-1 decompositions and majorization theory, the study provides a complete characterization of optimal dictionaries.
result Complete characterization of ℓ2-optimal dictionaries with polynomial time algorithms. Performing signal processing tasks on compressive measurements of data has received great attention in recent years. In this paper, we extend previous work on compressive dictionary learning by showing that more general random projections may be used, including sparse ones. More precisely, we examine compressive K-mean…
We study the theoretical properties of learning a dictionary from N signals xi∈RK for i=1,...,N via l1-minimization. We assume that xi's are i.i.d. random linear combinations of the K columns from a complete (i.e., square and invertible) reference dictionary $\mathbf D_0 \in…
New algorithm recovers dictionaries with arbitrary supports in polynomial time.
problem Learning dictionaries with arbitrary supports in polynomial time.
method Semirandom model with a mix of arbitrary and random supports; polynomial time algorithm.
result Polynomial time recovery of incoherent over-complete dictionaries with arbitrary supports.
Paper improves dictionary learning by addressing local and global coherence issues.
problem Improving dictionary learning by addressing local and global coherence issues.
method The paper uses the ITKrM algorithm to prove contraction under relaxed conditions and proposes replacing bad dictionaries with carefully designed candidates.
result The adaptive version of ITKrM can recover a generating dictionary from randomly initialized dictionaries of various sizes and learn meaningful dictionaries on image data.
When digitizing a print bilingual dictionary, whether via optical character recognition or manual entry, it is inevitable that errors are introduced into the electronic version that is created. We investigate automating the process of detecting errors in an XML representation of a digitized print dictionary using a hyb…
Subgradient descent learns orthogonal dictionaries efficiently.
problem Sparse coding and dictionary learning for data representation.
method Subgradient descent algorithm with random initialization.
result Subgradient descent can recover orthogonal dictionaries under mild assumptions.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Neurogenesis-inspired online learning adapts model architecture in changing environments.
problem Continuous adaptation of model architecture in non-stationary environments.
method Online dictionary-learning framework with adaptive addition and deletion of units, inspired by neurogenesis.
result Significant improvement in performance on nonstationary data compared to fixed-size online sparse coding.
Data sets are often modeled as point clouds in RD, for D large. It is often assumed that the data has some interesting low-dimensional structure, for example that of a d-dimensional manifold M, with d much smaller than D. When M is simply a linear subspace, one may exploit this assumption for encoding ef…
Study shows unique sharp local minimum in ℓ1-minimization for dictionary learning.
problem Global recovery of a dictionary from random linear combinations of atoms.
method Norm condition, explicit bound, perturbation-based test, Block Coordinate Descent algorithm.
result Reference dictionary is the unique sharp local minimum of the ℓ1 objective function. A new method for fiber sensing using sparse estimation and dictionary learning.
problem Compressed fiber sensing with severe dictionary coherence issues.
method Probabilistic hierarchical sparse model, selective shrinkage with Weibull prior, collective shrinkage based on local similarity, kernel function in joint prior density, hybrid inference technique (Hamilton Monte Carlo and Gibbs sampling), and two strategies for dictionary parameter estimation.
result Improved performance compared to existing methods in simulations and experiments.
Novel approach for estimating joint probability densities using tensor decompositions and dictionaries.
problem Estimating joint probability densities of mixed discrete and continuous variables.
method Low-rank tensor decomposition combined with dictionary learning.
result Better classification and lower error rates compared to existing methods.
This letter proposes a dictionary learning algorithm for blind one bit compressed sensing. In the blind one bit compressed sensing framework, the original signal to be reconstructed from one bit linear random measurements is sparse in an unknown domain. In this context, the multiplication of measurement matrix $\Ab$ an…
New method learns complete orthogonal dictionary from samples with theoretical guarantees and efficiency.
problem Learning a complete orthogonal dictionary from sparsely generated signals.
method Maximizes the \(\ell^4\)-norm over the orthogonal group, using a novel algorithm based on matching, stretching, and projection (MSP).
result The MSP algorithm provably converges locally at a superlinear (cubic) rate and is significantly more efficient than existing methods.
The paper presents a method for efficient high-dimensional inference using observable dictionary learning.
problem Efficiently inferring a high-dimensional distributed quantity from limited observations.
method Bayesian approach with hierarchical prior distribution, minimizing misfit with observations, and deriving an observable dictionary.
result The method provides superior performance in estimating velocity fields from limited sensors.
In sparse recovery we are given a matrix A (the dictionary) and a vector of the form AX where X is sparse, and the goal is to recover X. This is a central notion in signal processing, statistics and machine learning. But in applications such as sparse coding, edge detection, compression and super resolution, t…
Bayesian optimization for high-dimensional combinatorial spaces using embeddings.
problem Optimizing expensive functions over large, complex input spaces.
method Dictionary-based ordinal embeddings for high-dimensional combinatorial structures, using Gaussian process models.
result The proposed method outperforms state-of-the-art BO methods on diverse real-world benchmarks.
DeepCAM learns convolutional dictionaries for image processing.
problem Processing high-dimensional signals like images efficiently.
method Introduces a Deep Convolutional Analysis Dictionary Model (DeepCAM) using convolutional dictionaries.
result DeepCAM achieves performance comparable to other methods on single image super-resolution.
Bayesian method improves dictionary learning for complex problems.
problem Efficiently identifying relevant dictionary entries for complex inverse problems.
method Bayesian group sparsity coding and deflation steps to compress and identify relevant subdictionaries.
result Significant computational complexity reduction and improved glitch detection in LIGO experiment.
In dictionary learning, also known as sparse coding, the algorithm is given samples of the form y=Ax where x∈Rm is an unknown random sparse vector and A is an unknown dictionary matrix in Rn×m (usually m>n, which is the overcomplete case). The goal is to learn A and x. T…
New algorithms find global solutions for complex data factorization.
problem Finding global minimum solutions for complex data factorization.
method Practical alternating minimization algorithms for induced Dictionary Learning Models (DLMs).
result Alternating minimization converges to global minima for a large subclass of induced DLMs.
We consider the problem of sparse coding, where each sample consists of a sparse linear combination of a set of dictionary atoms, and the task is to learn both the dictionary elements and the mixing coefficients. Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficient…
This work learns sparse tensor representations using mixtures of separable dictionaries.
problem Learning sparse representations of tensor data with structured models.
method Proposes and explores learning a mixture of separable dictionaries with sufficient conditions for local identifiability.
result Developed computational algorithms for batch and online learning.
Estimates signals from a continuous dictionary with sparse mixtures using optimization.
problem Estimating signals from a continuous dictionary with unknown mixtures and noise.
method Formulates a regularized optimization problem with data fidelity and (ℓ1,Lp)-penalty. result High probability bounds on prediction error for the Group-Nonlinear-Lasso solution.
NOODL solves dictionary and coefficient recovery for online learning.
problem Non-convex optimization challenges in dictionary learning.
method NOODL: Neurally plausible alternating Optimization-based Online Dictionary Learning.
result Exact recovery of dictionary and coefficients at geometric rate.
DeepAM optimizes deep neural networks for image super-resolution.
problem Image super-resolution using deep neural networks.
method L-layer analysis dictionary model with IPAD and CAD.
result DeepAM outperforms deep neural networks with back-propagation.
We present a two-stage approach for learning dictionaries for object classification tasks based on the principle of information maximization. The proposed method seeks a dictionary that is compact, discriminative, and generative. In the first stage, dictionary atoms are selected from an initial dictionary by maximizing…
Many techniques in computer vision, machine learning, and statistics rely on the fact that a signal of interest admits a sparse representation over some dictionary. Dictionaries are either available analytically, or can be learned from a suitable training set. While analytic dictionaries permit to capture the global st…
RandNet learns from compressed image data, improving efficiency and accuracy.
problem Efficiency and accuracy in training neural networks with large datasets.
method RandNet uses compressed random measurements of images to train neural networks efficiently.
result RandNet achieves comparable accuracy to full data training with minimal loss.
CRsAE auto-encoder recovers convolutional dictionary from noisy signals.
problem Recovering a convolutional dictionary from noisy signals.
method Constrained recurrent sparse auto-encoder (CRsAE) architecture.
result CRsAE successfully recovers the underlying dictionary in the presence of noise.
A new method learns a sparse dictionary and optimizes its size for better image processing.
problem Choosing the right size of the dictionary for optimal performance.
method Employed a novel regularization method (GSCAD) combined with ADMM for simultaneous sparse dictionary learning and size selection.
result The method improves image denoising performance compared to existing approaches.
Efficient algorithm selects atoms from dictionaries with complex sparsity constraints.
problem Dictionary selection with complex sparsity constraints.
method Novel efficient greedy algorithm for dictionary selection.
result Outperforms known methods in faster running time and competitive performance.
Paper proposes robust dictionary learning using concave losses.
problem Sensitivity to outliers in traditional dictionary learning methods.
method Generic framework based on concave losses, with results on composition of concave functions.
result Method better detects outliers and generates better dictionaries.
Paper provides conditions for local recovery of tensor data's Kronecker-structured dictionaries.
problem Local recovery of Kronecker-structured dictionaries for tensor data.
method Derives sufficient conditions for local recovery of coordinate dictionaries.
result Sufficient conditions guarantee recovery of individual coordinate dictionaries up to specified error.
Sparse coding in learned dictionaries has been established as a successful approach for signal denoising, source separation and solving inverse problems in general. A dictionary learning method adapts an initial dictionary to a particular signal class by iteratively computing an approximate factorization of a training …
A new tensor decomposition method using a dictionary for better interpretability.
problem Ensuring interpretability in tensor decomposition models.
method Dictionary-based tensor canonical polyadic decomposition with sparse coding.
result Improves parameter identifiability and estimation accuracy in tensor decomposition.
A new deep learning tool learns multi-level dictionaries greedily.
problem Improving deep learning performance on benchmark datasets.
method Greedy learning of multi-level dictionaries, solving shallow dictionary learning problems sequentially.
result Our method outperforms other deep learning tools and state-of-the-art supervised dictionary learning methods.
A dictionary connects symplectic to contact geometry, with applications to complex and G-structures.
problem Formalizing the relationship between symplectic and contact geometry.
method Developing a Symplectic-to-Contact Dictionary.
result The dictionary can be applied to complex and G-structures, revealing new geometries.
A parallel algorithm learns efficient Kronecker product dictionaries.
problem Sparse representation of 2D signals like images and hyperspectral data.
method Highly parallelizable algorithm for learning separable dictionaries.
result Competitive sparse representations at lower computational cost.
In sparse signal representation, the choice of a dictionary often involves a tradeoff between two desirable properties -- the ability to adapt to specific signal data and a fast implementation of the dictionary. To sparsely represent signals residing on weighted graphs, an additional design challenge is to incorporate …
Sparse representations using learned dictionaries are being increasingly used with success in several data processing and machine learning applications. The availability of abundant training data necessitates the development of efficient, robust and provably good dictionary learning algorithms. Algorithmic stability an…
The paper provides guarantees for an alternating minimization algorithm in dictionary learning.
problem Dictionary learning problem of factorizing samples into a basis and sparse vectors.
method Alternating minimization procedure switching between ℓ1 minimization and gradient descent. result Local convergence guarantees for the alternating minimization algorithm under a new matrix infinity norm condition.
The paper tackles dictionary learning with almost sure error constraints.
problem Achieving desirable features in data representation with almost sure error constraints.
method Imposes almost sure recovery constraints and reformulates the problem as a convex-concave min-max problem, solved using gradient descent-ascent.
result Demonstrates the effectiveness of the proposed method in achieving almost sure error constraints in dictionary learning.
The paper shows how to find a sparse representation of signals without strict coherence assumptions.
problem Finding a sparse representation of signals without strict coherence assumptions.
method An algorithm for the threshold correlation problem, which applies to signals with outliers.
result Approximate guarantees for dictionary learning without strict coherence assumptions.
We study the Dictionary Learning (aka Sparse Coding) problem of obtaining a sparse representation of data points, by learning \emph{dictionary vectors} upon which the data points can be written as sparse linear combinations. We view this problem from a geometry perspective as the spanning set of a subspace arrangement,…