Convolutional factor analysis improves compressive sensing with fewer measurements.
problem Efficiently reconstructing images from limited compressed measurements.
method Learn convolutional dictionaries from compressed measurements using ADMM.
result Achieves comparable classification accuracy with 30% fewer measurements.
A new tensor-factor analysis model improves image denoising and classification.
problem Improving image denoising and classification performance.
method Introduces a deep convolutional tensor-factor analysis model for multi-way data.
result Improves PSNR by over 1dB in multi-way denoising and image classification.
The paper verifies deep neural networks' ability to approximate functions on spheres.
problem Theoretical verification of deep neural networks' performance on spherical functions.
method Spherical analysis using reproducing kernels and convolutional factorizations.
result Rates of uniform approximation for functions in Sobolev spaces and additive ridge forms.
The paper explains implicit regularization in hierarchical tensor factorization and deep CNNs.
problem Understanding implicit regularization in complex neural network architectures.
method Theoretical analysis using dynamical systems to overcome challenges in hierarchy.
result Established implicit regularization towards low hierarchical tensor rank, equivalent to locality in CNNs.
Two methods preserve spatial locality in aggregating multi-subject fMRI data.
problem Aggregating multi-subject fMRI data while preserving spatial brain locality.
method Combining shared response model with searchlight analysis and designing a multi-view convolutional autoencoder.
result Both methods preserve spatial locality and have competitive or better performance compared to standard searchlight analysis and shared response model.
Unified tensor factorization for efficient 3D convolutions in spatio-temporal emotion analysis.
problem Training deep 3D convolutions is computationally expensive and requires large datasets.
method Tensor factorization framework for separable higher-order convolutions.
result Improved spatio-temporal emotion estimation on large datasets.
New algorithm learns convolutional neural networks with overlapping patches.
problem Learning convolutional neural networks with overlapping patches.
method Algorithm draws from isotonic regression and landscape analysis.
result Algorithm works for general class of patches, including common computer vision structures.
The paper introduces a pooling mechanism for graph CNNs using NMF.
problem Pooling in graph structured data for efficient computation.
method Non-negative matrix factorization for node pooling.
result The pooling mechanism improves graph classification performance.
New algorithms improve NMF for extracting patterns from time series data.
problem Extracting short-lived temporal motifs from high-dimensional time series data.
method Extended HALS and ANLS algorithms for CNMF model.
result Improved performance on large-scale data compared to multiplicative updates.
Paper proposes an alternating back-propagation algorithm for generator networks.
problem Learning realistic generator models of natural images, video sequences, and sounds.
method Alternating back-propagation algorithm that iterates inferential and learning steps.
result The alternating back-propagation algorithm can learn realistic generator models of natural images, video sequences, and sounds.
New method uses low-rank tensor factor analysis for better image restoration.
problem Restoring images from limited data.
method Low-rank tensor factor analysis combined with ADMM.
result The method outperforms traditional approaches, especially at low sampling rates.
Generative model disentangles 3D shapes into independent factors.
problem Learning rich representations of deformable 3D shapes.
method Supervised 3D mesh-convolutional Variational AutoEncoder with latent feature disentanglement.
result Explicit disentanglement of latent factors improves shape generation and downstream tasks.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
Efficiently learns 3D convolutions with less data.
problem High parameter and data costs in 3D convolutions.
method Temporal factorization of 3D kernels.
result Significantly reduces training data requirement and parameter count.
Model disentangles font content and style.
problem Analyzing and reconstructing fonts.
method Variational inference and asymmetric transpose convolutional process.
result Model outperforms state-of-the-art models in font reconstruction.
CPGBN analyzes text sequences, capturing word order for better topic extraction.
problem Lack of word order in traditional text representations.
method CPFA processes words as sequences, CPGBN adds hierarchical topic modeling.
result CPGBN extracts high-quality latent representations capturing word order.
Paper analyzes Winograd convolution errors and proposes methods to reduce them.
problem Reduction of floating point error in Winograd convolution for deep neural networks.
method Analysis of worst case FP error, estimation of norm and conditioning, proposed evaluation orderings, sampling points selection, mixed-precision convolution, pairwise summation.
result Proposed methods significantly reduce FP error for a given block size, allowing larger block sizes and reduced computation.
Paper tackles cold start problem in recommendation systems using deep learning and latent factor models.
problem Cold start problem in recommendation systems for new users and items.
method Uses model-based approach and deep learning, specifically latent factor model and convolutional neural network.
result Significantly outperforms baseline estimators in experiments.
Paper presents a provably correct algorithm for CNMF under separable conditions.
problem Convolutive nonnegative matrix factorization (CNMF) under separable assumptions.
method Algorithm exploiting NMF model and existing separable NMF algorithms.
result Guaranteed solution in low noise settings, runs in polynomial time.
Improved generalization bounds for multi-class CNNs without explicit class dependence.
problem Generalization error bounds for deep learning with multi-class CNNs.
method Adapted Rademacher analysis to incorporate weight sharing, reducing dependence on the number of classes.
result Bounds have no explicit dependence on the number of classes, scaling with the norm of weight matrices.
Tensor methods have emerged as a powerful paradigm for consistent learning of many latent variable models such as topic models, independent component analysis and dictionary learning. Model parameters are estimated via CP decomposition of the observed higher order input moments. However, in many domains, additional inv…
DeepCAM learns convolutional dictionaries for image processing.
problem Processing high-dimensional signals like images efficiently.
method Introduces a Deep Convolutional Analysis Dictionary Model (DeepCAM) using convolutional dictionaries.
result DeepCAM achieves performance comparable to other methods on single image super-resolution.
DCCNNs reduce computational overhead and ambiguity in convolutional neural networks.
problem Reducing computational overhead and ambiguity in convolutional neural networks.
method Introducing a primal learning problem and constructing a dual convex training program, using Fenchel conjugates and Karush-Kuhn-Tucker conditions.
result Eliminates ambiguity and reduces computational overhead in constructing a large kernel matrix.
Paper proposes NNAFC for automatic financial factor construction.
problem Manual factor construction is time-consuming and prone to bias.
method NNAFC uses neural networks to automatically construct diversified financial factors.
result NNAFC outperforms GP in constructing more informative and diversified factors.
The paper explores stability and generalization of deep GCNs.
problem Understanding the stability and generalization of deep GCNs from a theoretical perspective.
method Theoretical analysis of stability and generalization properties of deep GCNs.
result The stability and generalization of deep GCNs are influenced by the maximum absolute eigenvalue of the graph filter operators and the depth of the network.
A new hypernetwork architecture improves link prediction in knowledge graphs.
problem Link prediction in incomplete knowledge graphs.
method Hypernetwork architecture generating simplified relation-specific convolutional filters.
result Hypernetwork outperforms ConvE and previous approaches across standard datasets.
Paper extends 2D β β β -CNMF with exact multiplicative updates.
problem Improving nonnegative matrix factor deconvolution for 2D data.
method Derives exact multiplicative updates for β β β -CNMF factors. result The updates lead to monotonically decreasing β β β -divergence. Paper proposes a new LSTM model for spatio-temporal learning.
problem Challenging video tasks require learning long-term spatio-temporal correlations.
method Introduces a higher-order convolutional LSTM model with tensor train decomposition.
result Model achieves state-of-the-art performance with significantly fewer parameters.
New updates for β \beta β -divergence in convolutional NMF are stable and consistent.
problem Improving the stability and consistency of NMF updates for convolutional data.
method Presented multiplicative updates for β \beta β -divergence in closed form. result The new updates are stable and consistent across common β \beta β values. Smooth algebra analysis for one-dimensional singular foliations.
problem Analyzing smooth algebras of one-dimensional singular foliations.
method Analyzing natural ideals and using Dixmier-Malliavin theorem.
result Smooth algebras of one-dimensional singular foliations are pairwise nonisomorphic.
Second-order optimization methods such as natural gradient descent have the potential to speed up training of neural networks by correcting for the curvature of the loss function. Unfortunately, the exact natural gradient is impractical to compute for large models, and most approximations either require an expensive it…
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
The paper proposes a factorization method to interpret deep neural networks.
problem Interpreting and debugging deep neural networks.
method Factorization based approach to understand deep neural networks.
result Identified patterns linking factorization rank to network training quality.
WaveletNet improves edge device efficiency with logarithmic convolution.
problem Efficiency and performance on edge devices for CNNs.
method Introduces WaveletNet architecture with wavelet convolution and depthwise fast wavelet transform.
result WaveletNet achieves superior and comparable performance to state-of-the-art models on CIFAR-10 and ImageNet.
GCNs improve multi-layer network classification by expanding the distance between means.
problem Improving multi-layer network classification with graphical information.
method Theoretical and empirical study of graph convolutions in multi-layer networks.
result Graph convolutions expand the classification regime by a factor of 1 / E m d e g 4 1/\sqrt[4]{\mathbb{E}{
m deg}} 1/ 4 E m d e g . Corrected graph convolutions improve node classification on graphs.
problem Oversmoothing in graph convolutions degrades performance.
method Theoretical analysis based on CSBM, spectral analysis for k rounds of corrected graph convolutions.
result Corrected graph convolutions can improve node classification performance exponentially.
Study shows fewer samples needed for CNNs and RNNs than FNNs.
problem Estimating the number of samples needed for CNNs and RNNs.
method Localized empirical process analysis and new lemma characterizing CNNs and RNNs.
result Sample complexity scales linearly with intrinsic dimension for CNNs and RNNs.
Modeling influenza spread using feature engineering and international flow deconvolution.
problem Predicting and mitigating influenza spread through feature extraction and international flow analysis.
method Discrete Fourier Transform, matrix completion, SVM, autoencoders, PCA, deconvolution of international flow.
result Significant environmental and economic features are crucial to influenza mortality.
Dilated convolutions model long-distance genomic dependencies effectively.
problem Detecting regulatory elements from raw DNA with long-distance dependencies.
method Developed and used a novel dataset for dilated convolutional neural networks.
result Dilated convolutions are effective at modeling regulatory elements in the human genome.
New pursuit algorithm for ML-CSC model with improved stability and dictionary learning.
problem Lack of exact pursuit algorithms and conditions for non-empty model in ML-CSC.
method Projection approach for pursuit algorithm, stability bounds, practical alternatives, online dictionary learning.
result Sound pursuit algorithm and practical dictionary learning for ML-CSC model.
New methods use Kronecker-factored approximations for faster deep learning optimization.
problem Optimizing deep learning models with rich curvature information.
method Approximate Hessian using Kronecker products for efficient quasi-Newton methods.
result New methods outperform first-order methods and perform comparably to second-order methods.
GroSS enables efficient search for grouped convolutional architectures.
problem Training grouped convolutional architectures efficiently and effectively.
method GroSS: Group-Size Series Decomposition for Grouped Architecture Search.
result Simultaneous training of differing numbers of groups within a single layer and all possible combinations between layers.
A new method for optimizing deep neural networks using TKFAC.
problem Optimizing deep neural networks with second-order methods.
method Proposes Trace-restricted Kronecker-factored Approximate Curvature (TKFAC) for Fisher information matrix approximation.
result TKFAC improves performance on deep network architectures compared to state-of-the-art algorithms.
New approach to deeper graph neural networks to avoid performance degradation.
problem Performance degradation of graph neural networks when going deeper.
method Decoupling representation transformation and propagation in graph convolution operations.
result Deeper graph neural networks can be used to learn graph node representations from larger receptive fields.
New bounds improve deep learning performance efficiently.
problem Improving generalization and robustness of deep learning models.
method Deriving four provable upper bounds on spectral norm of convolution layers, differentiable and efficient.
result Minimum of four bounds is a tight, differentiable and efficient upper bound on spectral norm.
Optimizes CNN architectures by analyzing receptive fields without training.
problem CNNs often have too many layers, making them resource-intensive and less effective.
method Layer-wise analysis of receptive fields to identify unproductive layers.
result Identifies and removes unproductive layers, improving CNN performance and efficiency.
Optimal task order improves continual learning performance.
problem Challenges in neural networks learning multiple tasks in sequence.
method Linear teacher-student model with latent factors, derived analytical expression.
result Two principles for optimal task order: least representative first and dissimilar adjacent tasks.
New layers improve CNN performance with limited data.
problem Limited labeled data for CNNs.
method Sparse factorization layers for neural networks.
result Sparse factorization layers improve CNN performance with limited data.