Convolutional networks outperform fully-connected ones in certain tasks.
problem Understanding the computational advantage of convolutional networks over fully-connected networks.
method Demonstrated a computational advantage through a specific problem class.
result Convolutional networks can solve certain problems that fully-connected networks cannot, even with gradient descent.
Fully convolutional neural networks (FCN) have been shown to achieve state-of-the-art performance on the task of classifying time series sequences. We propose the augmentation of fully convolutional networks with long short term memory recurrent neural network (LSTM RNN) sub-modules for time series classification. Our …
FCC-GAN combines fully connected and convolutional layers for improved GAN performance.
problem Lack of understanding in choosing GAN network architectures.
method Proposes FCC-GAN, a hybrid architecture combining fully connected and convolutional layers.
result FCC-GAN outperforms traditional GAN architectures in terms of learning speed and sample quality.
Paper presents efficient algorithms for convolutional neural networks using Winograd minimal filtering.
problem Resource-efficient implementation of convolutional neural networks.
method Winograd minimal filtering trick applied to M-tap filters (M=3,5,7,9,11) for parallel hardware implementation.
result Approximately 30% reduction in multipliers for fully parallel hardware implementation.
CCTV footage of beef cattle segmented into individual instances.
problem Early detection of animal welfare issues in beef cattle.
method Transformed a fully convolutional network into an instance segmentation network.
result Network outputs a single prediction for every animal in CCTV footage.
Convolutional nets require fewer samples than fully-connected nets for image classification.
problem Understanding why convolutional nets are more sample-efficient than fully-connected nets.
method Construction of a natural distribution and target function to demonstrate a sample complexity gap.
result Convolutional nets require O(1) samples for a single target function, while fully-connected nets require Ω(d2) samples. Proves DCNNs with expansive convolution are strongly universally consistent.
problem Theoretical consistency of deep convolutional neural networks (DCNNs).
method Empirical risk minimization on DCNNs with expansive convolution (with zero-padding).
result DCNNs with expansive convolution are strongly universally consistent.
FC-AIDE improves image denoising with a fully convolutional neural network.
problem Improving image denoising performance with neural networks.
method Fully convolutional neural network with adaptive fine-tuning.
result FC-AIDE outperforms state-of-the-art denoisers on benchmark datasets.
Proposes FCNs for text classification with size-invariant inputs.
problem Classifying text of varying sizes.
method Uses fully convolutional networks with modifications to attention mechanisms.
result Suboptimal results on ITAmoji task, proposes fixes.
The study shows removing fully connected output layers improves efficiency without sacrificing performance.
problem Large number of parameters in fully connected layers for high-category datasets.
method Examined architectures replacing fully connected output layers with fixed layers and compared performance.
result Fixed classifiers offer no additional benefit over removing the output layer and its parameters.
Paper uses CNNs for eye tracking data segmentation, generation, and reconstruction.
problem Semantic segmentation and reconstruction of raw eye tracking data.
method Fully convolutional neural networks (FCNNs) for data segmentation and generation, variational auto-encoder for data generation.
result FCNNs can process any input size directly without preprocessing, generating raw eye tracking data.
The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters. Reducing the number of parameters while preserving essentially the same predictive performance is critically important …
MS-FCN improves cardiac left ventricle segmentation accuracy.
problem Accurate segmentation of left ventricle from MRI images.
method Multi-scale fully convolutional network with dense connectivity.
result MS-FCN achieves state-of-the-art Dice scores (0.93 on endocardium, 0.96 on epicardium).
This study proposes a fully convolutional network (FCN) model for raw waveform-based speech enhancement. The proposed system performs speech enhancement in an end-to-end (i.e., waveform-in and waveform-out) manner, which dif-fers from most existing denoising methods that process the magnitude spectrum (e.g., log power …
U-Time uses a fully convolutional network for sleep stage classification.
problem Challenges in tuning and optimizing recurrent neural networks for sleep data.
method U-Time is a fully feed-forward deep learning approach based on U-Net architecture.
result U-Time outperforms state-of-the-art models for sleep stage classification.
Large filters improve performance but are costly; this work uses learned box filters and summed-area tables.
problem Improving performance in dense prediction tasks like human pose estimation with large filters.
method Adopted learnable box filters and summed-area tables to reduce computational cost and maintain performance.
result Demonstrated competitive performance on human pose estimation benchmarks.
Recently the generalization error of deep neural networks has been analyzed through the PAC-Bayesian framework, for the case of fully connected layers. We adapt this approach to the convolutional setting.
This article demonstrates that convolutional operation can be converted to matrix multiplication, which has the same calculation way with fully connected layer. The article is helpful for the beginners of the neural network to understand how fully connected layer and the convolutional layer work in the backend. To be c…
In this contribution, we present a novel approach for segmenting laser radar (lidar) imagery into geometric time-height cloud locations with a fully convolutional network (FCN). We describe a semi-supervised learning method to train the FCN by: pre-training the classification layers of the FCN with image-level annotati…
Proposes a low-rank deep CNN for multi-task learning.
problem Multi-task learning with deep neural networks.
method Low-rank deep network with nuclear norm and sparsity penalties.
result Improves performance on multiple tasks compared to standard models.
Random convolutional networks can be fooled with adversarial examples.
problem Existence of adversarial examples for random convolutional networks.
method Utilizing isoperimetric inequalities on the special orthogonal group so(d). result Adversarial examples exist for various random convolutional networks.
New method shows fully-connected networks can learn convolutional structures from data.
problem How to learn convolutional structures from translation-invariant data.
method Data-driven emergence of convolutional structure in neural networks.
result Initially fully-connected networks can learn convolutional structures from their inputs.
Paper examines Gaussian process perspective of CNNs.
problem Understanding when and why CNNs perform well.
method Casts CNNs in a Gaussian process framework.
result Gains insights into CNN performance and assumptions.
In cardiac magnetic resonance imaging, fully-automatic segmentation of the heart enables precise structural and functional measurements to be taken, e.g. from short-axis MR images of the left-ventricle. In this work we propose a recurrent fully-convolutional network (RFCN) that learns image representations from the ful…
Stanza separates convolutional and fully connected layers for faster deep learning training.
problem Heavy data transfer between workers and servers in distributed deep learning.
method Layer separation: most nodes train convolutional layers, others train fully connected layers only.
result Significant acceleration of training time (1.34x--13.9x) over current systems.
Paper tackles depth estimation and optic disc-cup segmentation from color fundus images.
problem Depth estimation and optic disc-cup segmentation from color fundus images.
method Uses fully convolutional networks for monocular retinal depth estimation and optic disc-cup segmentation.
result Demonstrates improved accuracy in depth estimation and optic disc-cup segmentation.
Global Planar Convolution boosts brain tumor segmentation by enhancing context perception.
problem Improving context perception in brain tumor segmentation networks.
method Introduced Global Planar Convolution module to enhance context aggregation in brain tumor segmentation.
result Global Planar Convolution eliminates the need for multiple representation levels in segmentation networks.
Proposes LC-ST-FCN for better ride-sourcing demand forecasting.
problem Local statistical differences in ride-sourcing demand across a city.
method LC-ST-FCN framework combining 3D and 2D convolutions, locally connected layers.
result Significant improvements in demand forecasting compared to baselines.
We present a new convolutional neural network-based time-series model. Typical convolutional neural network (CNN) architectures rely on the use of max-pooling operators in between layers, which leads to reduced resolution at the top layers. Instead, in this work we consider a fully convolutional network (FCN) architect…
Pruning FCNs reveals sub-networks that match CNNs' performance.
problem Understanding the inductive bias of pruning in neural networks.
method Iterative magnitude pruning of a simple FCN followed by analysis of the resulting architecture.
result Pruned FCNs exhibit key features of CNNs, suggesting new architectural biases.
Deep learning improves sparse representation for better classification.
problem Improving classification accuracy using sparse representation.
method A transductive deep learning network combining convolutional autoencoder and fully-connected layers.
result The proposed network achieves better classification results than state-of-the-art SRC methods.
FCN improves speech clarity in noisy environments.
problem Improving speech clarity in noisy environments.
method Fully convolutional neural network (FCN) for speech enhancement.
result FCN can generalize to new speakers and robust to varying noise.
LeanConvNets reduces CNN costs without sacrificing accuracy.
problem High computational cost in fully-coupled CNNs.
method Sparsifying fully-coupled operators in CNNs.
result LeanConvNets achieve similar accuracy to state-of-the-art networks but with lower computational cost.
We show that gradient descent on full-width linear convolutional networks of depth L converges to a linear predictor related to the ℓ2/L bridge penalty in the frequency domain. This is in contrast to linearly fully connected networks, where gradient descent converges to the hard margin linear support vector m…
New method learns convolution-like structures from scratch.
problem Learning convolution-like structures from data.
method Minimum description length principle and β-LASSO algorithm. result Learned architectures achieve state-of-the-art accuracies.
The paper improves CNN models for polyp segmentation, enhancing their accuracy and interpretability.
problem Uncertainty and interpretability in CNNs for medical image analysis.
method Develop and evaluate recent advances in uncertainty estimation and model interpretability in FCNs for polyp segmentation.
result Highest performing model achieves 76.06% mean IOU accuracy on the EndoScene dataset.
Paper introduces DropFilter and DropFilter-PLUS for CNN regularization.
problem Overfitting in Convolutional Neural Networks (CNNs).
method Randomly modifies convolution filters in CNNs.
result Improves performance on image classification tasks.
A new neural network separates singing voices more effectively.
problem Separating singing voices from mixed signals with high accuracy.
method MBR-FCN that processes different frequency bands with varying resolutions and filters.
result The MBR-FCN achieves better performance with fewer parameters.
V-FCNN neural network segments atria from clinical images.
problem Automatic segmentation of atria from high-resolution images.
method Volumetric Fully Convolution Neural Network (V-FCNN) with a mixed loss function.
result 92.5% Dice similarity coefficient in 54 patients.
This study presents a novel end-to-end architecture that learns hierarchical representations from raw EEG data using fully convolutional deep neural networks for the task of neonatal seizure detection. The deep neural network acts as both feature extractor and classifier, allowing for end-to-end optimization of the sei…
Reduces CNN parameters by 2x with minimal loss in accuracy.
problem Parameter efficiency in distributed machine learning and edge devices.
method Introduces a Convolutional Slice Generator (CSG) to reduce CNN parameters.
result Reduces CNN parameters by approximately 2x with minimal accuracy loss.
Improved time series classification with GRU-FCN model.
problem Time series classification challenges.
method Hybrid LSTM-GRU model for univariate time series classification.
result GRU-FCN model outperforms state-of-the-art models.
Paper uses FMCW radar and FCN for object detection and 3D estimation.
problem Object detection and 3D estimation using FMCW radar.
method Employed deep learning (FCN) over traditional signal processing. Normalization method applied to radar signal.
result System successfully detects and estimates 3D position of objects in noisy environments.
SFCNeXt estimates brain age from small MRI datasets.
problem Efficiently estimating brain age from limited MRI data.
method Simple fully convolutional network (SFCNeXt) with SPEC and HRL.
result SFCNeXt outperforms complex models in small sample size scenarios.
Convolutional network converts speaker voices without text.
problem Speaker conversion without text-based methods.
method Fully convolutional wav-to-wav network with ASR pre-training.
result Successfully converts TTS robot's voice to narrated audiobook voices.
New bounds for CNNs show better generalization than previous models.
problem Improving understanding of CNNs' generalization ability.
method Proposed tighter generalization bounds for CNNs by exploiting the sparse and permutation structure of weight matrices and spectral norms of convolution operations.
result Theoretical and experimental results show tighter bounds for CNNs than existing bounds.
Convolutional networks outperform shallow classifiers on certain tasks due to hierarchical structure.
problem Understanding why convolutional networks outperform shallow classifiers on specific tasks.
method Approximation theory, visual tasks with deterministic scrambling, and network performance evaluation.
result Hierarchical structure is crucial for convolutional networks' performance on certain tasks, but not all.
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…