Convolutional networks outperform fully-connected ones in certain tasks.
problem Understanding the computational advantage of convolutional networks over fully-connected networks.
method Demonstrated a computational advantage through a specific problem class.
result Convolutional networks can solve certain problems that fully-connected networks cannot, even with gradient descent.
Fully convolutional neural networks (FCN) have been shown to achieve state-of-the-art performance on the task of classifying time series sequences. We propose the augmentation of fully convolutional networks with long short term memory recurrent neural network (LSTM RNN) sub-modules for time series classification. Our …
Paper presents efficient algorithms for convolutional neural networks using Winograd minimal filtering.
problem Resource-efficient implementation of convolutional neural networks.
method Winograd minimal filtering trick applied to M-tap filters (M=3,5,7,9,11) for parallel hardware implementation.
result Approximately 30% reduction in multipliers for fully parallel hardware implementation.
Proves DCNNs with expansive convolution are strongly universally consistent.
problem Theoretical consistency of deep convolutional neural networks (DCNNs).
method Empirical risk minimization on DCNNs with expansive convolution (with zero-padding).
result DCNNs with expansive convolution are strongly universally consistent.
The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters. Reducing the number of parameters while preserving essentially the same predictive performance is critically important …
Recently the generalization error of deep neural networks has been analyzed through the PAC-Bayesian framework, for the case of fully connected layers. We adapt this approach to the convolutional setting.
Paper uses CNNs for eye tracking data segmentation, generation, and reconstruction.
problem Semantic segmentation and reconstruction of raw eye tracking data.
method Fully convolutional neural networks (FCNNs) for data segmentation and generation, variational auto-encoder for data generation.
result FCNNs can process any input size directly without preprocessing, generating raw eye tracking data.
New method shows fully-connected networks can learn convolutional structures from data.
problem How to learn convolutional structures from translation-invariant data.
method Data-driven emergence of convolutional structure in neural networks.
result Initially fully-connected networks can learn convolutional structures from their inputs.
Paper examines Gaussian process perspective of CNNs.
problem Understanding when and why CNNs perform well.
method Casts CNNs in a Gaussian process framework.
result Gains insights into CNN performance and assumptions.
The study shows removing fully connected output layers improves efficiency without sacrificing performance.
problem Large number of parameters in fully connected layers for high-category datasets.
method Examined architectures replacing fully connected output layers with fixed layers and compared performance.
result Fixed classifiers offer no additional benefit over removing the output layer and its parameters.
MS-FCN improves cardiac left ventricle segmentation accuracy.
problem Accurate segmentation of left ventricle from MRI images.
method Multi-scale fully convolutional network with dense connectivity.
result MS-FCN achieves state-of-the-art Dice scores (0.93 on endocardium, 0.96 on epicardium).
Paper introduces DropFilter and DropFilter-PLUS for CNN regularization.
problem Overfitting in Convolutional Neural Networks (CNNs).
method Randomly modifies convolution filters in CNNs.
result Improves performance on image classification tasks.
We present a new convolutional neural network-based time-series model. Typical convolutional neural network (CNN) architectures rely on the use of max-pooling operators in between layers, which leads to reduced resolution at the top layers. Instead, in this work we consider a fully convolutional network (FCN) architect…
This study proposes a fully convolutional network (FCN) model for raw waveform-based speech enhancement. The proposed system performs speech enhancement in an end-to-end (i.e., waveform-in and waveform-out) manner, which dif-fers from most existing denoising methods that process the magnitude spectrum (e.g., log power …
A new neural network separates singing voices more effectively.
problem Separating singing voices from mixed signals with high accuracy.
method MBR-FCN that processes different frequency bands with varying resolutions and filters.
result The MBR-FCN achieves better performance with fewer parameters.
Convolutional nets require fewer samples than fully-connected nets for image classification.
problem Understanding why convolutional nets are more sample-efficient than fully-connected nets.
method Construction of a natural distribution and target function to demonstrate a sample complexity gap.
result Convolutional nets require O(1) samples for a single target function, while fully-connected nets require Ω(d2) samples. New bounds for CNNs show better generalization than previous models.
problem Improving understanding of CNNs' generalization ability.
method Proposed tighter generalization bounds for CNNs by exploiting the sparse and permutation structure of weight matrices and spectral norms of convolution operations.
result Theoretical and experimental results show tighter bounds for CNNs than existing bounds.
This study presents a novel end-to-end architecture that learns hierarchical representations from raw EEG data using fully convolutional deep neural networks for the task of neonatal seizure detection. The deep neural network acts as both feature extractor and classifier, allowing for end-to-end optimization of the sei…
We present an instance segmentation algorithm trained and applied to a CCTV recording of beef cattle during a winter finishing period. A fully convolutional network was transformed into an instance segmentation network that learns to label each instance of an animal separately. We introduce a conceptually simple framew…
Proposes a low-rank deep CNN for multi-task learning.
problem Multi-task learning with deep neural networks.
method Low-rank deep network with nuclear norm and sparsity penalties.
result Improves performance on multiple tasks compared to standard models.
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…
Reduces CNN parameters by 2x with minimal loss in accuracy.
problem Parameter efficiency in distributed machine learning and edge devices.
method Introduces a Convolutional Slice Generator (CSG) to reduce CNN parameters.
result Reduces CNN parameters by approximately 2x with minimal accuracy loss.
The study examines deep convolutional neural networks and their learning ability.
problem Understanding the learning ability of deep convolutional neural networks (DCNNs).
method Examines DCNNs under both underparameterized and overparameterized settings, using a novel network deepening scheme.
result Establishes the first learning rates of underparameterized DCNNs and shows how adding layers can create interpolating DCNNs with good learning rates.
This article demonstrates that convolutional operation can be converted to matrix multiplication, which has the same calculation way with fully connected layer. The article is helpful for the beginners of the neural network to understand how fully connected layer and the convolutional layer work in the backend. To be c…
Deep neural networks are powerful learning models that achieve state-of-the-art performance on many computer vision, speech, and language processing tasks. In this paper, we study a fundamental question that arises when designing deep network architectures: Given a target network architecture can we design a smaller ne…
ESM-CNN uses error feedback to build a random CNN for time series forecasting.
problem Improving time series forecasting accuracy with CNNs.
method Incrementally adding random filters and neurons to adaptively construct a CNN.
result ESM-CNN outperforms state-of-the-art models in prediction accuracy and efficiency.
We study alignment in linear neural networks and its relation to gradient descent.
problem Understanding alignment in linear neural networks and its impact on training.
method Defined alignment for fully connected networks, analyzed alignment under gradient descent, and compared gradient descent to projected gradient descent for layer-constrained networks.
result Gradient descent can converge linearly to a global minimum when alignment is invariant, and alignment is impossible with large datasets in layer-constrained networks.
LeanConvNets reduces CNN costs without sacrificing accuracy.
problem High computational cost in fully-coupled CNNs.
method Sparsifying fully-coupled operators in CNNs.
result LeanConvNets achieve similar accuracy to state-of-the-art networks but with lower computational cost.
FiniteNet uses a neural network to improve PDE solving methods.
problem Improving accuracy in solving time-dependent PDEs.
method Fully convolutional LSTM network trained on simulation data.
result Reduces error by a factor of 2 to 3 compared to baseline methods.
Pruning FCNs reveals sub-networks that match CNNs' performance.
problem Understanding the inductive bias of pruning in neural networks.
method Iterative magnitude pruning of a simple FCN followed by analysis of the resulting architecture.
result Pruned FCNs exhibit key features of CNNs, suggesting new architectural biases.
Large filters improve performance but are costly; this work uses learned box filters and summed-area tables.
problem Improving performance in dense prediction tasks like human pose estimation with large filters.
method Adopted learnable box filters and summed-area tables to reduce computational cost and maintain performance.
result Demonstrated competitive performance on human pose estimation benchmarks.
FQ-Conv quantizes CNNs for efficient inference with low-precision weights and activations.
problem Reducing precision in DNNs leads to reduced accuracy.
method Fully quantized convolutional neural networks (FQ-Conv) using novel quantization and training techniques.
result Ternary-weight CNNs perform nearly as well as full-precision networks.
The paper verifies deep neural networks' ability to approximate functions on spheres.
problem Theoretical verification of deep neural networks' performance on spherical functions.
method Spherical analysis using reproducing kernels and convolutional factorizations.
result Rates of uniform approximation for functions in Sobolev spaces and additive ridge forms.
Proposes LC-ST-FCN for better ride-sourcing demand forecasting.
problem Local statistical differences in ride-sourcing demand across a city.
method LC-ST-FCN framework combining 3D and 2D convolutions, locally connected layers.
result Significant improvements in demand forecasting compared to baselines.
FCC-GAN combines fully connected and convolutional layers for improved GAN performance.
problem Lack of understanding in choosing GAN network architectures.
method Proposes FCC-GAN, a hybrid architecture combining fully connected and convolutional layers.
result FCC-GAN outperforms traditional GAN architectures in terms of learning speed and sample quality.
PBN combines generative and discriminative capabilities in a neural network.
problem Combining generative and discriminative capabilities in neural networks.
method Convolutional PBN, sharing FF-NN embodiment, combining generative and discriminative qualities.
result PBN shows excellent qualities from either generative or discriminative viewpoint.
The segmentation of the left ventricle (LV) from CINE MRI images is essential to infer important clinical parameters. Typically, machine learning algorithms for automated LV segmentation use annotated contours from only two cardiac phases, diastole, and systole. In this work, we present an analysis work-flow for fully-…
Empirical study shows standard CNNs deviate from NTK predictions.
problem Understanding how standard finite-width CNNs behave compared to their infinite-width NTK counterparts.
method Empirical analysis of AlexNet and LeNet architectures.
result Standard CNNs deviate significantly from their NTK counterparts, but deviation decreases with wider networks.
Paper explores offline and online image recognition using neural networks.
problem Challenges in evolving image recognition through different settings.
method Used Convolutional Neural Networks and Multi-layer Perceptrons.
result Encouraging preliminary results in offline and online image classification.
U-Time uses a fully convolutional network for sleep stage classification.
problem Challenges in tuning and optimizing recurrent neural networks for sleep data.
method U-Time is a fully feed-forward deep learning approach based on U-Net architecture.
result U-Time outperforms state-of-the-art models for sleep stage classification.
One-shot path planning for multiple agents using neural networks.
problem Efficiently generating optimal or near-optimal paths for multiple agents in robotics.
method Utilizes fully convolutional neural networks for one-shot multi-agent path planning.
result Demonstrates successful generation of optimal or near-optimal paths in over 85% of cases for multi-path planning.
Study examines noise sensitivity of DNNs for binary classification.
problem Understanding non-robustness of DNN classifiers under noise.
method Defined and extended noise sensitivity and stability concepts for Boolean functions, applied to DNN models.
result Sorted out the relation between definitions and properties of DNN architectures under noise.
In cardiac magnetic resonance imaging, fully-automatic segmentation of the heart enables precise structural and functional measurements to be taken, e.g. from short-axis MR images of the left-ventricle. In this work we propose a recurrent fully-convolutional network (RFCN) that learns image representations from the ful…
We propose a new image denoising algorithm, dubbed as Fully Convolutional Adaptive Image DEnoiser (FC-AIDE), that can learn from an offline supervised training set with a fully convolutional neural network as well as adaptively fine-tune the supervised model for each given noisy image. We significantly extend the frame…
The paper applies Information Bottleneck theory to CNNs and finds compression phase not always present.
problem Understanding the behavior of convolutional neural networks.
method Employed Information Bottleneck theory to analyze CNNs.
result Compression phase not observed in all CNN cases.
Structured linear substitutions improve both efficiency and accuracy in neural networks.
problem Improving neural network efficiency and accuracy tradeoff.
method Replacing linear components in pointwise convolutions with structured linear decompositions.
result Structured layers provide Pareto-optimal benefits in efficiency/accuracy.
New graph CNN layers improve accuracy on graph datasets.
problem Graph data relations are better represented as graphs, not grids.
method Proposed new graph CNN layers for vertex and edge features.
result Improved classification accuracy on graph datasets.
Convolutional network converts speaker voices without text.
problem Speaker conversion without text-based methods.
method Fully convolutional wav-to-wav network with ASR pre-training.
result Successfully converts TTS robot's voice to narrated audiobook voices.