New CNN learns depth features from scratch, outperforming transfer learning.
problem Limited depth data for RGB-D scene recognition.
method Bottom-up approach combining local weakly supervised training and global fine-tuning, modified CNN architecture.
result State-of-the-art accuracy on NYU2 and SUN RGB-D datasets.
Deep CNN architectures improve neonatal seizure detection accuracy.
problem Improving EEG-based neonatal seizure detection accuracy.
method Design and test of deep convolutional networks of varying depths compared to a shallow SVM-based detector.
result A deep 11-layer CNN architecture significantly outperforms shallow architectures, improving AUC90 from 82.6% to 86.8%.
The paper introduces a method to explain redundancy in deep CNNs using unit impulse response.
problem Redundancy in deep CNNs leads to unnecessary computations and increased cost.
method Empirical demonstration and unit impulse response analysis to identify and quantify redundancy across layers and depth.
result Identifies and quantifies redundancy in deep CNNs, providing better insights into their internal dynamics.
CNNs improve generalization to unseen audio devices with increased width, not depth.
problem CNNs are sensitive to specific audio recording devices in acoustic scene classification.
method Investigated the relationship between over-parameterization and generalization in CNNs for audio classification.
result Increasing width improves generalization to unseen devices without increasing the number of parameters.
CNN improves neutrino event reconstruction in IceCube DeepCore.
problem Difficulties in distinguishing muon neutrinos and reconstructing inelasticity at GeV scale energies.
method 2D Convolutional Neural Network exploiting time and depth translational symmetry.
result CNN model outperforms conventional methods for flavor identification and inelasticity reconstruction.
Unified learning-rate scale for CNNs and ResNets, avoiding depth imbalance.
problem Challenges in choosing an appropriate learning rate for deep networks, especially as depth increases.
method Introduces Arithmetic-Mean μ μ μ P (AM- μ μ μ P), constraining network-wide average pre-activation second moment to a constant scale, combined with residual-aware He fan-in initialization. result Demonstrates a − 3 / 2 -3/2 − 3/2 scaling law for learning rates across depths, enabling zero-shot learning-rate transfer. Over-parameterized CNNs show U-shaped test risk with depth increase.
problem Understanding the impact of depth on test risk in over-parameterized CNNs.
method Empirical image classification experiments and linear regression framework.
result Test risk is U-shaped with increasing depth in over-parameterized CNNs.
CNN model predicts fluvial floods quickly and accurately.
problem Real-time flood prediction is computationally demanding.
method Deep Convolutional Neural Network (CNN) trained on 2D hydraulic model outputs.
result CNN model outperforms SVR in predicting flood inundation.
Deep learning improves aerosol optical depth predictions from reanalysis data.
problem Biases in reanalysis datasets like MERRA-2 against ground truth AOD measurements.
method A hybrid CNN model combining MERRA-2 reanalysis with deep learning.
result The CNN-based model provides better AOD estimates than MERRA-2 alone.
Wide CNNs with shared weights and max pooling have linearly independent features and can achieve zero training error.
problem Understanding the optimization landscape and expressiveness of deep CNNs.
method Analysis of loss landscape and expressiveness of practical deep CNNs with shared weights and max pooling layers.
result Wide CNNs can achieve zero training error and have a well-behaved loss surface with almost no bad local minima.
DNArch learns CNN architectures by backpropagation.
problem Discovering optimal CNN architectures.
method Differentiable Neural Architectures (DNArch) learns CNN architectures by backpropagation, controlling kernel sizes, channels, downsampling positions, and depth.
result DNArch finds performant CNN architectures across various tasks.
Theoretical analysis of CNNs' inductive biases and their efficiency in approximating functions.
problem Understanding and optimizing the inductive biases in deep CNNs.
method Theoretical analysis combining multichanneling, downsampling, weight sharing, and locality.
result Deep CNNs with O ( log d ) \mathcal{O}(\log d) O ( log d ) depth can approximate any continuous function, and require O ~ ( log 2 d ) \widetilde{\mathcal{O}}(\log^2d) O ( log 2 d ) samples for sparse functions. Extracts decision trees from CNNs to explain concept importance.
problem Understanding how CNNs make decisions about human-understandable concepts.
method Inferring labeled concept data from CNN hidden layer activations and creating a shallow decision tree.
result Extracted decision trees accurately represent CNN classifications.
This paper reduces the complexity of deep CNNs by optimizing their topology.
problem Reduces computational complexity and storage in deep CNNs for resource-constrained platforms.
method Analyzes the impact of CNN topology (depth and width) on feature extraction, focusing on scattering networks.
result Designs networks of fixed depth to retain a significant portion of input signal energy in the feature vector.
New deep fusion methods improve human action recognition using depth and inertial sensor data.
problem Existing multimodal HAR frameworks lack mid-level feature fusion.
method Proposes three deep multilevel multimodal fusion frameworks, transforming depth and inertial sensor data into images and using convolution with Prewitt filter to create modality within modality.
result Supremacy of proposed fusion frameworks over existing methods on three publicly available datasets.
CNNs solve inverse problems during training, proving mutual coherence crucial for convergence.
problem Validation of CNN learning during training.
method Proved CNN elements solve inverse problems, discussed mutual coherence, and set training rules.
result Mutual coherence is necessary for CNNs to converge to optimum solutions.
Unified framework explains geometric properties of CNNs.
problem Understanding why encoder-decoder CNNs perform well.
method Unified mathematical framework based on recent neural network theories.
result Encoder-decoder CNNs are related to nonlinear basis representation using combinatorial convolution frames.
MBS reduces CNN model size with minimal accuracy loss.
problem Reducing CNN model size while maintaining accuracy.
method Adaptive macroblock scaling based on effective flops.
result Significant model size reduction across various CNN architectures.
Integrates CNN and GRU for precise stock market risk alerts.
problem Predicting future stock market risks and providing early warnings.
method Uses CNN for feature extraction and GRU for time series analysis.
result Effective early warnings of future stock market risks.
Gradient descent on deep linear CNNs converges to a penalty-based solution.
problem Understanding gradient descent convergence in deep linear convolutional networks.
method Gradient descent on full-width linear convolutional networks of varying depth.
result Gradient descent converges to a penalty-based solution, not the hard margin SVM solution.
The paper classifies soil texture using 1D CNNs on hyperspectral data.
problem Classifying soil texture from hyperspectral data.
method Developed and implemented three 1D CNNs: LucasCNN, LucasResNet, and LucasCoordConv.
result LucasCoordConv achieves the best performance in accuracy.
Study of infinitely deep but narrow neural networks using NTK theory.
problem Analyzing the role of depth in deep learning with overparameterized networks.
method Infinite-depth limit analysis of MLP and CNN using Neural Tangent Kernel (NTK) theory.
result Established trainability guarantee for infinitely deep but narrow neural networks.
GP-CNAS uses genetic programming to automatically design CNN architectures.
problem Designing optimal CNN architectures is laborious and error-prone.
method GP-CNAS uses a tree-based representation of CNNs and dynamic crossover operators to search for optimal architectures.
result GP-CNAS finds optimal CNN architectures with balanced depth and width in limited trials.
Ultra-efficient CNN accelerator for mobile and embedded devices.
problem Power consumption and performance in mobile and embedded applications.
method Design and production of a 28nm CNN-DSA accelerator.
result 9.3TOPS/Watt power efficiency with 140fps image classification.
A new data-level recombination strategy improves RGB-D salient object detection.
problem RGB-D salient object detection struggles with depth information.
method Proposes a novel data-level recombination strategy to fuse RGB and depth data before feature extraction.
result Achieves a new state-of-the-art performance in RGB-D salient object detection.
New tighter generalization bounds for deep networks like CNNs and ResNets.
problem Establishing tighter bounds for deep neural networks' generalization error.
method Introducing a new characterization of Lipschitz properties and margin-based data-dependent error bounds.
result Significantly tighter generalization bounds for deep neural networks, including CNNs and ResNets.
Paper explores modern CNNs for IoT-based farms.
problem Insufficient insight from agricultural IoT data.
method Review of state-of-the-art CNN architectures and their applications.
result Benchmarking guide for selecting CNN architectures.
A new CNN-based code generator outperforms RNNs by 5%.
problem Capturing long sequences in code generation using RNNs.
method Grammar-based structural CNN with tree-based and pre-order convolution modules.
result Significantly outperforms previous state-of-the-art methods by 5 percentage points.
Deep CNNs can approximate any function, proving their efficiency.
problem Understanding the approximation and generalization ability of deep learning methods.
method Demonstrated universality of deep convolutional neural networks (CNNs).
result Deep CNNs can approximate any continuous function to arbitrary accuracy.
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
CNNs reconstruct medium properties from wave probing responses.
problem Determining medium properties from wave responses.
method Deep convolutional neural networks (CNNs) for nonlinear wave equations.
result Quantitative dependence of network depth and units on medium complexity.
Study on memorization vs. generalization in overparameterized networks.
problem Understanding the trade-off between memorization and generalization in neural networks.
method Examined fully-connected and convolutional networks trained to minimize reconstruction error.
result Different architectures exhibit distinct inductive biases, affecting generalization from a single training example.
The paper applies Information Bottleneck theory to CNNs and finds compression phase not always present.
problem Understanding the behavior of convolutional neural networks.
method Employed Information Bottleneck theory to analyze CNNs.
result Compression phase not observed in all CNN cases.
CNNs generalize well despite learning surface statistical regularities.
problem CNNs' extreme sensitivity to adversarial examples raises doubts about learning high-level abstractions.
method Fourier filtering to construct datasets with same high-level abstractions but different surface statistics.
result CNNs exhibit a tendency to learn surface statistical regularities, leading to a 28% generalization gap.
RIFLE improves deep transfer learning by reinitializing fully-connected layers.
problem Limited improvement in transfer learning accuracy with pre-trained models on small datasets.
method Re-Initializing fully-connected layers with random scratch during fine-tuning.
result Significant improvement in deep transfer learning accuracy across various datasets.
Deep CNNs can be trained without special architectures.
problem Training extremely deep CNNs (10,000 layers) is challenging due to vanishing/exploding gradients.
method Developed a mean field theory for signal propagation and conditions for dynamical isometry. Derived an algorithm for generating random initial orthogonal convolution kernels.
result Vanilla CNNs with ten thousand layers can be efficiently trained using appropriate initialization schemes.
CNNs use a bottleneck structure to focus on a few frequencies, affecting function representation.
problem Understanding how CNNs focus on specific frequencies in their feature learning.
method Defined Convolution Bottleneck (CBN) structure, measured CBN rank, and analyzed parameter norms.
result Parameter norm scales with depth and CBN rank, and networks with optimal parameters exhibit this structure.
Study uses CNNs to detect sleep arousals more accurately.
problem Accurate detection of sleep arousals for better sleep quality assessment.
method Investigated five 1D CNN models on EEG signals from PhysioNet/Computing in Cardiology Challenge 2018.
result Best 1D CNN model achieved high precision and recall metrics.
Butterfly-Net improves CNN performance with structured connections and initialization.
problem Improving the performance of convolutional neural networks (CNNs).
method Butterfly-Net introduces structured and sparse cross-channel connections, and Butterfly initialization strategy.
result Butterfly-Net approximates Fourier representations with exponentially decaying error as depth increases.
Optimized CNNs for AMC on edge devices reduce complexity without sacrificing accuracy.
problem Developing efficient DL models for AMC on resource-constrained edge devices.
method Pruning, quantization, and knowledge distillation techniques applied to CNNs.
result Optimized models maintain or improve AMC accuracy with reduced complexity.
Improved CNNs detect Alzheimer's with 14% accuracy boost.
problem Early detection of Alzheimer's Disease using MRI scans.
method Optimized 3D CNNs with instance normalization, spatial downsampling, model widening, and age information.
result 14% increase in test accuracy distinguishing AD, MCI, and controls.
AVH scores measure sample hardness, improving model calibration.
problem CNNs' poor calibration and overconfidence issues.
method AVH score based on normalized angular distance between feature embeddings and target classifiers.
result AVH scores correlate with human visual hardness and improve model calibration.
Unified model for automated driving tasks improves efficiency and accuracy.
problem Efficiently handle multiple visual perception tasks in automated driving systems.
method Joint multi-task network design sharing convolutional layers, multi-stream learning, auxiliary learning.
result Unified model outperforms single-task models in many cases.
Paper proposes DeCEF layers to reduce CNN complexity.
problem Reduces complexity of CNNs without pre-trained models.
method Develops Depthwise Convolutional Eigen-Filter (DeCEF) layers.
result Achieves similar or higher accuracy with 2/3 parameters and 2/3 FLOPs.
IC-Network improves CNNs by integrating elastic collision units.
problem Designing more effective basic units in neural networks.
method Developed IC layer and IC block units combining the IC structure with convolution operations.
result Significant performance improvements in existing CNNs, reducing top-1 error from 22.85% to 21.49% on imagenet.
New approach predicts generalization of deep neural networks in proportional-width regime.
problem Predicting generalization of deep neural networks in proportional-width regime.
method Equivalent Wishart Ansatz for hierarchical empirical kernels, renormalized NNGP kernel.
result Renormalized NNGP kernel captures dominant stochastic fluctuations in deep neural networks.
Emotion being a subjective thing, leveraging knowledge and science behind labeled data and extracting the components that constitute it, has been a challenging problem in the industry for many years. With the evolution of deep learning in computer vision, emotion recognition has become a widely-tackled research problem…
Capsule Networks improve autonomous navigation in sparse environments.
problem Challenges in reinforcement learning for sparse reward environments.
method Caps-EM pairs CapsNets with Advantage Actor Critic, using fewer parameters.
result Caps-EM achieves significant time improvements and fewer parameters compared to competing methods.