Reduced CNN complexity for exoplanet detection without significant loss in accuracy.
problem Reducing the number of parameters in CNNs for time-series data.
method Two methods for complexity reduction in AstroNet: tactical reduction of layers and modification of input data.
result Non-substantial loss in accuracy with up to 85% reduction in training time.
LeanResNet reduces CNN complexity without sacrificing accuracy.
problem High computational cost in CNNs, especially in ResNets.
method Introduces lean convolution operators that reduce parameters and complexity.
result LeanResNet achieves comparable results to other reduced architectures with fewer parameters.
In this paper we propose the macroblock scaling (MBS) algorithm, which can be applied to various CNN architectures to reduce their model size. MBS adaptively reduces each CNN macroblock depending on its information redundancy measured by our proposed effective flops. Empirical studies conducted with ImageNet and CIFAR-…
PixelHop uses SSL for image classification, outperforming CNN.
problem Image classification accuracy improvement.
method PixelHop combines SSL with supervised and unsupervised dimension reduction.
result PixelHop outperforms CNN on MNIST, Fashion MNIST, and CIFAR-10 datasets.
Convolutional Neural Networks (CNNs) require a large amount of annotated data to learn from, which is often difficult to obtain in the medical domain. In this paper we show that the sample complexity of CNNs can be significantly improved by using 3D roto-translation group convolutions (G-Convs) instead of the more conv…
New approximative kernels improve PDE-G-CNNs for geometric deep learning.
problem Inaccurate approximations of exact kernels in PDE-G-CNNs.
method Developed new approximative kernels that work regardless of spatial anisotropy.
result New kernels provide better error estimates and maintain reflectional symmetries.
New metric assesses image dataset complexity.
problem Assessing the complexity of image classification datasets.
method Cumulative Spectral Gradient (CSG) derived from spectral clustering.
result CSG correlates with CNN test accuracy and dataset separability.
A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…
Enhanced CNN for financial data improves predictive accuracy and stability.
problem Complexity and variability in financial data.
method Normalization and Gradient Reduction Architecture.
result Improvement in model accuracy and stability.
This paper proposes BRIEF, a backward reduction algorithm that explores compact CNN-model designs from the information flow perspective. This algorithm can remove substantial non-zero weighting parameters (redundant neural channels) of a network by considering its dynamic behavior, which traditional model-compaction te…
The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and the design of accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition…
Study on geodesic distances on SE(3)/SO(2) in machine learning.
problem Investigating the efficiency of computationally efficient sections in selecting geodesic distances.
method Analyzing geodesic distances on reductive homogeneous spaces, proving the efficiency of minimal distance sections.
result Minimal distance sections are not always geodesic minimizers, but minimal horizontal geodesics are.
Paper proposes scalable multi-label classification for edge devices using CNN.
problem Challenges in deploying multi-label CNN models on edge devices due to high computation and memory requirements.
method Extends existing multi-label classification methods with a single CNN model and multiple loss and accuracy layers.
result Achieves comparable accuracy with 1.8x less MACC operations, 0.97x reduction in latency and 0.5x, 0.84x, 0.97x reduction in size for generated CNN models.
In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly so-called pooling. Maximation and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convoluti…
Optimized CNNs for AMC on edge devices reduce complexity without sacrificing accuracy.
problem Developing efficient DL models for AMC on resource-constrained edge devices.
method Pruning, quantization, and knowledge distillation techniques applied to CNNs.
result Optimized models maintain or improve AMC accuracy with reduced complexity.
CNNs improve InSAR image denoising and coherence estimation.
problem Noise in InSAR imagery corrupts ground movement estimates.
method Autoencoder CNN architectures for denoising and preprocessing.
result Proposed method outperforms four established methods.
In a previous work we have detailed the requirements to obtain a maximal performance benefit by implementing fully connected deep neural networks (DNN) in form of arrays of resistive devices for deep learning. This concept of Resistive Processing Unit (RPU) devices we extend here towards convolutional neural networks (…
GrateTile optimizes CNN feature map storage for efficient data access.
problem Efficient storage and access of sparse CNN feature maps.
method Divides feature maps into uneven-sized subtensors, compresses and stores them in a compressed yet accessible format.
result Average 55% DRAM bandwidth reduction with minimal indexing overhead.
New method for nonlinear SDR of complex non-Euclidean data.
problem Nonlinear SDR for complex non-Euclidean random objects.
method Fréchet Cumulative Covariance (FCCov) and neural networks.
result Robust and unbiased nonlinear SDR for complex data.
Discrete Fourier transforms provide a significant speedup in the computation of convolutions in deep learning. In this work, we demonstrate that, beyond its advantages for efficient computation, the spectral domain also provides a powerful representation in which to model and train convolutional neural networks (CNNs).…
New complexity measure shows similar generalization bounds for CNNs and non-CNNs.
problem Understanding why CNNs generalize well despite fitting random labels.
method Theoretical and empirical investigation of spectral complexity measure insensitivity to CNN invariances.
result Spectral complexity measure results in the same upper bound complexity estimates for CNNs and non-CNNs, contradicting common intuition.
In this paper, we propose a deep multimodal fusion network to fuse multiple modalities (face, iris, and fingerprint) for person identification. The proposed deep multimodal fusion algorithm consists of multiple streams of modality-specific Convolutional Neural Networks (CNNs), which are jointly optimized at multiple fe…
New algorithm reduces sample complexity for learning CNNs.
problem Learning one-hidden-layer CNNs with various activation functions.
method Approximate gradient descent algorithm for training CNNs.
result Sample complexity matches information-theoretic lower bound for linear activation functions.
It is widely believed that the practical success of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) owes to the fact that CNNs and RNNs use a more compact parametric representation than their Fully-Connected Neural Network (FNN) counterparts, and consequently require fewer training examples to…
Large receptive field CNNs improve distant speech recognition.
problem Degrading performance of ASR systems in noisy environments.
method Investigated large receptive field CNN variants including recursive, dilated, and hourglass networks.
result Stacked hourglass networks show significant improvements in distant speech recognition.
New multigrid approach reduces CNN parameters by focusing on structured convolutions.
problem Redundancy in standard CNNs leads to high parameter count.
method Replace standard convolutions with structured multilevel convolutions.
result Linearly proportional number of parameters to network width, no loss in accuracy.
CBC makes CNNs robust against adversarial attacks with minimal computational overhead.
problem Making CNNs robust against adversarial attacks without increasing computational complexity.
method CBC uses a stacked encoder-convolutional model where an auto-encoder encodes the input image, and the latent representation is used for classification.
result CBC is more robust to adversarial examples and has significantly lower computational complexity.
CNN improves OSA diagnosis accuracy from PSG data.
problem Manual PSG analysis by specialists is tedious, time-consuming, and prone to errors.
method 1D CNN architecture with convolutional and FCN layers for OSA severity classification.
result Proposed CNN model achieves excellent classification results without manual preprocessing.
Alternative to convolutions using decision trees for neural networks.
problem Replacing complex convolutions with simpler decision-based layers.
method Binary decisions as indices to conditional distributions, trained using backpropagation.
result Performance similar to conventional neural networks, with runtime improvements.
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the K/R ratio, for compressed CNNs. In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the…
Ensemble of diverse CNNs detects and mitigates adversarial attacks.
problem Detecting and defending against adversarial attacks.
method An ensemble of specialized CNNs with a voting mechanism.
result Significant reduction in adversarial attack risk rate.
Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.
problem CNNs lack accurate data representation in their built-in prototypes.
method Introduces k-means-based post-hoc explanations for CNNs, leveraging spatial consistency of convolutional receptive fields.
result Using shallower, less compressed feature activations improves semantic fidelity at the cost of slight predictive performance.
Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
GenInSAR uses CNNs to filter InSAR phase and estimate coherence without supervision.
problem Improving accuracy in InSAR phase filtering and coherence estimation.
method Unsupervised CNN-based generative model for joint phase filtering and coherence estimation.
result GenInSAR outperforms five related methods in residue reduction and coherence estimation.
Convolutional Neural Networks(CNNs) are both computation and memory intensive which hindered their deployment in mobile devices. Inspired by the relevant concept in neural science literature, we propose Synaptic Pruning: a data-driven method to prune connections between input and output feature maps with a newly propos…
CNN improves medium-range temperature forecasts with limited resources.
problem Limited computational resources for high-resolution temperature forecasts.
method CNN post-processing with ensemble NWP models for bias correction and spatial downscaling.
result High-resolution (5-km) surface temperature forecasts with lead times up to 5.5 days.
Approximate multipliers boost CNN training speed, power, and area at slight accuracy cost.
problem Improving CNN training performance in terms of speed, power, and area.
method Simulation of approximate multipliers' impact on CNN training, hybrid training method combining approximate and exact multipliers.
result Using approximate multipliers for most of training enhances speed, power, and area with minimal accuracy loss.
This work studies the entity-wise topical behavior from massive network logs. Both the temporal and the spatial relationships of the behavior are explored with the learning architectures combing the recurrent neural network (RNN) and the convolutional neural network (CNN). To make the behavioral data appropriate for th…
Reduces CNN parameters by 2x with minimal loss in accuracy.
problem Parameter efficiency in distributed machine learning and edge devices.
method Introduces a Convolutional Slice Generator (CSG) to reduce CNN parameters.
result Reduces CNN parameters by approximately 2x with minimal accuracy loss.
2D CNNs approximate Korobov functions with near-optimal rates.
problem Approximating Korobov functions using 2D CNNs.
method Constructive approach for 2D CNNs with ReLU activations and fully connected layers.
result 2D CNNs achieve near-optimal approximation rates for Korobov functions.
Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, suc…
Efficient Winograd convolution for INT8 networks using RNS.
problem Difficulty in applying Winograd algorithm to low-precision quantized networks.
method Extends Winograd algorithm to Residue Number System (RNS) for efficient INT8 convolution.
result Arithmetic complexity reduction up to 7.03x with performance improvement up to 2.30x-4.69x.
In recent years, deep neural networks have achieved great success in the field of computer vision. However, it is still a big challenge to deploy these deep models on resource-constrained embedded devices such as mobile robots, smart phones and so on. Therefore, network compression for such platforms is a reasonable so…
New method reduces version space for CNNs, improving active learning performance.
problem Sampling bias in active learning hinders optimal hypothesis finding in neural networks.
method Version space reduction through prior mass reduction and diameter reduction, proposing a new Gibbs-vote disagreement method.
result Diameter-based querying method reduces version space more effectively than prior mass reduction and other methods.
Collage inference uses redundancy to reduce cloud image classification latency variance.
problem Reducing latency variance in cloud image classification.
method Integrates collage-cnn for low-cost redundancy in multi-image classification.
result Significant reduction in 99th percentile tail latency and inference latency variation.
New method uses eigenspectrum to optimize object detector architectures.
problem Understanding the effects of ImageNet pre-training on object detectors.
method Analysis of eigenspectrum dynamics of feature maps in object detectors.
result Object detectors trained from scratch and ImageNet pre-trained models behave differently.
This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the …