New metrics differentiate effective OOD sets for training calibrated CNNs.
problem Vanilla CNNs struggle with out-of-distribution (OOD) samples.
method Developed metrics based on generalization errors of Augmented-CNN.
result Most protective OOD sets lead to better A-CNN performance.
Proposes a new architecture to speed up large-scale CNN training.
problem Time-consuming training process of large-scale CNNs.
method Bi-layered Parallel Training (BPT-CNN) architecture with outer-layer and inner-layer parallelism.
result Improves training performance of CNNs while maintaining accuracy.
CNNs show sensitivity to low-frequency signals due to image frequency distribution.
problem Understanding why CNNs are sensitive to low-frequency signals.
method Theoretical analysis of CNN representations in frequency space.
result CNNs sensitivity to low-frequency signals is due to the frequency distribution of natural images.
GradiVeQ reduces CNN training time by 50% with 5X faster gradient aggregation.
problem Significant communication costs in gradient aggregation for distributed CNN training.
method GradiVeQ uses PCA to vector quantize gradients for direct RAR aggregation.
result GradiVeQ reduces wall-clock gradient aggregation time by more than 5X.
A distributed deep learning approach speeds up precipitation nowcasting model training.
problem Training deep neural networks for precipitation nowcasting is time-consuming.
method Data-parallel model with distributed training across multiple compute nodes and GPUs.
result Training time reduced from 59 hours to 1 hour for CNN models.
Inspired by recent advances in deep learning, we propose a novel iterative BP-CNN architecture for channel decoding under correlated noise. This architecture concatenates a trained convolutional neural network (CNN) with a standard belief-propagation (BP) decoder. The standard BP decoder is used to estimate the coded b…
Proposes a FoE prior for improving CNN performance in distribution shifts.
problem Improving CNN performance in image analysis tasks with distribution shifts.
method Uses a field-of-experts (FoE) prior to match feature distributions of test and training images.
result Outperforms previous TTA methods in lesion segmentation and most healthy tissue segmentation tasks.
A new method prunes activation gradients to speed up CNN training.
problem Challenges in accelerating CNN training using sparsity.
method Randomly prunes small activation gradients in back-propagation.
result Substantial speedups (up to 5.92x) with minimal accuracy loss.
Paper introduces zero-space memory protection for CNNs without ECC overhead.
problem Ensuring reliability of CNNs in safety-critical applications.
method In-place zero-space ECC with weight distribution-oriented training.
result First known zero-space cost memory protection for CNNs.
Improved CNN training speed with K-FAC on large datasets.
problem Challenges in training large-scale neural networks with distributed processors.
method Scalable K-FAC design, layer-wise distribution, inverse-free second-order gradient evaluation, dynamic K-FAC update decoupling.
result Distributed K-FAC implementation converges to 75.9% MLPerf baseline in 18-25% less time than SGD.
Reduces CNN parameters by 2x with minimal loss in accuracy.
problem Parameter efficiency in distributed machine learning and edge devices.
method Introduces a Convolutional Slice Generator (CSG) to reduce CNN parameters.
result Reduces CNN parameters by approximately 2x with minimal accuracy loss.
A two-stage CNN approach for brain tumor segmentation.
problem Brain tumor segmentation with class imbalance.
method Cascade V-Net architecture with ROI masks.
result Improved brain tumor segmentation accuracy.
New CNN layer selects important channels to improve model capacity.
problem Improving model capacity under resource constraints.
method Selective allocation of channels in convolutional layers.
result New layer allows new optima that generalize better.
MPT improves CNN and energy-based models' OOD detection and generalization.
problem Challenging out-of-distribution detection in computer vision.
method Applying Maximum Probability Theorem as a regularization scheme in CNN and energy-based models.
result MPT-based regularization strategy stabilizes and improves generalization and robustness of base models.
Ego-CNN detects critical structures in graphs efficiently.
problem Lack of precise detection of critical structures in existing graph embedding models.
method Ego-CNN uses ego-convolutions at each layer and stacks them in an ego-centric way.
result Ego-CNN achieves comparable task performance to state-of-the-art models and can incorporate scale-free priors.
Two-layer CNN learns key patterns in specific data distributions.
problem Understanding why CNNs work well in computer vision.
method Investigated learning dynamics of a two-layer nonlinear CNN over specific data distributions.
result Convolutional filters learn key patterns in data and dominate during training.
Convolutional neural networks (CNNs) work well on large datasets. But labelled data is hard to collect, and in some applications larger amounts of data are not available. The problem then is how to use CNNs with small data -- as CNNs overfit quickly. We present an efficient Bayesian CNN, offering better robustness to o…
AT-CNNs show improved shape recognition over texture recognition.
problem Understanding adversarial training's impact on CNNs' feature learning.
method Systematic qualitative and quantitative approaches to interpret AT-CNNs.
result Adversarial training reduces texture bias and improves shape recognition.
Study improves CNN medical image segmentation accuracy and reliability.
problem Over-confident predictions and silent failures in out-of-distribution data.
method Multi-task learning and spectral analysis of CNN feature maps.
result Joint multi-task learning models outperform dedicated models and detect OOD data more accurately.
Un-trained neural networks outperform trained methods in MRI reconstruction.
problem Accelerated MRI reconstruction with minimal training data.
method Variation of Deep Decoder without training data.
result Un-trained approach significantly outperforms other methods in reconstruction accuracy.
Paper improves image classification accuracy with a new Noise Modeling Network.
problem Improving performance of multi-label image classifiers with noisy or missing labels.
method Integrates a Noise Modeling Network (NMN) with a CNN to jointly learn noise distribution and CNN parameters.
result Consistently improves classification performance on MSR-COCO and MSR-VTT datasets.
Bayes-optimal classifiers are robust to adversarial attacks, unlike CNNs trained on the same data.
problem The vulnerability of modern CNN classifiers to adversarial examples.
method Constructing realistic image datasets and deriving analytic conditions for Bayes-optimal classifiers.
result Bayes-optimal classifiers are robust to adversarial attacks, unlike CNNs trained on the same data.
SCNN improves video object detection speed by 178%.
problem Limited throughput in existing CNNs for video object detection.
method Proposes SCNN, a statistical CNN that processes correlated distributions.
result Achieves 178% speedup over existing CNNs for video object detection.
Proposes GPCA module for channel attention in CNNs using Gaussian processes.
problem Improving performance in visual tasks through effective channel selection.
method Integrates Gaussian processes into channel attention mechanisms for probabilistic modeling of channel correlations.
result Demonstrates improved performance of GPCA module in end-to-end CNN training.
LEGW improves large-batch training for both CNNs and RNNs.
problem Efficient large-batch training for deep neural networks.
method Linear-epoch gradual-warmup (LEGW) for large-batch training.
result Improved large-batch training for both CNNs and RNNs with Sqrt Scaling scheme.
Designs a new generative model for images using CNNs and improves generalization and performance.
problem Improving image generation and generalization in CNNs.
method Deconvolutional Generative Model (DGM) using a conjugate prior and Rendering Path Normalization (RPN).
result Improves generalization and performance in semi-supervised and supervised learning tasks.
CNNs improve medical image classification with few samples.
problem Classifying medical images with limited training data.
method Transfer learning using CNNs, representation extraction, and a novel metric for performance prediction.
result CNN-based transfer learning outperforms feature-based methods with high correlation to test set performance.
In this paper, we address the issue of how to enhance the generalization performance of convolutional neural networks (CNN) in the early learning stage for image classification. This is motivated by real-time applications that require the generalization performance of CNN to be satisfactory within limited training time…
E2-Train reduces training energy by 80%+ for state-of-the-art CNNs.
problem Efficient training of energy-hungry CNNs on edge devices.
method Selective layer update, stochastic mini-batch dropping, and sign prediction for low-precision backpropagation.
result Achieves >90% energy savings for training ResNet-74 on CIFAR-10.
Paper proposes a new technique to compress CNNs while maintaining accuracy.
problem CNNs struggle with traditional low-rank approximation methods, leading to degraded accuracy.
method Introduces a training technique that finds a flat minimum in low-rank approximation without a decomposed structure.
result CNN models can be compressed with higher accuracy and lower computation than conventional methods.
Proposes a curriculum-based scheme to smooth CNN feature embeddings.
problem Distortion artifacts in early training stages of CNNs.
method Smoothes feature embedding using Gaussian kernels to control high-frequency information.
result Significant performance improvements on various vision tasks.
Batch Normalization (BN) is essential to effectively train state-of-the-art deep Convolutional Neural Networks (CNN). It normalizes inputs to the layers during training using the statistics of each mini-batch. In this work, we study BN from the viewpoint of Fisher kernels. We show that assuming samples within a mini-ba…
FPDeep accelerates CNN training on FPGA clusters with high parallelism and energy efficiency.
problem Scaling DNN training to large clusters with high utilization and balanced workload.
method Hybrid model and layer parallelism, fine-grained pipeline, balanced workload partitioning.
result FPDeep achieves high parallelism and utilization, reducing storage demand to on-chip memory.
CNN predicts optimal filter parameters for BM3D denoising.
problem Optimizing denoising filter performance in real-time.
method Trained a CNN to predict the best filter parameter value for BM3D.
result CNN-guided BM3D outperforms unguided BM3D across different noise levels.
CNNs can be trained with clinically available segmentations for OARs in radiotherapy.
problem Lack of dedicated training volumes for CNNs in radiotherapy.
method Used clinically available segmentations from PACS, applied multi-label segmentation, empirically assessed training set size.
result Clinically available segmentations can be used to train an accurate OAR segmentation model.
Approximate multipliers boost CNN training speed, power, and area at slight accuracy cost.
problem Improving CNN training performance in terms of speed, power, and area.
method Simulation of approximate multipliers' impact on CNN training, hybrid training method combining approximate and exact multipliers.
result Using approximate multipliers for most of training enhances speed, power, and area with minimal accuracy loss.
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
Boosting CNNs with dynamic feature selection and boosting weights improves accuracy and efficiency.
problem Expensive training of deep CNNs and manual architecture design.
method Dynamic feature selection, importance sampling, and boosting weights integrated into network training.
result Boosted CNN variants consistently outperform conventional CNNs in accuracy and training speed.
Optimizes CNN architectures by analyzing receptive fields without training.
problem CNNs often have too many layers, making them resource-intensive and less effective.
method Layer-wise analysis of receptive fields to identify unproductive layers.
result Identifies and removes unproductive layers, improving CNN performance and efficiency.
Unified framework denoises data and abstains from uncertain predictions.
problem Data quality and predictive uncertainty in deep neural networks.
method Unified filtering framework leveraging data density.
result Framework outperforms state-of-the-art techniques in denoising and abstaining.
CNN detects phase transitions in Potts models without prior knowledge.
problem Detecting phase transitions in q-state Potts models using deep learning. method Trained a deep CNN on Ising model spin configurations and temperatures, then tested on Potts model images.
result Deep CNN accurately detects phase transitions in Potts models, including high- and low-temperature regions.
End-to-end DA method for domain-invariant CNNs using parallel audio recordings.
problem Distribution mismatches between training and application data in machine listening.
method Enforcing equal hidden layer representations for domain-parallel samples.
result Learn domain-invariant classifiers without requiring classification labels.
The paper improves DFA for CNN and RNN training to match BP accuracy.
problem Low accuracy in CNN and RNN training using DFA.
method Divide network into modules, apply DFA within, use sparse backward weight, and incorporate dilated convolution and sparse matrix multiplication.
result Achieves BP-level accuracy in CNN and RNN training.
The paper introduces a method to explain redundancy in deep CNNs using unit impulse response.
problem Redundancy in deep CNNs leads to unnecessary computations and increased cost.
method Empirical demonstration and unit impulse response analysis to identify and quantify redundancy across layers and depth.
result Identifies and quantifies redundancy in deep CNNs, providing better insights into their internal dynamics.
Convolutional neural networks (CNN) have achieved state of the art performance on both classification and segmentation tasks. Applying CNNs to microscopy images is challenging due to the lack of datasets labeled at the single cell level. We extend the application of CNNs to microscopy image classification and segmentat…
Improved CNN training with BDFA reduces computational cost and improves accuracy.
problem Low training performance of DFA in CNN.
method Combining DFA with BP, introducing feedback weight initialization, and proposing BDFA.
result BDFA shows better performance than conventional BP, especially in small datasets.
ROIreg refines CNN's classification ability by focusing on pixel blocks.
problem Improving CNN's performance in semi-supervised and supervised learning.
method ROIreg divides pixel blocks, replaces them with random images, and trains CNN to maintain the maximum probability.
result ROIreg and VAT have complementary training effects, improving overall performance.
Mapping the spatial distribution of poverty in developing countries remains an important and costly challenge. These "poverty maps" are key inputs for poverty targeting, public goods provision, political accountability, and impact evaluation, that are all the more important given the geographic dispersion of the remain…