arXiv research

Efficient BNNs learn latent distributions for robust uncertainty quantification.

LTP learns per-layer thresholds for efficient pruning of deep networks.

Deep Learning detects cyclists' orientation for safer roads.

Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.

Averaging recent model checkpoints speeds up training time.

The human visual system is remarkably robust against a wide range of naturally occurring variations and corruptions like rain or snow. In contrast, the performance of modern image recognition models strongly degrades when evaluated on previously unseen corruptions. Here, we demonstrate that a simple but properly tuned …

Optical Coherence Tomography allows ophthalmologist to obtain cross-section imaging of eye retina. Assisted with digital image analysis methods, effective disease detection could be performed. Various methods exist to extract feature from OCT images. The proposed study aims to compare the effectiveness of handcrafted a…

Deep neural networks (DNNs) have achieved superior performance in various prediction tasks, but can be very vulnerable to adversarial examples or perturbations. Therefore, it is crucial to measure the sensitivity of DNNs to various forms of perturbations in real applications. We introduce a novel perturbation manifold …

We investigate whether ResNet architectures can outperform more traditional Convolutional Neural Networks on the task of fine-grained vehicle classification. We train and test ResNet-18, ResNet-34 and ResNet-50 on the Comprehensive Cars dataset without pre-training on other datasets. We then modify the networks to use …

Paper proposes a new black-box attack approach to minimize visual distortion.

We ask whether the neural network interpretation methods can be fooled via adversarial model manipulation, which is defined as a model fine-tuning step that aims to radically alter the explanations without hurting the accuracy of the original models, e.g., VGG19, ResNet50, and DenseNet121. By incorporating the interpre…

NG+ method improves deep learning efficiency and accuracy.

CNNs help diagnose diabetic retinopathy by localizing lesions.

Study identifies and mitigates causes of image misclassifications in CNN models.

TaskSet dataset speeds up hyperparameter optimization.

Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget. Existing methods rely on uniform or heuristic non-uniform sparsity budgets which have sub-optimal layer-wise parameter allocation resulting in a) lower prediction accuracy o…

State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resource intensive. Much research has been done on pruning or compressing these models to reduce the cost of inference, but little work has address…

Novel covariance function improves Bayesian optimization efficiency.

REPAIR mitigates variance collapse to enable linear interpolation between SGD solutions.

A new framework compresses neural networks using sparse optimization.

We design a new sparse projection method for a set of vectors that guarantees a desired average sparsity level measured leveraging the popular Hoyer measure (an affine function of the ratio of the $\ell_1$ and $\ell_2$ norms). Existing approaches either project each vector individually or require the use of a regulariz…

Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model. Although trained on images of aspecific size, it is well established that CNNs can be used to evaluate a wide range of image sizes at test time, by adjusting the size of intermediate feature maps.…

Channel Pruning, widely used for accelerating Convolutional Neural Networks, is an NP-hard problem due to the inter-layer dependency of channel redundancy. Existing methods generally ignored the above dependency for computation simplicity. To solve the problem, under the Bayesian framework, we here propose a layer-wise…

AdaRound improves post-training quantization of neural networks.

Large-batch training approaches have enabled researchers to utilize large-scale distributed processing and greatly accelerate deep-neural net (DNN) training. For example, by scaling the batch size from 256 to 32K, researchers have been able to reduce the training time of ResNet50 on ImageNet from 29 hours to 2.2 minute…

It is inevitable to train large deep learning models on a large-scale cluster equipped with accelerators system. Deep gradient compression would highly increase the bandwidth utilization and speed up the training process but hard to implement on ring structure. In this paper, we find that redundant gradient and gradien…

Majority of semantic segmentation algorithms operate on a single frame even in the case of videos. In this work, the goal is to exploit temporal information within the algorithm model for leveraging motion cues and temporal consistency. We propose two simple high-level architectures based on Recurrent FCN (RFCN) and Mu…

We present a method that trains large capacity neural networks with significantly improved accuracy and lower dynamic computational cost. We achieve this by gating the deep-learning architecture on a fine-grained-level. Individual convolutional maps are turned on/off conditionally on features in the network. To achieve…

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

A new pruning method finds sparse minimizers in flat regions of deep neural networks.

Proposes efficient training method for deep thin networks.

Deep learning ensemble improves Alzheimer vs. Mild Cognitive Impairment diagnosis.

Improved model robustness against corruptions using online adaptation.

Develops an efficient method for large-scale deep learning problems.

Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes are becoming increasingly relevant. A major issue in distributed training is the limited communication bandwidth between contributing nodes o…

Improved self-supervised learning on ImageNet achieves top-1 accuracy of 77.1%.

Discovering oral cavity cancer (OCC) at an early stage is an effective way to increase patient survival rate. However, current initial screening process is done manually and is expensive for the average individual, especially in developing countries worldwide. This problem is further compounded due to the lack of speci…

A new method quantifies feature-map discriminativeness for efficient pruning of deep neural networks.

Improved neural quantization reduces accuracy loss to less than 1% with 4-bit weights.

Adversarial perturbations are more effective in Y-channel of YCbCr color space.

Weather is an important factor affecting transportation and road safety. In this paper, we leverage state-of-the-art convolutional neural networks in labelling images taken by street and highway cameras located across across North America. Road camera snapshots were used in experiments with multiple deep learning frame…