arXiv research

DANCE optimizes neural network and accelerator design for faster, more efficient DNN execution.

Improved NiNo networks accelerate Adam training by up to 50%.

Optimized neural networks for Edge TPU achieve high accuracy in real-time image classification.

Graph neural networks speed up nonnegative matrix factorization.

Improved SDE-BNN model reduces NFEs and accelerates convergence.

Smartfluidnet accelerates Eulerian fluid simulation with neural networks.

Paper proposes NASAIC framework for co-designing neural architectures and heterogeneous ASICs.

A new method accelerates deep neural network training using minimal margin score.

DeepLight accelerates CTR predictions in ad serving by 46X.

Memory-augmented neural networks (MANNs) are designed for question-answering tasks. It is difficult to run a MANN effectively on accelerators designed for other neural networks (NNs), in particular on mobile devices, because MANNs require recurrent data paths and various types of operations related to external memory a…

Stacking improves deep neural network training efficiency.

In this paper we explore acceleration techniques for large scale nonconvex optimization problems with special focuses on deep neural networks. The extrapolation scheme is a classical approach for accelerating stochastic gradient descent for convex optimization, but it does not work well for nonconvex optimization typic…

Recent technological advances have proliferated the available computing power, memory, and speed of modern Central Processing Units (CPUs), Graphics Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs). Consequently, the performance and complexity of Artificial Neural Networks (ANNs) is burgeoning. While…

Survey of Graph Neural Networks for efficient computation.

Paper proposes a new binary quantization method for faster DNN inference.

Fast emulators built with neural search accelerate expensive scientific simulations.

GPU-accelerated particle methods outperform neural samplers in LFT benchmarks.

Survey on pruning CNN models to reduce size for edge devices.

Polyak's momentum accelerates training of neural networks.

We consider efficiency in the implementation of deep neural networks. Hardware accelerators are gaining interest as machine learning becomes one of the drivers of high-performance computing. In these accelerators, the directed graph describing a neural network can be implemented as a directed graph describing a Boolean…

Differentiable voxelization for 3D meshes with GPU acceleration.

New algorithms accelerate SVGD convergence using deep unfolding.

Paper develops MMOT framework for financial applications with neural acceleration.

GRU models with Adam optimizer outperform other combinations in stock market forecasting.

Two new algorithms improve neural architecture search efficiency.

Super-acceleration of gradient descent with momentum improves loss function minimization.

BONAS accelerates NAS while maintaining reliability.

GD and NAG accelerate matrix factorization and neural networks.

A common problem in training neural networks is the vanishing and/or exploding gradient problem which is more prominently seen in training of Recurrent Neural Networks (RNNs). Thus several algorithms have been proposed for training RNNs. This paper proposes a novel adaptive stochastic Nesterov accelerated quasiNewton (…

Accelerates sampling from Gibbs distributions using ARWP method.

New method accelerates Parallel Tempering using neural samplers.

The paper models and analyzes faults in TPU-based neural networks.

New insights into GNN optimization reveal skip connections and depth accelerate training.

Recently algorithms incorporating second order curvature information have become popular in training neural networks. The Nesterov's Accelerated Quasi-Newton (NAQ) method has shown to effectively accelerate the BFGS quasi-Newton method by incorporating the momentum term and Nesterov's accelerated gradient vector. A sto…

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and weight sharing, can reduce the number of parameters and increase processing speed dur…

In this paper, we propose a novel progressive parameter pruning method for Convolutional Neural Network acceleration, named Structured Probabilistic Pruning (SPP), which effectively prunes weights of convolutional layers in a probabilistic manner. Unlike existing deterministic pruning approaches, where unimportant weig…

Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent…

ALONE uses neural networks to accelerate cine MRI without needing ground truth.

Paper tackles NAS problem by modeling it as a sparse supernet.

New method accelerates neural network training by focusing on flat directions.

The Regularized Nonlinear Acceleration (RNA) algorithm is an acceleration method capable of improving the rate of convergence of many optimization schemes such as gradient descend, SAGA or SVRG. Until now, its analysis is limited to convex problems, but empirical observations shows that RNA may be extended to wider set…

Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as w…

The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and the design of accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition…

New attacks exploit neural network energy and latency, increasing costs by 10-200x.

This paper speeds up simulations of hypersonic reentry by combining traditional and neural methods.

Interneurons improve learning in neural networks by accelerating convergence.

Paper proposes a faster method for fuzzy neural networks by removing unsuitable hyperboxes.

Paper tackles noisy neural networks and proposes a method to enhance their robustness.