Paper evaluates security of deep learning from side-channel attacks.
problem Security of deep learning algorithms from side-channel attacks.
method Uses hardware performance counters to test for information leakage in CNNs.
result Shows alarms for potential data privacy breaches in CNNs.
Fuelled by increasing computer power and algorithmic advances, machine learning techniques have become powerful tools for finding patterns in data. Since quantum systems produce counter-intuitive patterns believed not to be efficiently produced by classical systems, it is reasonable to postulate that quantum computers …
BatchNorm helps train quantized networks by avoiding gradient explosion.
problem Training quantized neural networks is difficult due to gradient issues.
method Investigated the impact of BatchNorm on both full-precision and quantized networks.
result BatchNorm avoids gradient explosion in quantized networks, contrary to expectations.
Complex-valued neural networks improve wireless fingerprinting robustness.
problem Distinguish between devices sending the same message, robust against spoofing.
method Used complex-valued neural networks for supervised learning of fingerprints from wireless signals.
result Noise augmentation by adding white Gaussian noise leads to significant performance gains.
Paper proposes a faster method for evaluating DNN hardware and software designs.
problem Reducing time for evaluating different DNN hardware and software designs.
method Using virtual hardware models to estimate DNN performance at the concept phase.
result Up to 92% accuracy in predicting DNN inference processing time.
Exploration is a fundamental aspect of Reinforcement Learning, typically implemented using stochastic action-selection. Exploration, however, can be more efficient if directed toward gaining new world knowledge. Visit-counters have been proven useful both in practice and in theory for directed exploration. However, a m…
DANCE optimizes neural network and accelerator design for faster, more efficient DNN execution.
problem Challenges in optimizing neural network and accelerator design for efficient DNN execution.
method Differentiable approach to co-exploration of accelerator and network architecture design.
result Significantly shorter time to achieve superior accuracy and hardware cost metrics.
Design automation optimizes deep learning models for various hardware.
problem Designing efficient deep learning models requires balancing algorithm and hardware.
method Proposes design automation techniques for specialized neural networks, including auto pruning and quantization.
result Learning-based automation achieves superior performance and efficiency compared to human design.
Optimized neural networks for Edge TPU achieve high accuracy in real-time image classification.
problem Designing neural networks for hardware accelerators to achieve optimal performance.
method Hardware-aware neural architecture search and model customization for Edge TPU.
result Improved accuracy-latency tradeoff on Pixel 4's Edge TPU compared to existing models.
PABO optimizes DNN and hardware hyperparameters for efficient edge device acceleration.
problem Joint optimization of DNN accuracy and hardware cost for edge devices.
method Bayesian optimization with pseudo agent-based approach for memristive crossbar accelerators.
result PABO achieves significant speed-ups and superior performance compared to state-of-the-art methods.
Benchmark for DL inference on embedded HWAs, focusing on autonomous driving.
problem Lack of comprehensive benchmarks for DL hardware.
method Developed a benchmark for inference on embedded HWAs, focusing on autonomous driving. Proposed new granularity, benchmark procedures, and performance indicators.
result Identifies mismatches between HWAs and DL models.
Researchers show NN-based communication algorithms can be implemented on hardware without significant performance loss.
problem Reducing complexity and improving performance of NN-based communication algorithms for practical hardware implementation.
method Implementation of NN-based algorithms in fixed-point arithmetic with quantized weights on specialized hardware (FPGAs, ASICs).
result It is possible to implement NN-based algorithms in fixed-point arithmetic with quantized weights on hardware without significant performance loss.
New Ising models improve consensus clustering on specialized hardware.
problem Consensus clustering optimization problems.
method Formulated consensus clustering as Ising models and evaluated on specialized hardware.
result Our Ising models outperform existing techniques on consensus clustering.
Stealthy hardware Trojan exploits DLA architecture vulnerabilities.
problem Security of DLA deployed on hardware accelerators.
method Input Interception Attack (IIA) exploiting statistical properties of DLA outputs.
result Stealthy Trojan can trigger with some definiteness.
Study on pairwise counter-monotonicity, a type of negative dependence.
problem Understanding and quantifying extremal negative dependence structures.
method Established stochastic representation and invariance property; showed implications and connections.
result Pairwise counter-monotonicity implies negative association and joint mix dependence.
Novel NAS method balances performance and hardware metrics efficiently.
problem Challenging multi-objective optimization in neural architecture search.
method Parameterizes joint architectural distribution via hypernetwork conditioned on hardware features and preferences.
result Zero-shot transferability to new devices with representative and diverse architectures.
Method predicts hardware resource usage by control software with guaranteed linear convergence.
problem Predicting time-varying hardware resource availability in control software.
method Path structured multimarginal Schrödinger bridge (MSBP) for learning stochastic resource usage.
result Guaranteed linear convergence to accurate prediction of hardware resource utilization.
New method uses surrogate gradients to train efficient spiking networks on neuromorphic hardware.
problem Training high-performing spiking networks on analog neuromorphic hardware is challenging due to device mismatch and lack of efficient algorithms.
method Introduces a general in-the-loop learning framework based on surrogate gradients.
result Learning self-corrects for device mismatch, resulting in competitive spiking network performance.
A scalable DL benchmarking platform for evaluating and comparing models, frameworks, and hardware.
problem Lack of a uniform DL benchmarking platform for evaluating and comparing innovations.
method Identified 10 design features for a DL benchmarking platform, proposed MLModelScope, and implemented as an open-source project.
result Demonstrated how model, hardware, and framework selection affect accuracy and performance under different scenarios.
This paper characterizes deep learning models in Facebook's data centers and suggests optimizations.
problem Improving performance of deep learning models in data centers.
method Detailed characterizations, high performance optimizations, co-design suggestions.
result Need for better co-design of algorithms, numerics, and computing platforms.
In [1] Zawadoski introduces a banking network model in which the asset and counter-party risks are treated separately and the banks hedge their assets risks by appropriate OTC contracts. In his model, each bank has only two counter-party neighbors, a bank fails due to the counter-party risk only if at least one of its …
Binary and ternary weights simplify RNNs for mobile devices.
problem Complexity and memory intensity of RNNs on mobile devices.
method Learn binary and ternary weights during training.
result Significant memory saving and inference speedup on ASIC platform.
This paper highlights new opportunities for designing large-scale machine learning systems as a consequence of blurring traditional boundaries that have allowed algorithm designers and application-level practitioners to stay -- for the most part -- oblivious to the details of the underlying hardware-level implementatio…
MLPerf benchmarks ML inference systems across diverse hardware.
problem Challenges in evaluating ML inference systems due to variety and complexity.
method MLPerf prescribes a set of rules and best practices for comparability.
result First call for submissions yielded over 600 reproducible measurements.
VegasFlow accelerates complex simulations across various hardware platforms.
problem Complex calculations and simulations requiring high-dimensional integrals.
method Monte Carlo integration techniques using Vegas algorithm and TensorFlow.
result Significantly faster performance on various hardware platforms.
Paper presents efficient algorithms for convolutional neural networks using Winograd minimal filtering.
problem Resource-efficient implementation of convolutional neural networks.
method Winograd minimal filtering trick applied to M-tap filters (M=3,5,7,9,11) for parallel hardware implementation.
result Approximately 30% reduction in multipliers for fully parallel hardware implementation.
Recent breakthroughs in Deep Learning (DL) applications have made DL models a key component in almost every modern computing system. The increased popularity of DL applications deployed on a wide-spectrum of platforms have resulted in a plethora of design challenges related to the constraints introduced by the hardware…
CoCoPIE shows AI can run on regular devices without special hardware.
problem Real-time AI on mobile devices without special hardware.
method Compression-compiler co-design using CoCoPIE framework.
result Real-time AI applications on off-the-shelf mobile devices.
Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and performance for hardware implementations of deep neural networks (DNNs). An algorithm-hardware co-optimization framework is developed, which i…
Relay simplifies deep learning compilation across diverse hardware.
problem Challenges in extending DL frameworks to diverse models and hardware.
method Relay introduces a unified, expressive intermediate representation and extension mechanisms for hardware portability.
result Relay achieves competitive performance across various hardware platforms.
New activation networks improve model efficiency and performance.
problem Creating hardware-efficient deep learning models.
method Restructurable Activation Networks (RANs) with RAN-explicit and RAN-implicit methods.
result RANs achieve state-of-the-art results with improved hardware efficiency.
This paper proposes structurally sparse RNNs to reduce computational and memory costs.
problem Heavy computational and memory burden in fully connected RNNs.
method Study structurally sparse RNNs, reducing recurrent operations and weights.
result Structurally sparse RNNs achieve competitive performance with reduced costs.
Loihi neuromorphic chip outperforms conventional hardware in keyword spotting efficiency.
problem Benchmarking keyword spotting efficiency on neuromorphic hardware.
method Comparative analysis of a two-layer neural network trained to recognize a single phrase on Intel's Loihi neuromorphic chip and conventional hardware devices.
result Loihi outperforms conventional hardware on energy cost per inference for this keyword spotting application.
We introduce Dimple, a fully open-source API for probabilistic modeling. Dimple allows the user to specify probabilistic models in the form of graphical models, Bayesian networks, or factor graphs, and performs inference (by automatically deriving an inference engine from a variety of algorithms) on the model. Dimple a…
Packed-Ensembles improve uncertainty estimation in constrained hardware.
problem Hardware limitations restrict the size of ensembles and network capacity, degrading performance.
method Packed-Ensembles (PE) design and train lightweight structured ensembles by modulating encoding space and parallelizing into a single backbone.
result PE accurately preserves diversity and maintains performance on key metrics like accuracy, calibration, and out-of-distribution detection.
Self-supervised GAN prevents forgetting in sequential tasks.
problem Discriminator forgetting in GANs leads to training instability.
method Add self-supervision to the discriminator to maintain useful representations.
result Self-supervised GAN outperforms regular GANs in learning better representations.
Prognosticator improves performance in non-stationary MDPs.
problem Real-world applications often violate the stationary assumption in reinforcement learning.
method A policy gradient algorithm that maximizes a forecast of future performance.
result Prognosticator outperforms online adaptation techniques in non-stationary MDPs.
Bio-inspired neuromorphic hardware is a research direction to approach brain's computational power and energy efficiency. Spiking neural networks (SNN) encode information as sparsely distributed spike trains and employ spike-timing-dependent plasticity (STDP) mechanism for learning. Existing hardware implementations of…
TRASHFIRE improves model robustness by analyzing training rates and costs.
problem Understanding and predicting model robustness under adversarial conditions.
method Survival models, worst-case examples, cost-aware analysis.
result Deeper models offer marginal robustness gains due to inference time, not inherent robustness.
This paper analyzes mobile device training of deep learning models.
problem Performance characterization of training deep learning models on mobile devices.
method Experiments on NVIDIA TX2, benchmark suite, and tools for performance analysis.
result Interesting performance problems and opportunities revealed.
This paper shows how to calculate risk measures for sums of two counter-monotonic risks.
problem Calculating risk measures for sums of two counter-monotonic risks.
method Using a fixed distortion function and expressing the risk measure of a sum as the sum of two related measures of the marginals.
result The risk measure of a sum of two counter-monotonic risks can be expressed as the sum of two related distortion risk measures of the marginals.
We give a counter example to a conjecture of E. Bueler stating the equality between the DeRham cohomology of complete Riemannian manifold and a weighted L2 cohomology where the weight is the heat kernel.
Paper reduces AI complexity with pre-defined sparsity and hardware acceleration.
problem Reduction of computational and storage complexity in neural networks.
method Pre-defined sparsity and hardware acceleration architecture.
result Significant reduction in storage and computational complexity (5X+ reduction) without significant performance loss.
New attacks exploit neural network energy and latency, increasing costs by 10-200x.
problem Energy and latency vulnerabilities in neural networks.
method Crafted sponge examples to maximize energy and latency.
result Adversaries can increase energy consumption by 10-200x and delay decisions.
MLPerf benchmarks ML training to drive performance improvements.
problem Unique challenges in ML training benchmarks.
method Developed MLPerf to overcome ML training's specific challenges.
result Quantitatively evaluated MLPerf's effectiveness.
Improved neural network training with ADMM for hardware compatibility.
problem Training scalable and parallelizable neural networks.
method Alternating Direction Method of Multipliers (ADMM) combined with iterative least-square methods.
result Achieved significant accuracy improvements over SGD and Adam.
Deep learning has revolutionised many fields, but it is still challenging to transfer its success to small mobile robots with minimal hardware. Specifically, some work has been done to this effect in the RoboCup humanoid football domain, but results that are performant and efficient and still generally applicable outsi…
Specialized Deep Learning (DL) acceleration stacks, designed for a specific set of frameworks, model architectures, operators, and data types, offer the allure of high performance while sacrificing flexibility. Changes in algorithms, models, operators, or numerical systems threaten the viability of specialized hardware…