Review of efficient neural networks for TinyML on resource-constrained devices.
problem Resource constraints on ultra-low power MCUs for deep learning models.
method Model compression, quantization, low-rank factorization, model pruning, hardware acceleration, algorithm-architecture co-design.
result Optimized neural network architectures for minimal resource utilization on MCUs.
FANN-on-MCU enables efficient neural network inference on IoT devices.
problem Energy-efficient neural network inference on resource-constrained IoT devices.
method Open-source toolkit for ARM Cortex-M and RISC-V microcontrollers.
result Efficient neural network execution with low latency and power consumption.
SEFR is a fast, energy-efficient classifier for ultra-low power devices.
problem Running machine learning on battery-powered devices is challenging due to time and energy constraints.
method SEFR is an ultra-low power classifier with linear time complexity for training and testing.
result SEFR is 63 times faster and 70 times more energy efficient than state-of-the-art classifiers.
Designing deep learning models for highly-constrained hardware would allow imbuing many edge devices with intelligence. Microcontrollers (MCUs) are an attractive platform for building smart devices due to their low cost, wide availability, and modest power usage. However, they lack the computational resources to run ne…
Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.
problem Memory limitations prevent accurate deployment of DNN models on tiny MCUs.
method Automated mixed-precision quantization using Reinforcement Learning for MCU constraints.
result Mixed-precision models achieve high accuracy with uniform quantization policies.
MCU-Net combines U-Net and Monte Carlo Dropout for uncertainty in medical image segmentation.
problem Lack of uncertainty representation in deep learning methods for patient-centered healthcare decisions.
method MCU-Net framework using U-Net and Monte Carlo Dropout with four uncertainty metrics.
result MCU-Net maximizes automated performance and refers truly uncertain cases.
This work optimizes DNN inference for energy-harvesting devices by compressing and selectively executing neural network exits.
problem Inference delays and energy inefficiency in energy-harvesting devices.
method Developed a power trace-aware and exit-guided network compression algorithm for multi-exit neural networks.
result Superior accuracy and reduced latency compared to state-of-the-art techniques.
There is growing interest in being able to run neural networks on sensors, wearables and internet-of-things (IoT) devices. However, the computational demands of neural networks make them difficult to deploy on resource-constrained edge devices. To meet this need, our work introduces a new recurrent unit architecture th…
This paper tackles energy-efficient machine learning on low-power devices.
problem Energy consumption in machine learning due to data communication.
method Dynamic averaging for integer exponential families on low-power processors.
result Achieves comparable model quality with significantly less communication and energy.
Computer vision performances have been significantly improved in recent years by Convolutional Neural Networks(CNN). Currently, applications using CNN algorithms are deployed mainly on general purpose hardwares, such as CPUs, GPUs or FPGAs. However, power consumption, speed, accuracy, memory footprint, and die size sho…
Implantable, closed-loop devices for automated early detection and stimulation of epileptic seizures are promising treatment options for patients with severe epilepsy that cannot be treated with traditional means. Most approaches for early seizure detection in the literature are, however, not optimized for implementati…
In natural hazard warning systems fast decision making is vital to avoid catastrophes. Decision making at the edge of a wireless sensor network promises fast response times but is limited by the availability of energy, data transfer speed, processing and memory constraints. In this work we present a realization of a wi…
Automated tool reduces FPGA inference latency to 5 μs for deep neural networks.
problem Deploying ultra low-latency, low-power deep neural networks on FPGAs.
method Extending hls4ml library, using model compression techniques like pruning and quantization-aware training.
result Achieved inference latency of 5 μs with 97% resource reduction.
PolyLUT uses polynomials to reduce FPGA latency.
problem Reducing latency in FPGA-based neural network inference.
method Training neural networks using multivariate polynomials as basic building blocks.
result Achieved significant latency and area improvements.
Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and F…
TinyLSTMs reduces speech enhancement model size and latency for hearing aids.
problem Large RNNs limit practical deployment in hearing aid hardware.
method Model compression techniques (pruning, integer quantization, state update skipping) for RNN speech enhancement.
result Reduction in model size and operations by 11.9imes and 2.9imes, respectively, without perceptual degradation. Intelligent transportation systems (ITSs) will be a major component of tomorrow's smart cities. However, realizing the true potential of ITSs requires ultra-low latency and reliable data analytics solutions that can combine, in real-time, a heterogeneous mix of data stemming from the ITS network and its environment. Su…
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to an…
Improves bit error tolerance in RRAM-based BNNs without overfitting.
problem Bit errors in RRAM-based BNNs reduce accuracy and overfit to training error rates.
method Proposes straight-through gradient approximation and a novel regularizer.
result Improves BNNs' robustness to bit errors without overfitting.
New framework reduces LLM complexity by directly finetuning in Boolean domain.
problem Reducing the complexity of large language models (LLMs) while maintaining performance.
method Proposes a novel framework using multi-kernel Boolean parameters for direct finetuning in the Boolean domain.
result Significantly reduces complexity during both finetuning and inference, outperforming recent techniques.
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
problem Reducing multiplication complexity in low-resolution neural networks.
method Adapting neural networks to use low-resolution (8-bit) additions in accumulators, with a cyclic activation layer and overflow penalty regularizer.
result Achieves comparable classification accuracy to 32-bit counterparts using low-resolution additions.
GPU-accelerates multiuser detection for 5G URLLC systems.
problem Efficiently detecting payloads in OFDM frames with ultra-low latency.
method Implemented partially linear multiuser detection in RKHSs on a GPU-accelerated platform.
result Sub-millisecond latency detection in 5G URLLC systems.
Characterizing statistical properties of solutions of inverse problems is essential for decision making. Bayesian inversion offers a tractable framework for this purpose, but current approaches are computationally unfeasible for most realistic imaging applications in the clinic. We introduce two novel deep learning bas…
A major challenge in computed tomography (CT) is to reduce X-ray dose to a low or even ultra-low level while maintaining the high quality of reconstructed images. We propose a new method for CT reconstruction that combines penalized weighted-least squares reconstruction (PWLS) with regularization based on a sparsifying…
Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference…
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
New pension design reduces volatility without guarantees.
problem Pension volatility and guarantees issues.
method Split premium, invest in funds, redistribute to smooth volatility.
result Maximizes total accumulated capital at retirement.
In this paper, we consider the problem of fast and efficient indexing techniques for sequences evolving in non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search, and recognition in very high dimensional spaces. The problem is ma…
A new high-frequency market making strategy using Deep Hawkes process.
problem Optimizing high-frequency trading in volatile markets.
method Developed a Deep Hawkes process to model order arrivals and their effects on the limit order book.
result The new strategy outperforms traditional methods in market making.
STARK improves denoising of low-depth spatial transcriptomics images.
problem Denoising spatial transcriptomics images at ultra-low sequencing depths.
method Adaptive regularization with kernel ridge regression and graph Laplacian.
result STARK optimizes denoising performance over competing methods.
Generative compression technique reduces neural network size and improves performance on microcontrollers.
problem Memory constraints on microcontrollers limit the size of neural networks, especially for 1x1 pointwise (PW) mixers.
method HYPER-TINYPW uses a shared micro-MLP to generate PW kernels from tiny per-layer codes, reducing memory usage.
result HYPER-TINYPW achieves comparable performance to larger models while being significantly smaller (225 kB vs 1.4 MB).
Framework predicts and prepares for rain-induced microwave link attenuation.
problem Severe signal attenuation due to weather conditions degrades network performance.
method Predictive Network Reconfiguration (PNR) framework using LSTM for attenuation prediction and MSNR for dynamic routing.
result Framework improves network utilization by more than 200% compared to reactive algorithms.
New algorithm extracts device profiles for short-term power predictions in commercial buildings.
problem Short-term power prediction in commercial buildings with high accuracy.
method Unsupervised extraction of device profiles from aggregate power measurements, disaggregation using particle swarm optimization, and state changes forecast by artificial neural networks.
result Developed approach outperforms existing methods with high accuracy.
Power quandles improve group invariants and allow group presentations.
problem Understanding and classifying groups using algebraic structures.
method Introducing power quandles, which retain conjugation and power maps, and showing their effectiveness in group theory.
result Power quandles determine central quotients and centers of groups, and can approximate any group.
Novel power transform unifies various mathematical functions.
problem Normalizing and standardizing datasets.
method Presented a novel power transform.
result Unified various mathematical functions.
Study of metrics on positive-definite matrices from power potential, linking to power means.
problem Understanding metrics on positive-definite matrices derived from power potential.
method Explicit expressions for geodesics and distance function derived from Hessian of power potential.
result Geodesics and distance function converge to weighted matrix geometric mean as β tends to zero.
The paper establishes conditions for strict power concavity in convolutions.
problem Conditions for strict power concavity in convolutions.
method Analyzes sufficient conditions for strict parabolic power concavity of convolutions.
result Establishes sufficient conditions for strict power concavity of convolutions.
Power system studies require the topological structures of real-world power networks; however, such data is confidential due to important security concerns. Thus, power grid synthesis (PGS), i.e., creating realistic power grids that imitate actual power networks, has gained significant attention. In this letter, we cas…
Power laws detected in financial data, modeled with random multipliers.
problem Detecting power laws in financial data.
method Investigated data from financial instruments, proposed a model based on sums of Maxwell-Boltzmann distributions with random multipliers.
result Detected power laws with various exponents in financial data, proposed a universal model.
Bayesian method models multivalued power data from wind farms.
problem Accurate modeling of power curves with multivalued relationships.
method Overlapping mixture of probabilistic regression models.
result Model accurately represents practical power data.
WindDragon forecasts wind power with deep learning.
problem Accurate short-term wind power forecasting is crucial for grid operation.
method Automated Deep Learning combined with Numerical Weather Predictions.
result Automated Deep Learning improves wind power forecasting accuracy.
Paper examines power consumption in neural networks using various activation functions.
problem Power consumption in machine learning models.
method Examines power consumption for different activation functions.
result Substantial differences in power consumption exist between activation functions.
Paper introduces reinforcement learning for managing power grids.
problem Balancing power flows and maintaining grid stability in real-time.
method Reinforcement Learning applied to power network operations.
result Demonstrates feasibility of machine learning in power grid management.
Study of origamis' singularities for groups of prime-power order.
problem Classifying singularities of origamis for groups of prime-power order.
method Geometric and group-theoretic ideas used to classify strata.
result Many groups of prime-power order have only one stratum, but some do not.
Photovoltaic systems have been widely deployed in recent times to meet the increased electricity demand as an environmental-friendly energy source. The major challenge for integrating photovoltaic systems in power systems is the unpredictability of the solar power generated. In this paper, we analyze the impact of havi…
Auto-regressive conditionally heteroskedastic (ARCH) family models are still used, by practitioners in business and economic policy making, as a conditional volatility forecasting models. Furthermore ARCH models still are attracting an interest of the researchers. In this contribution we consider the well known GARCH(1…
Survey on GNNs' power and limitations.
problem Theoretical limitations of GNNs.
method Comprehensive overview of GNNs and their variants.
result Provably powerful variants of GNNs.
GP CC-OPF solves uncertain power grid optimization with Gaussian Process.
problem Uncertainty in power grid operations due to high renewables integration.
method Data-driven Gaussian Process regression for solving non-convex CC-OPF problem.
result Effective economic dispatch optimization in uncertain power grids.