Paper reduces AI complexity with pre-defined sparsity and hardware acceleration.
problem Reduction of computational and storage complexity in neural networks.
method Pre-defined sparsity and hardware acceleration architecture.
result Significant reduction in storage and computational complexity (5X+ reduction) without significant performance loss.
Researchers show NN-based communication algorithms can be implemented on hardware without significant performance loss.
problem Reducing complexity and improving performance of NN-based communication algorithms for practical hardware implementation.
method Implementation of NN-based algorithms in fixed-point arithmetic with quantized weights on specialized hardware (FPGAs, ASICs).
result It is possible to implement NN-based algorithms in fixed-point arithmetic with quantized weights on hardware without significant performance loss.
Graph neural networks improve charged particle tracking on FPGAs.
problem Charged particle trajectory determination in high interaction density conditions.
method Graph neural networks (GNNs) embedded in tracker data as graphs, classifying edges as track segments.
result GNNs implemented on FPGAs for charged particle tracking, enabling future HL-LHC experiments.
A hardware-based reservoir computing system predicts time series with high speed and accuracy.
problem Processing time-dependent signals with high speed and accuracy.
method A hardware-based reservoir computing system using a field-programmable gate array (FPGA) for both the reservoir and output layers.
result Achieves comparable accuracy to software approaches but with a superior real-time prediction rate up to 160 MHz.
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
Accelerator boosts energy efficiency for MANNs on FPGAs.
problem Efficiently running MANNs on accelerators designed for other NNs.
method Data flow architecture, inference thresholding.
result Higher energy efficiency compared to NVIDIA GPU.
Real-time semantic segmentation for autonomous vehicles on FPGA reduces latency and power consumption.
problem Efficient real-time semantic segmentation for autonomous vehicles.
method Compressed ENet architecture, FPGA deployment, batch processing, filter reduction, quantization-aware training.
result Reduced latency to 3 ms per image with batch size of ten and 40% resource utilization.
New framework reduces cost of financial option pricing simulations on FPGAs.
problem Efficiently simulate financial option pricing with reduced computational cost.
method Nested MLMC framework with low precision calculations on FPGAs.
result Higher computational savings compared to existing mixed-precision MLMC frameworks.
LeFlow enables quick FPGA synthesis from Tensorflow models.
problem Manual translation of Tensorflow models to FPGA RTL is time-consuming and requires expertise.
method Uses XLA compiler to emit synthesizable LLVM code from Tensorflow specifications, which is then synthesized.
result Allows users to generate Deep Neural Networks with just a few lines of Python code.
Kolmogorov-Arnold Networks enable ultrafast online learning with fixed-point quantization.
problem Efficient online learning for high-frequency systems with strict memory constraints.
method Fixed-point online training on FPGAs exploiting B-spline locality in KANs.
result Kolmogorov-Arnold Networks are more efficient and expressive than MLPs for low-latency tasks.
Efficient FPGA dropout algorithm reduces memory usage.
problem Overfitting in deep neural networks.
method Hardware-oriented dropout algorithm for FPGA implementation.
result Significant resource reduction in FPGA implementation.
In this work, we design a machine learning based method, online adaptive primal support vector regression (SVR), to model the implied volatility surface (IVS). The algorithm proposed is the first derivation and implementation of an online primal kernel SVR. It features enhancements that allow efficient online adaptive …
PolyLUT uses polynomials to reduce FPGA latency.
problem Reducing latency in FPGA-based neural network inference.
method Training neural networks using multivariate polynomials as basic building blocks.
result Achieved significant latency and area improvements.
New approach uses Boolean circuits to optimize neural networks.
problem Improving efficiency of neural network implementations on hardware accelerators.
method Formalized neural networks as Boolean circuits, showing binarized networks are functionally complete.
result Binarized neural networks are functionally complete, suggesting new possibilities for neural network accelerators.
Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and performance for hardware implementations of deep neural networks (DNNs). An algorithm-hardware co-optimization framework is developed, which i…
PoET-BiN reduces power consumption in neural networks on embedded devices.
problem Power inefficiency in neural network implementations on embedded platforms.
method Look-Up Table based implementation with a modified Decision Tree approach.
result Near state-of-the-art results with up to 6 orders of magnitude energy reduction.
A new method improves RVFL networks for resource-efficient machine learning.
problem Deploying machine learning on edge devices with limited resources.
method Density encoding and hyperdimensional computing operations.
result The proposed method achieves higher accuracy and lower energy consumption.
Trading system uses NP-hard optimization to select stocks for high Sharpe ratio trading.
problem Finding profitable, uncorrelated stocks for high Sharpe ratio trading.
method NP-hard combinatorial optimization using Ising machine and simulated bifurcation algorithm.
result Trading strategy with FPGA-based system achieves 164 μs response latency.
FPGAs accelerate DNNs with binarized weights, improving power and speed.
problem Power and speed limitations of GPU-accelerated DNNs.
method Binarized neural networks on FPGAs using OpenCL.
result Near state-of-the-art performance with >16x power savings.
New methodology optimizes complex systems design trade-offs.
problem Complex multi-objective optimization with unknown feasibility constraints.
method Introduces HyperMapper 2.0 for multi-objective optimization, handling categorical/ordinal variables.
result HyperMapper 2.0 provides better Pareto fronts and 8x improvement in sampling budget.
Paper proposes NASAIC framework for co-designing neural architectures and heterogeneous ASICs.
problem Designing efficient neural architectures and ASICs for multiple tasks.
method Build ASIC templates and propose NASAIC framework for simultaneous design of architectures and ASICs.
result NASAIC ensures design specifications and maximizes accuracy with minimal performance loss.
The rapid growth of data size and accessibility in recent years has instigated a shift of philosophy in algorithm design for artificial intelligence. Instead of engineering algorithms by hand, the ability to learn composable systems automatically from massive amounts of data has led to ground-breaking performance in im…
Channel gating reduces CNN computation cost by skipping ineffective feature regions.
problem Reducing computation cost in CNNs while maintaining accuracy.
method Dynamic, fine-grained pruning scheme that identifies and skips computation on ineffective feature regions.
result 2.7-8.0x reduction in FLOPs and 2.0-4.4x reduction in memory accesses with minimal accuracy loss.
NeuraLUT maps neural networks to lookup tables, reducing latency and improving expressivity.
problem Reducing latency in deep neural networks for FPGA accelerators.
method Mapping entire sub-networks to a single lookup table, introducing skip connections.
result Up to 4.3x lower latency for the same accuracy.
SNRA combines power-efficient probabilistic and deterministic computing for deep belief networks.
problem Efficiently training and evaluating deep belief networks with low power consumption.
method Developed a spintronic neuromorphic reconfigurable array (SNRA) for in-circuit training and evaluation of deep belief networks (DBNs). Used probabilistic spin logic devices and a four-state finite state machine for unsupervised training.
result SNRA achieves more than 80% reduction in combined dynamic and static power dissipation compared to SRAM-based configurable fabrics.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
The paper studies 4-qubit Clifford states and their properties.
problem Understanding the set and properties of 4-qubit Clifford states.
method Analyzing the 293760 4-qubit Clifford states, splitting them into 18 groups, and studying the action of CNOT gates and local gates.
result There are 293760 4-qubit Clifford states with specific entanglement entropies, and any pair can be connected with local gates and at most 3 CNOT gates.
Analog arrays speed up ConvNets by parallelizing kernel matrix training.
problem Early ConvNets struggle with analog arrays due to small kernel matrices.
method Replicate kernel matrix on multiple analog arrays, training in parallel.
result Analog arrays achieve high acceleration factors (16-128) for ConvNets.
Improved logistic MoE with sigmoid gate shows better sample efficiency.
problem Improving sample efficiency in logistic MoE models.
method Comprehensive analysis of multinomial logistic MoE with modified sigmoid gate, incorporating temperature parameter and using Euclidean score.
result The sigmoid gate leads to lower sample complexity than softmax gate for both parameter and expert estimation.
New method prepares 3-qubit states using local gates and controlled-Z gates.
problem Preparation of 3-qubit states using quantum gates.
method Uses Ry(θ) gates and controlled-Z gates, with an optimal number of controlled-Z gates. result Optimal number of controlled-Z gates for preparing 3-qubit states is four. System for automatic differentiation in a functional array-processing language.
problem Efficient automatic differentiation in functional array languages.
method Automatic differentiation in a higher-order functional array-processing language with source-to-source support and global optimizations.
result The system outperforms state-of-the-art tools on machine learning and computer vision benchmarks.
Method provides statistical guarantees for identifying subgroups in ML studies.
problem Bias and noise in estimating conditional average treatment effects (CATE).
method Develops uniform confidence bands (GATES) for estimating group average treatment effects (GATEs).
result Identifies subgroups with statistical guarantees, regardless of effect size.
Improves magnetic field mapping using an array of magnetometers with noisy input.
problem Improving magnetic field maps in indoor environments with noisy magnetometer data.
method Uses Gaussian process regression with an array of magnetometers, incorporating known array positions and relative magnetometer locations.
result The method produces higher quality magnetic field maps compared to using a single magnetometer.
Sigmoid gating is more sample efficient than softmax in mixture of experts.
problem Softmax gating leads to unnecessary competition among experts, causing representation collapse.
method Theoretical analysis of a regression framework with mixture of experts, identifying identifiability conditions and convergence rates.
result Sigmoid gating requires fewer samples to achieve the same expert estimation error as softmax gating.
In quantum computation, series of quantum gates have to be arranged in a predefined sequence that led to a quantum circuit in order to solve a particular problem. What if the sequence of quantum gates is known but both the problem to be solved and the outcome of the so defined quantum circuit remain in the shadow? This…
Simpler proof for non-basic sets in 2D.
problem Proving non-basic sets in 2D.
method Defining Sternfeld arrays and proving non-basic sets.
result Simpler proof of non-basic sets in 2D.
Proposes a new LSTM gate structure using bivariate Beta distribution.
problem Inflexibility of sigmoid gates in modeling multi-modality and skewness, and lack of modeling correlation between gates.
method Introduces a bivariate Beta distribution gate structure within LSTM cells.
result Empirically shows higher gradient values and improved model performance.
Paper improves DOA estimation in sparse arrays using Siamese neural networks.
problem Challenges in DOA estimation with limited snapshots in sparse linear arrays.
method Introduces a Siamese neural network with a sparse augmentation layer for enhanced signal feature embedding.
result Demonstrates improved DOA estimation accuracy in sparse arrays.
Direction of arrival (DoA) estimation of targets improves with the number of elements employed by a phased array radar antenna. Since larger arrays have high associated cost, area and computational load, there is recent interest in thinning the antenna arrays without loss of far-field DoA accuracy. In this context, a c…
Bayesian method sparsifies gated RNNs, improving speed and interpretability.
problem Sparsifying neural networks to reduce complexity and improve performance.
method Bayesian approach to sparsify weights, neurons, and gates in LSTM architectures.
result Sparsified gated RNNs speed up forward pass and improve compression.
We sparsify gated RNNs by simplifying their structure.
problem Improving efficiency of RNNs by reducing their complexity.
method Adjust existing sparsification techniques to gated RNNs, sparsifying preactivations of gates.
result Simplified LSTM structure improves model performance and efficiency.
Gated attention improves model curvature, enhancing performance on nonlinear tasks.
problem Understanding the geometric implications of gating in attention mechanisms.
method Modeling attention outputs as Gaussian distributions and analyzing Fisher--Rao geometry.
result Gated attention enables non-flat geometries, including positively curved manifolds.
Improved RNNs with flexible gates using kernel activation functions.
problem Modeling long-term dependencies in sequential data.
method Designed a more flexible architecture with adaptable parameters using kernel activation functions.
result Improved accuracy with negligible computational cost and speed-up in training iterations.
Gated attention improves performance by using a hierarchical mixture of experts.
problem Improving performance of self-attention mechanisms in Transformers.
method Rigorously show that gated attention can be modeled as a hierarchical mixture of experts, providing a theoretical justification for its benefits.
result Gated attention is more sample-efficient than multi-head self-attention, requiring fewer data points to achieve the same estimation error.
Novel CNN array for sign language recognition using wearable IMUs.
problem Efficiently recognizing sign language from wearable IMU signals.
method Two-dimensional Convolutional Neural Network array architecture for Indian sign language recognition.
result Peak classification accuracies of 94.20% for general sentences and 95.00% for interrogative sentences achieved.
The CHAMPION study clusters multi-dimensional accelerometer data to understand health links.
problem Clustering multi-dimensional data from pediatric longitudinal studies.
method Developed a finite mixture of multidimensional arrays model for clustering 4-dimensional accelerometer data.
result Demonstrated the feasibility and utility of clustering higher order data.
Given the success of the gated recurrent unit, a natural question is whether all the gates of the long short-term memory (LSTM) network are necessary. Previous research has shown that the forget gate is one of the most important gates in the LSTM. Here we show that a forget-gate-only version of the LSTM with chrono-ini…
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
problem Training challenges in RNNs due to exploding or vanishing gradients.
method Random matrix theory and mean-field theory applied to GRUs and LSTMs.
result Gates in GRUs and LSTMs lead to accumulation of slow modes and control phase-space complexity.