Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A n…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “neural implementation”
This research explores complex-valued neural networks and their implementation.
problem The challenges of implementing complex-valued neural networks and their potential for non-complex data.
method Detailed theory and implementation of CVNN, including Wirtinger calculus, complex backpropagation, and modules like complex layers and activation functions. Python implementation using cvnn toolbox.
result Demonstrates the potential of CVNN for non-complex data through simulations.
Gradient Boosting Decision Tree (GBDT) are popular machine learning algorithms with implementations such as LightGBM and in popular machine learning toolkits like Scikit-Learn. Many implementations can only produce trees in an offline manner and in a greedy manner. We explore ways to convert existing GBDT implementatio…
A neural-network-based approach is presented to efficiently implement digital backpropagation (DBP). For a 32x100 km fiber-optic link, the resulting "learned" DBP significantly reduces the complexity compared to conventional DBP implementations.
Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4mlcs.LG
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
New proof shows neural networks can represent all multivariate functions.
problem Representing all multivariate functions with neural networks.
method Proved that three-layer neural networks can represent both continuous and discontinuous functions.
result Three-layer neural networks can represent all multivariate functions, including discontinuous ones.
Recursive neural networks have widely been used by researchers to handle applications with recursively or hierarchically structured data. However, embedded control flow deep learning frameworks such as TensorFlow, Theano, Caffe2, and MXNet fail to efficiently represent and execute such neural networks, due to lack of s…
NAIS co-design boosts AI model and implementation efficiency.
problem Efficient design of high-quality AI models and their implementations.
method Simultaneous DNN/implementation co-design methodology (NAIS).
result Significant reduction in search cost while maintaining solution quality.
Convolutional neural networks have recently achieved significant breakthroughs in various image classification tasks. However, they are computationally expensive,which can make their feasible mplementation on embedded and low-power devices difficult. In this paper convolutional neural network binarization is implemente…
Paper presents efficient algorithms for convolutional neural networks using Winograd minimal filtering.
problem Resource-efficient implementation of convolutional neural networks.
method Winograd minimal filtering trick applied to M-tap filters (M=3,5,7,9,11) for parallel hardware implementation.
result Approximately 30% reduction in multipliers for fully parallel hardware implementation.
We derive scaling laws for optimizing neural networks in hardware.
problem Optimizing the large parameter space of neural networks in hardware.
method Analytical derivation of scaling laws for Coordinate Descent optimization.
result Convergence is exponential and scales linearly with the number of neurons.
Large-scale deep neural networks are both memory intensive and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated in both industry and academia. Specific forms of binary neural networks (BNNs) and sto…
In recent years, deep neural networks have revolutionized many application domains of machine learning and are key components of many critical decision or predictive processes. Therefore, it is crucial that domain specialists can understand and analyze actions and pre- dictions, even of the most complex neural network …
PoET-BiN reduces power consumption in neural networks on embedded devices.
problem Power inefficiency in neural network implementations on embedded platforms.
method Look-Up Table based implementation with a modified Decision Tree approach.
result Near state-of-the-art results with up to 6 orders of magnitude energy reduction.
SurvBeNIM explains survival models using neural networks.
problem Explaining predictions of survival models.
method Extends Beran estimator with neural importance functions.
result Improved explanation of survival models compared to existing methods.
Bayesian Neural Networks improve uncertainty reasoning in NNs.
problem Frequentist implementation of NNs cannot reason about uncertainty in predictions.
method Introduces Bayesian Neural Networks and compares approximate inference methods.
result Future research can improve on current methods of inference.
AWDO trains neural networks for digit classification.
problem Training feedforward neural networks.
method Adaptive Wind Driven Optimization (AWDO).
result AWDO outperforms steepest descent method in digit classification.
Biological neural network mimics CCA for multi-channel data.
problem Implementing CCA in a biologically plausible neural network.
method Derive an online CCA algorithm with local synaptic updates for multi-compartmental neurons.
result The derived neural network architecture and synaptic updates resemble cortical pyramidal neuron behavior.
We address the problem of verifying neural-based perception systems implemented by convolutional neural networks. We define a notion of local robustness based on affine and photometric transformations. We show the notion cannot be captured by previously employed notions of robustness. The method proposed is based on re…
A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraintscs.LG
Simplifies neural network constraints with computationally efficient method.
problem Implementing hard output constraints in neural networks.
method Additional neural network layer for output constraints.
result Computational simplicity with complexity O(n*m) for linear constraints.
Autoencoder neural network is implemented to estimate the missing data. Genetic algorithm is implemented for network optimization and estimating the missing data. Missing data is treated as Missing At Random mechanism by implementing maximum likelihood algorithm. The network performance is determined by calculating the…
Blog post discusses various implementations of Fisher Information for EWC in continual learning.
problem Improving Elastic Weight Consolidation (EWC) results by optimizing Fisher Information computation.
method Empirically compares different implementations of Fisher Information for EWC.
result Many reported EWC results can be improved by changing Fisher Information computation methods.
This paper optimizes SMPC for neural network inference, reducing memory and time.
problem Memory and time constraints in secure neural network inference.
method Implemented ABY2.0 protocol, optimized memory usage, and used a helper node.
result MNIST inference reduced from 8.03 GB RAM and 200s to 0.2 GB RAM and 32s.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
New framework boosts neural network performance and resilience.
problem Susceptibility of compact neural network implementations to system disturbances.
method Realistic crossbar simulations and Mosaics framework to re-use synaptic connections.
result Compact neural networks are noise-immune and perform well under disturbances.
Tensor2Tensor is a library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model.
Analog deep learning shows promise but faces scalability challenges.
problem Bottlenecks in deep neural networks' calculation and optimization.
method Evaluation of eight analog deep learning methodologies.
result Analog deep learning has potential for consumer-level applications but scalability remains an issue.
Most neural network designs for FPGAs are inflexible. In this paper, we propose a flexible VHDL structure that would allow any neural network to be implemented on multiple FPGAs. Moreover, the VHDL structure allows for testing as well as training multiple neural networks. The VHDL design consists of multiple processor …
Improved neural network training with ADMM for hardware compatibility.
problem Training scalable and parallelizable neural networks.
method Alternating Direction Method of Multipliers (ADMM) combined with iterative least-square methods.
result Achieved significant accuracy improvements over SGD and Adam.
Paper improves speech separation by using deep neural networks for more accurate density priors.
problem Improving the accuracy of source priors for independent vector analysis in speech separation.
method Estimating the derivative of speech density using deep neural networks to optimize performance indices.
result Neural network density priors outperform previous ones in convergence speed and SIR.
With the rising popularity of machine learning and the ever increasing demand for computational power, there is a growing need for hardware optimized implementations of neural networks and other machine learning models. As the technology evolves, it is also plausible that machine learning or artificial intelligence wil…
Neural networks can separate non-separable data using feature maps.
problem Non-separable data in neural networks.
method Characterization of feedforward neural networks and use of feature maps.
result ReLU neural networks can separate concentric data.
NPE improves scalability and efficiency for ERGMs.
problem Scalability and efficiency issues in Bayesian ERGM estimation.
method Neural posterior estimation (NPE) for ERGMs using neural network density estimation.
result NPE provides more efficient and scalable inference for ERGMs.
We consider the use of look-up tables (LUT) to simplify the hardware implementation of a deep learning network for inferencing after weights have been successfully trained. The use of LUT replaces the matrix multiply and add operations with a small number of LUTs and addition operations resulting in a completely multip…
Model predicts competition between similar products in sales.
problem Predicting cannibalization between similar products in sales.
method Developed a neural network model that computes a 'competitiveness' function based on product features.
result The model outperforms traditional methods in predicting market share.
There is a recent interest in neural network (NN)-based communication algorithms which have shown to achieve (beyond) state-of-the-art performance for a variety of problems or lead to reduced implementation complexity. However, most work on this topic is simulation based and implementation on specialized hardware for f…
Blog post comparing neural network methods for causal inference.
problem Estimating heterogeneous treatment effects in causal inference.
method Developed and compared a fully connected neural network implementation of Bayesian Causal Forest.
result Improvements in performance in simulation settings.
jinns is a JAX library for physics-informed neural networks.
problem Physics-informed neural networks for forward and inverse problems.
method Physics-informed neural networks using JAX ecosystem.
result Efficient prototyping and extensions for real problems.
Survey on quantum computing and neural networks.
problem Understanding and comparing quantum computing and neural networks.
method Introduction to quantum computing concepts, explanation of quantum computing paradigms, and analysis of quantum neural networks.
result Current state-of-the-art in quantum neural networks.
PrototypeML simplifies neural network design and development.
problem Error-prone code and time-consuming model design.
method Visual interface for neural networks, abstracting PyTorch.
result Reduces model design and development time, easier debugging.
Comparative study of neural networks for short-term FOREX forecasting.
problem Simulating expert judgment in foreign exchange market forecasting.
method Implemented and compared LSTM and ANN architectures for short-term FOREX forecasting.
result ANN custom architecture outperforms LSTM in prediction quality and resource efficiency.
Analog BNNs perform similarly regardless of noise distribution shape.
problem Difficulty in precisely controlling noise distribution shape in analog devices.
method Used real device noise as the variational distribution in MFVI training.
result Predictive distributions converge to the same distribution regardless of noise shape.
We present a class of algorithms capable of directly training deep neural networks with respect to large families of task-specific performance measures such as the F-measure and the Kullback-Leibler divergence that are structured and non-decomposable. This presents a departure from standard deep learning techniques tha…
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
In this paper we study the real time deployment of deep learning algorithms in low resource computational environments. As the use case, we compare the accuracy and speed of neural networks for smile detection using different neural network architectures and their system level implementation on NVidia Jetson embedded p…
Spektral simplifies graph neural networks with TensorFlow and Keras.
problem Building graph neural networks efficiently.
method Open-source library implementing various graph neural network methods.
result Performance of implemented methods in node, graph, and graph regression tasks.
Bayesian neural networks tutorial via MCMC in Python.
problem Bayesian inference for parameter estimation and uncertainty quantification in deep learning models.
method MCMC sampling methods to implement Bayesian inference, including advanced proposal distributions.
result Challenges in sampling multi-modal posterior distributions for Bayesian neural networks.
Proposes a new method to prevent forgetting in neural networks.
problem Catastrophic forgetting in neural networks.
method Total absolute signal passing through each connection.
result Simpler and closer to brain processes.