arXiv research

We provide a proof of backpropagation algorithm in matrix notation.

In this paper, deep neural network (DNN) is utilized to improve the belief propagation (BP) detection for massive multiple-input multiple-output (MIMO) systems. A neural network architecture suitable for detection task is firstly introduced by unfolding BP algorithms. DNN MIMO detectors are then proposed based on two m…

Study compares RNN, LSTM, and BP neural networks for forex rate prediction.

Existing methods for arterial blood pressure (BP) estimation directly map the input physiological signals to output BP values without explicitly modeling the underlying temporal dependencies in BP dynamics. As a result, these models suffer from accuracy decay over a long time and thus require frequent calibration. In t…

Modified BP attribution methods often ignore later layers' information, leading to misleading explanations.

New algorithms improve binary neural network configurations.

We prove that the global minimum of the backpropagation (BP) training problem of neural networks with an arbitrary nonlinear activation is given by the ridgelet transform. A series of computational experiments show that there exists an interesting similarity between the scatter plot of hidden parameters in a shallow ne…

The paper improves DFA for CNN and RNN training to match BP accuracy.

BPNNs learn to solve combinatorial problems faster and more accurately.

There were many algorithms to substitute the back-propagation (BP) in the deep neural network (DNN) training. However, they could not become popular because their training accuracy and the computational efficiency were worse than BP. One of them was direct feedback alignment (DFA), but it showed low training performanc…

Due to the extremely volatile nature of financial markets, it is commonly accepted that stock price prediction is a task full of challenge. However in order to make profits or understand the essence of equity market, numerous market participants or researchers try to forecast stock price using various statistical, econ…

BP-DIP combines DIP and backprojection for image restoration.

Predictive coding networks use inference learning for efficient AI modeling.

Proposes a semi-implicit back propagation method for neural networks.

Adapts BP-based algorithms for deep learning, improving performance and accuracy.

A neural network model minimizes region-based free energy for faster inference in MRFs.

Bio-inspired neural networks use predictive coding for efficient weight updates.

Recent changes to greenhouse gas emission policies are catalyzing the electric vehicle (EV) market making it readily accessible to consumers. While there are challenges that arise with dense deployment of EVs, one of the major future concerns is cyber security threat. In this paper, cyber security threats in the form o…

The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to 2nd order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear qu…

Three-layer networks learn complex hierarchical polynomials of multiple nonlinear features.

Study uses neural networks to predict credit risk in banks.

Layer-wise networks have a closed-form solution and a stopping criterion.

Layer-wise networks have a closed-form solution and a stopping criterion.

Inspired by recent advances in deep learning, we propose a novel iterative BP-CNN architecture for channel decoding under correlated noise. This architecture concatenates a trained convolutional neural network (CNN) with a standard belief-propagation (BP) decoder. The standard BP decoder is used to estimate the coded b…

Improved backpropagation with consequentialism weight updates for neural networks.

In an era when the performance of a single compute device plateaus, software must be designed to scale on massively parallel systems for better runtime performance. However, in the context of training deep learning models, the popular back-propagation (BP) algorithm imposes a strong sequential dependency in the process…

The backpropagation algorithm for calculating gradients has been widely used in computation of weights for deep neural networks (DNNs). This method requires derivatives of objective functions and has some difficulties finding appropriate parameters such as learning rate. In this paper, we propose a novel approach for c…

Unified deep learning from noisy crowds using BP and MF.

Theoretical framework for target propagation shows differences from backpropagation.

BP, a method for sparse recovery, shows generalization error decreases with more features.

Variational quantum computing faces a flat optimization landscape problem.

EP algorithm improved for CNNs and real-time learning.

Stochastic variance-reduced gradient (SVRG) is a classical optimization method. Although it is theoretically proved to have better convergence performance than stochastic gradient descent (SGD), the generalization performance of SVRG remains open. In this paper we investigate the effects of some training techniques, mi…

The backpropagation (BP) algorithm is often thought to be biologically implausible in the brain. One of the main reasons is that BP requires symmetric weight matrices in the feedforward and feedback pathways. To address this "weight transport problem" (Grossberg, 1987), two more biologically plausible algorithms, propo…

This paper proposes an in-depth re-thinking of neural computation that parallels apparently unrelated laws of physics, that are formulated in the variational framework of the least action principle. The theory holds for neural networks that are also based on any digraph, and the resulting computational scheme exhibits …

Study forecasts aortic pressure with deep learning models.

We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in \cite{bafna2018thwarting} to defend neural networks against $\ell_0$-norm, $\ell_2$-norm, and $\ell_{\infty}$-norm attacks. Our results are general as they can be applied to most unitary tr…

Max-product Belief Propagation (BP) is a popular message-passing algorithm for computing a Maximum-A-Posteriori (MAP) assignment over a distribution represented by a Graphical Model (GM). It has been shown that BP can solve a number of combinatorial optimization problems including minimum weight matching, shortest path…

Enhances inference of spreading processes using neural-network priors.

Backpropagation-free RL method trains layers using local signals.

New $\alpha$-BP algorithm improves belief propagation for graphs with loops.

The backpropagation of error algorithm (BP) is impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain might learn across multiple layers, and hence how it might approximate BP. As of yet, none of these prop…

Gaussian belief propagation (BP) has been widely used for distributed inference in large-scale networks such as the smart grid, sensor networks, and social networks, where local measurements/observations are scattered over a wide geographical area. One particular case is when two neighboring agents share a common obser…

Bayesian Power Steering fine-tunes large diffusion models for domain adaptation.

Finding the most probable assignment (MAP) in a general graphical model is known to be NP hard but good approximations have been attained with max-product belief propagation (BP) and its variants. In particular, it is known that using BP on a single-cycle graph or tree reweighted BP on an arbitrary graph will give the …

New cobordism invariants derived from BPS q-series.

Semi-supervised anomaly detection is an approach to identify anomalies by learning the distribution of normal data. Backpropagation neural networks (i.e., BP-NNs) based approaches have recently drawn attention because of their good generalization capability. In a typical situation, BP-NN-based models are iteratively op…

In manufacture, steel and other metals are mainly cut and shaped during the fabrication process by computer numerical control (CNC) machines. To keep high productivity and efficiency of the fabrication process, engineers need to monitor the real-time process of CNC machines, and the lifetime management of machine tools…