A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
GAIT-prop derives a biologically plausible learning rule from backpropagation.
New algorithm shows neural networks can learn without full backpropagation.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
Configuring deep Spiking Neural Networks (SNNs) is an exciting research avenue for low power spike event based computation. However, the spike generation function is non-differentiable and therefore not directly compatible with the standard error backpropagation algorithm. In this paper, we introduce a new general back…
A new model uses 'ghost units' to enable efficient backpropagation in deep neural networks.
Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile…
We introduce the chi-square test neural network: a single hidden layer backpropagation neural network using chi-square test theorem to redefine the cost function and the error function. The weights and thresholds are modified using standard backpropagation algorithm. The proposed approach has the advantage of making co…
ZORB speeds up neural network training without sacrificing accuracy.
Backpropagation algorithm is indispensable for the training of feedforward neural networks. It requires propagating error gradients sequentially from the output layer all the way back to the input layer. The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully l…
Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neur…
Backpropagation-free RL method trains layers using local signals.
Artificial neural network training with stochastic gradient descent can be destabilized by "bad batches" with high losses. This is often problematic for training with small batch sizes, high order loss functions or unstably high learning rates. To stabilize learning, we have developed adaptive learning rate clipping (A…
New learning rules for wide neural networks without backpropagation.
Boolean logic used for neural network training and inference, with convergence analysis.
AR algorithm simplifies backpropagation with improved scalability and biological plausibility.
Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network. The backward locking prevents us from updating network layers in parallel and fully leveraging the computing resources. Recently, there are several works trying to decouple and parallelize the ba…
Recently, neural networks in machine learning use rectified linear units (ReLUs) in early processing layers for better performance. Training these structures sometimes results in "dying ReLU units" with near-zero outputs. We first explore this condition via simulation using the CIFAR-10 dataset and variants of two popu…
Boosts linear neurons for faster convergence in neural networks.
Optical co-processor speeds up neural network training.
Deep learning has seen tremendous success over the past decade in computer vision, machine translation, and gameplay. This success rests in crucial ways on gradient-descent optimization and the ability to learn parameters of a neural network by backpropagating observed errors. However, neural network architectures are …
Belief propagation recovers backpropagation results.
New loss function connects learning rate and momentum.
FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.
The backpropagation algorithm for calculating gradients has been widely used in computation of weights for deep neural networks (DNNs). This method requires derivatives of objective functions and has some difficulties finding appropriate parameters such as learning rate. In this paper, we propose a novel approach for c…
Paper proposes DAG-DB for learning discrete DAGs via backpropagation.
Paper proposes a neural network method for fast, interpretable AR model estimation.
We propose a novel deep learning method for local self-supervised representation learning that does not require labels nor end-to-end backpropagation but exploits the natural order in data instead. Inspired by the observation that biological neural networks appear to learn without backpropagating a global error signal,…
A learning rule for first-spike times in neural networks reduces energy consumption and reaction times.
CoNNTrA trains DNNs with low-power, low-memory constraints.
A new method computes gradients without backpropagation.
Spiking neural networks (SNNs) have garnered a great amount of interest for supervised and unsupervised learning applications. This paper deals with the problem of training multi-layer feedforward SNNs. The non-linear integrate-and-fire dynamics employed by spiking neurons make it difficult to train SNNs to generate de…
Many data mining and data analysis techniques operate on dense matrices or complete tables of data. Real-world data sets, however, often contain unknown values. Even many classification algorithms that are designed to operate with missing values still exhibit deteriorated accuracy. One approach to handling missing valu…
Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical stud…
This paper proposes an in-depth re-thinking of neural computation that parallels apparently unrelated laws of physics, that are formulated in the variational framework of the least action principle. The theory holds for neural networks that are also based on any digraph, and the resulting computational scheme exhibits …
Skip connections improve biologically-inspired learning rules.
Pipelined Backpropagation trains large models without batches efficiently.
Mean field theory explains gradient backpropagation in deep dropout networks.
In real-world scenarios, it is appealing to learn a model carrying out stochastic operations internally, known as stochastic computation graphs (SCGs), rather than learning a deterministic mapping. However, standard backpropagation is not applicable to SCGs. We attempt to address this issue from the angle of cost propa…
VSML unifies meta learning concepts and enables simple backpropagation.
Sideways trains video models by overwriting activations as new frames arrive, potentially improving generalization.
Study shows Direct Feedback Alignment fails to offer more efficient scaling than backpropagation.
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simpl…
Article presents QR and LQ decomposition algorithms for various matrix sizes and ranks.
Gradient flossing stabilizes RNN training by controlling Lyapunov exponents.
Survey examines deep neural networks' ability to approximate functions.
New learning rules from information bottleneck improve deep learning without precise labels.
This paper presents a new approach for training artificial neural networks using techniques for solving the constraint satisfaction problem (CSP). The quotient gradient system (QGS) is a trajectory-based method for solving the CSP. This study converts the training set of a neural network into a CSP and uses the QGS to …
A faster, more stable method for optimizing topological functions.