We consider time-domain digital backpropagation with chromatic dispersion filters jointly optimized and quantized using machine-learning techniques. Compared to the baseline implementations, we show improved BER performance and >40% power dissipation reductions in 28-nm CMOS.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “digital backpropagation”
We propose a low-complexity sub-banded DSP architecture for digital backpropagation where the walk-off effect is compensated using simple delay elements. For a simulated 96-Gbaud signal and 2500 km optical link, our method achieves a 2.8 dB SNR improvement over linear equalization.
Optical co-processor speeds up neural network training.
problem Expensive training costs for large neural networks.
method Direct feedback alignment, optical error projection.
result Optical co-processor trains neural networks for handwritten digit recognition.
A neural-network-based approach is presented to efficiently implement digital backpropagation (DBP). For a 32x100 km fiber-optic link, the resulting "learned" DBP significantly reduces the complexity compared to conventional DBP implementations.
A machine learning model for PMD compensation in dual-polarization systems.
problem Compensating for polarization-mode dispersion (PMD) in dual-polarization systems.
method Model-based machine learning approach using the split-step Fourier method for the Manakov-PMD equation.
result The model converges to within 1% of peak dB performance after 428 iterations, achieving a 0.30 dB reduction in effective signal-to-noise ratio compared to PMD-free case.
We propose a model-based machine-learning approach for polarization-multiplexed systems by parameterizing the split-step method for the Manakov-PMD equation. This approach performs hardware-friendly DBP and distributed PMD compensation with performance close to the PMD-free case.
Paper proposes an efficient method to optimize neural networks without backpropagation.
problem Computational inefficiency and scalability issues in neural network optimization.
method Derives explicit solutions to optimize neural networks, reducing computational costs.
result Explicit solutions achieve near-optimality and can discover better optima than backpropagation.
Many transformations in deep learning architectures are sparsely connected. When such transformations cannot be designed by hand, they can be learned, even through plain backpropagation, for instance in attention mechanisms. However, during learning, such sparse structures are often represented in a dense form, as we d…
New loss function connects learning rate and momentum.
problem Finding optimal learning rate and momentum empirically.
method Proposes a new information-theoretical loss function.
result Loss, learning rate, and momentum are closely connected.
Physics-based deep learning improves fiber-optic communication efficiency.
problem Improving signal propagation in fiber-optic communication systems.
method Parameterizing the split-step method of solving the nonlinear Schrödinger equation as a deep neural network.
result Filters can be pruned to as few as 3 taps/step without sacrificing performance.
Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data se…
In a previous work we have detailed the requirements to obtain a maximal performance benefit by implementing fully connected deep neural networks (DNN) in form of arrays of resistive devices for deep learning. This concept of Resistive Processing Unit (RPU) devices we extend here towards convolutional neural networks (…
Paper uses ANFIS to predict cryptocurrency prices.
problem Predicting cryptocurrency prices for seven days.
method Adaptive Network Based Fuzzy Inference System (ANFIS) with hybrid and backpropagation algorithms.
result The method can predict cryptocurrency prices in a short time.
We propose the BinaryGAN, a novel generative adversarial network (GAN) that uses binary neurons at the output layer of the generator. We employ the sigmoid-adjusted straight-through estimators to estimate the gradients for the binary neurons and train the whole network by end-to-end backpropogation. The proposed model …
Belief propagation recovers backpropagation results.
problem Connection between backpropagation and belief propagation poorly understood.
method Converted backpropagation input to belief propagation input and showed results.
result Backpropagation is a special case of belief propagation.
Paper proposes DAG-DB for learning discrete DAGs via backpropagation.
problem Learning Directed Acyclic Graphs (DAGs) from data.
method DAG-DB uses Discrete Backpropagation with I-MLE and Straight-Through Estimation.
result DAG-DB learns DAGs effectively using probabilistic sampling and backpropagation.
GAIT-prop derives a biologically plausible learning rule from backpropagation.
problem Biological implausibility in traditional backpropagation for neural networks.
method GAIT-prop uses a top-down model to convert output error into plausible targets for weight updates.
result GAIT-prop and backpropagation give identical weight updates under certain conditions.
Analog method solves portfolio optimization problems faster and more efficiently.
problem Accurate covariance matrix estimation and fast optimal portfolio selection for financial applications.
method Two-step process using equilibrium propagation and analog Hopfield networks.
result Fully analog pipeline calculates optimal portfolios in energy-efficient manner.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
problem The efficiency of backpropagation in training neural networks.
method Unified vectorized feedback framework for loss-based and reward-based learning, introducing synthetic gradients.
result Synthetic gradients can achieve lower gradient-estimation mean squared error than backpropagation under certain conditions.
A new method computes gradients without backpropagation.
problem Optimization of machine learning models.
method Forward mode automatic differentiation to compute gradients.
result Forward gradient is an unbiased estimate of the gradient, eliminating the need for backpropagation.
New algorithm shows neural networks can learn without full backpropagation.
problem Stochastic gradient descent with backpropagation is non-biologically plausible.
method Random and fixed backpropagation weights in a feedback alignment algorithm.
result Error converges to zero exponentially fast in overparameterized networks.
Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical stud…
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.
ZORB speeds up neural network training without sacrificing accuracy.
problem Slow and strenuous training of neural networks using gradient descent.
method Uses pseudoinverse of targets instead of gradients for backpropagation.
result ZORB converges 300 times faster than Adam on MNIST without hyperparameter tuning.
Backpropagation algorithm is indispensable for the training of feedforward neural networks. It requires propagating error gradients sequentially from the output layer all the way back to the input layer. The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully l…
Analog deep learning shows promise but faces scalability challenges.
problem Bottlenecks in deep neural networks' calculation and optimization.
method Evaluation of eight analog deep learning methodologies.
result Analog deep learning has potential for consumer-level applications but scalability remains an issue.
Backpropagation-free RL method trains layers using local signals.
problem Vanishing or exploding gradients in backpropagation-based RL.
method Local pairwise distance matching for layer-wise training without backpropagation.
result Backpropagation-free method achieves competitive performance and stability.
In real-world scenarios, it is appealing to learn a model carrying out stochastic operations internally, known as stochastic computation graphs (SCGs), rather than learning a deterministic mapping. However, standard backpropagation is not applicable to SCGs. We attempt to address this issue from the angle of cost propa…
Digital money could reduce germ spread during coronavirus.
problem Spreading of germs via paper money during coronavirus.
method Policy recommendations for mobile wallets, digital currencies, and data protection.
result Adopting digital money can help reduce germ spread.
VSML unifies meta learning concepts and enables simple backpropagation.
problem Improving and unifying meta learning concepts for neural networks.
method Unified approach using variable shared meta learning and simple weight-sharing.
result Simple backpropagation can be implemented and meta learned without gradient calculation.
Study of digital topology concepts like hyperspaces and function graphs.
problem Adapting classical topology concepts to digital topology.
method Define digital hyperspaces and function graphs, study their properties.
result Some relationships and graphical properties of digital hyperspaces and function graphs.
Study shows Direct Feedback Alignment fails to offer more efficient scaling than backpropagation.
problem Understanding and optimizing training methods for neural networks.
method Use of scaling laws to compare Direct Feedback Alignment (DFA) and backpropagation.
result DFA fails to offer more efficient scaling than backpropagation.
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that bi…
The paper highlights issues with fixed point claims in digital images.
problem Flaws in published assertions about fixed points in digital images.
method Continues a series of studies examining digital topology.
result Identifies and discusses problems with fixed point claims.
Find limiting sets for digital cones and suspensions.
problem Digital topology cone and suspension constructions.
method Identify (m, n)-limiting sets, especially (0, 0)-freezing sets.
result Discover (0, 0)-limiting sets for digital cones and suspensions.
Article presents QR and LQ decomposition algorithms for various matrix sizes and ranks.
problem Solving least squares problems in machine learning and computer vision.
method Developed novel matrix backpropagation algorithms for QR and LQ decompositions of different matrix sizes and ranks.
result Numerical stability and computational efficiency of the proposed methods.
Corrects incorrect assertions about fixed points in digital topology.
problem Incorrect or incorrectly proven assertions about fixed points in digital metric spaces.
method Analysis of existing assertions and proofs.
result Identifies and corrects errors in published assertions.
Study AFPP of unions of convex digital disks in 2D.
problem Conditions for AFPP of union of convex disks in digital plane.
method Use results from [6] to analyze AFPP.
result Conditions for AFPP of union of convex disks.
The paper addresses flaws in fixed point assertions for digital images.
problem Deficiencies in previously published works on fixed point assertions for digital images.
method Continues a series of studies to identify and rectify issues in fixed point assertions.
result Identifies and corrects flaws in fixed point assertions for digital images.
Study minimal freezing sets in convex digital disks.
problem Finding minimal freezing sets in convex digital disks.
method Showed how to find minimal freezing sets for convex disks in digital plane.
result Found minimal freezing sets for convex disks in digital plane.
Nowadays, deep learning can be employed to a wide ranges of fields including medicine, engineering, etc. In deep learning, Convolutional Neural Network (CNN) is extensively used in the pattern and sequence recognition, video analysis, natural language processing, spam detection, topic categorization, regression analysi…
Study on cold and freezing sets in digital images.
problem Properties of cold sets in digital images.
method Analysis of properties and relationships between cold and freezing sets.
result Examined relationships between cold and freezing sets.
Incorrect fixed point assertions in digital topology are discussed.
problem Incorrect or poorly stated fixed point assertions in digital topology.
method Discussion of problematic publications in digital metric spaces.
result Clarification of incorrect fixed point assertions.
In this paper, we show how to construct graph theoretical models of n-dimensional continuous objects and manifolds. These models retain topological properties of their continuous counterparts. An LCL collection of n-cells in Euclidean space is introduced and investigated. If an LCL collection of n-cells is a cover of a…
Incorrect fixed point assertions in digital topology are discussed.
problem Incorrect, incorrectly proven, or trivial fixed point assertions in digital topology.
method Continues earlier work on identifying and critiquing bad fixed point assertions.
result Clarifies the nature and extent of incorrect fixed point assertions in digital topology.
A faster, more stable method for optimizing topological functions.
problem Optimizing topological functions is computationally expensive and unstable.
method Introduces a novel backpropagation scheme for faster and more robust optimization.
result Produces more robust optima and stable visualizations.
New learning rules for wide neural networks without backpropagation.
problem Training wide neural networks efficiently and without backpropagation.
method Input-weight alignment driven by gradient descent in the NTK regime.
result Biologically-motivated learning rules equivalent to backpropagation in wide networks.
Digital trees have approximate fixed point property, and conditions for products are explored.
problem Conditions for the approximate fixed point property in digital tree products.
method Analyzes digital trees and their products, explores conditions for the AFPP.
result Conditions are found for the AFPP in digital tree products.