The Resilient Propagation (Rprop) algorithm has been very popular for backpropagation training of multilayer feed-forward neural networks in various applications. The standard Rprop however encounters difficulties in the context of deep neural networks as typically happens with gradient-based learning algorithms. In th…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “resilient backpropagation”
Adversarial learning approximates unknown quantum states on near-term quantum computers.
problem Approximating unknown quantum pure states on near-term quantum computers.
method Two parametrized circuits optimized adversarially, with resilient backpropagation and bipartite entanglement entropy.
result Resilient backpropagation algorithms perform well in optimizing the two circuits.
Method creates resilient classifiers by focusing on highly predictive features.
problem Adversarial attacks exploit weakly correlated features learned during training.
method Resilient feature engineering to prioritize highly predictive features.
result Serial and Parallel Composition Resilience theorems support the design of resilient classifiers.
In this paper, we explore and detail our experiments in a high-dimensionality, multi-class image classification problem often found in the automatic recognition of Sign Languages. Here, our efforts are directed towards comparing the characteristics, advantages and drawbacks of creating and training Support Vector Machi…
Paper introduces a new index to measure financial and workplace resilience of firms.
problem Corporate resilience and its types in turbulent markets.
method Quantitative analysis of earnings expectations and implied discount rates.
result Evidence of workplace resilience amplification by financial status in the COVID-19 era.
Measures financial resilience using BSDEs and their properties.
problem Measuring financial resilience in dynamic risk environments.
method Developed stochastic calculus for BSDEs with jumps, revealing resilience rate as expectation of generator.
result Resilience rate can be represented as expectation of BSDE generator, revealing properties of dynamic risk measures.
Python tool assesses European agricultural production resilience.
problem Estimating resilience of European agricultural production systems.
method Python software to compute annual production resilience indicator.
result Demonstrates resilience of European agricultural production.
This paper measures financial market resilience in China and identifies key uncertainties.
problem Measuring financial market resilience in China.
method Quantitative analysis of total financial market and sub-markets, Diebold-Yilmaz connectedness approach.
result Financial market resilience in China is event-driven and influenced by geopolitical risks, economic and trade policy uncertainty, and U.S.-China tensions.
Study proposes a new resilience metric for stock market performance analysis.
problem Quantifying resilience cycles in stock market performance.
method Systems-oriented approach with Robustness Range and Elasticity Threshold.
result New metric quantifies non-linear resilience cycles in stock markets.
The resilience of low-degree Rademacher chaos is studied, providing probabilistic lower bounds.
problem Understanding how much a Rademacher chaos can withstand adversarial sign-flips without significant probability changes.
method Probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary degree.
result Probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary degree, especially meaningful for constant degree.
New model shows negative resilience can improve trading efficiency.
problem Optimal trade execution in limit order books with negative resilience.
method Stochastic order book model with negative resilience.
result Negative resilience can lead to more efficient trading.
Proposes resilience metrics for large blackout costs with logarithmic resilience.
problem Large variations in blackout costs make estimating risk impractical.
method Uses mean of log of large blackout costs, tail slope index, and frequency.
result Solves problems of heavy tail and large variations in blackout costs.
Gated Linear Networks bypass feature learning for fast online learning.
problem Fast online learning and feature learning trade-offs in neural networks.
method Distributed and local credit assignment mechanism, data-dependent gating, online convex optimization.
result GLNs achieve universal learning capabilities and resilience to catastrophic forgetting.
Stocks of more resilient firms outperformed during the pandemic, reflecting disaster risk.
problem The impact of social distancing on firms' operations and stock performance.
method Cross-sectional analysis of firms' resilience and stock performance, controlling for risk factors.
result Stocks of more resilient firms are expected to yield significantly lower returns than less resilient ones, reflecting disaster risk.
ResiliNet improves distributed neural network inference resilience.
problem Physical node failures in distributed neural networks cause performance drops.
method Skip hyperconnection and failout technique.
result ResiliNet provides inference resiliency for distributed neural networks.
Belief propagation recovers backpropagation results.
problem Connection between backpropagation and belief propagation poorly understood.
method Converted backpropagation input to belief propagation input and showed results.
result Backpropagation is a special case of belief propagation.
Paper benchmarks DRL policies' resilience to state transitions.
problem Measuring DRL policies' resilience to state perturbations.
method Disentangled representation learning and RL-based techniques.
result Demonstrated feasibility of resilience benchmarking in DQN, A2C, and PPO2.
New method quantifies resilience of electric distribution systems from historical data.
problem Large blackouts caused by extreme winds have significant costs and impacts.
method Formulate large event risk from utility outage data, quantify resilience improvements through investments.
result Investments in wind hardening and faster restoration can reduce the probability of large cost events.
Paper proposes DAG-DB for learning discrete DAGs via backpropagation.
problem Learning Directed Acyclic Graphs (DAGs) from data.
method DAG-DB uses Discrete Backpropagation with I-MLE and Straight-Through Estimation.
result DAG-DB learns DAGs effectively using probabilistic sampling and backpropagation.
New framework for resilient bi-criteria optimization under noisy feedback.
problem Bi-criteria combinatorial optimization with noisy function evaluations.
method Introducing -resilience and developing a black-box framework.
result Achieves sublinear regret and constraint violation for bi-criteria bandit problems.
GAIT-prop derives a biologically plausible learning rule from backpropagation.
problem Biological implausibility in traditional backpropagation for neural networks.
method GAIT-prop uses a top-down model to convert output error into plausible targets for weight updates.
result GAIT-prop and backpropagation give identical weight updates under certain conditions.
Improved BER with reduced power in time-domain digital backpropagation.
problem Improving BER performance in time-domain digital backpropagation.
method Jointly optimized and quantized chromatic dispersion filters using machine learning.
result Improved BER performance and power dissipation reductions.
Paper tackles Byzantine resilience in distributed multi-task learning.
problem Resilience of distributed algorithms in the presence of Byzantine agents.
method Online weight assignment rule based on accumulated loss and filtering.
result Aggregation with proposed weight assignment rule improves expected regret.
Proposes a new backpropagation algorithm for deep learning with guaranteed convergence.
problem Backward locking in backpropagation limits parallel updates in deep neural networks.
method Decouples gradients and splits the network into modules for parallel updates, proving convergence for non-convex problems.
result The proposed algorithm achieves significant speedup without accuracy loss in training deep convolutional neural networks.
This paper uses robust optimization to analyze supply chain resilience.
problem Supply chain resilience analysis of multi-modal logistics networks.
method Robust optimization with budget-of-uncertainty.
result Interactive effects of network size, disruption scale, and degree on resilience.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
problem The efficiency of backpropagation in training neural networks.
method Unified vectorized feedback framework for loss-based and reward-based learning, introducing synthetic gradients.
result Synthetic gradients can achieve lower gradient-estimation mean squared error than backpropagation under certain conditions.
A new method computes gradients without backpropagation.
problem Optimization of machine learning models.
method Forward mode automatic differentiation to compute gradients.
result Forward gradient is an unbiased estimate of the gradient, eliminating the need for backpropagation.
Small stocks drive market crashes by suppressing resilience.
problem The single-security price limit exacerbates market panic during crashes.
method Simplified dynamic model on networks of investors and stocks, empirical verification.
result Unexpected linear association between price limit and critical market confidence.
Unified framework for faster neural network training with less information loss.
problem Time-consuming backpropagation and loss of unpropagated gradient information.
method Unified sparse backpropagation framework and memorized sparse backpropagation algorithm.
result Convergence in probability with certain conditions and effective information loss mitigation.
Backprop-Q extends standard backpropagation for stochastic computation graphs.
problem Applying standard backpropagation to stochastic computation graphs is challenging.
method Construct Q-functions for each stochastic node and use them to train the SCG with standard backpropagation.
result Generalized backpropagation for stochastic computation graphs is feasible and extends learning signals beyond gradients.
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Conditioncs.LG
In 1963, Polyak proposed a simple condition that is sufficient to show a global linear convergence rate for gradient descent. This condition is a special case of the Łojasiewicz inequality proposed in the same year, and it does not require strong convexity (or even convexity). In this work, we show that this much-older…
Improved deep neural network generalization through noise resilience.
problem Understanding and predicting generalization error of deep neural networks.
method Noise resilience measures to predict generalization error.
result Secured 5th position in the PGDL competition at NeurIPS 2020.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
problem Byzantine attacks and communication inefficiency in federated learning.
method Transformed robust aggregation for zero-order optimization under client heterogeneity.
result CyBeR-0 achieves stable performance with minimal communication costs and reduced memory usage.
A methodology for resilience analysis of Capsule Networks under approximation errors.
problem Resilience of Capsule Networks under approximation errors.
method Modeling and analyzing approximation errors in Capsule Networks' inference.
result Capsule Networks are more resilient to errors during dynamic routing than other stages.
Firms' collaboration networks can decline but remain resilient.
problem Resilience of firms' collaboration networks during decline.
method Analysis of 21,500 R&D collaborations over 25 years, simulating drop-out cascades.
result Firms' collaboration networks can adapt to mitigate decline and recover.
This paper evaluates how well outlier detection methods work with samples of data.
problem Resilience of outlier detection methods to sampling.
method Estimating resilience to sampling of outlier detection methods and their ensembles.
result Outlier detection methods are not equally resilient to sampling, and careful selection of sampling scheme and method is necessary.
New algorithm shows neural networks can learn without full backpropagation.
problem Stochastic gradient descent with backpropagation is non-biologically plausible.
method Random and fixed backpropagation weights in a feedback alignment algorithm.
result Error converges to zero exponentially fast in overparameterized networks.
Automated design of resilient, efficient DNNs for hardware.
problem Designing reliable and efficient DNNs for hardware.
method Evolutionary optimization technique for DNN architecture design.
result Strong correlation between predicted and actual error resilience.
We present a large-scale study of commonality in liquidity and resilience across assets in an ultra high-frequency (millisecond-timestamped) Limit Order Book (LOB) dataset from a pan-European electronic equity trading facility. We first show that extant work in quantifying liquidity commonality through the degree of ex…
Paper evaluates ML's resilience in detecting ransomware.
problem Resilience of ML algorithms for ransomware detection.
method Used GAN to generate dynamic features to test ML models.
result ML models degrade in performance when faced with GAN-generated samples.
Study reveals clusters of resilient and vulnerable Spanish agri-food firms post-Ukraine-Russia war.
problem Financial resilience of agri-food companies in Spain during the Ukraine-Russia conflict.
method Cluster analysis using centred log-ratios for compositional data of financial ratios.
result Increase in resilient firms by 2023, highlighting sectoral adaptation to economic challenges.
Mitigates faults in DNNs by clipping activation values, improving their resilience.
problem Fault tolerance of DNNs in safety-critical applications.
method Clipping activation functions to reduce impact of faulty weights.
result Significant improvement in classification accuracy (68.92%) for fault mitigation.
New framework boosts neural network performance and resilience.
problem Susceptibility of compact neural network implementations to system disturbances.
method Realistic crossbar simulations and Mosaics framework to re-use synaptic connections.
result Compact neural networks are noise-immune and perform well under disturbances.
BRIDGE improves decentralized learning resilience against Byzantine failures.
problem Decentralized learning in distributed systems is vulnerable to Byzantine failures.
method Introduces BRIDGE, a scalable Byzantine-resilient decentralized gradient descent algorithm.
result Proves algorithmic and statistical convergence guarantees for BRIDGE.
Skip connections improve biologically-inspired learning rules.
problem Biologically-inspired learning rules often underperform compared to backpropagation.
method Introduced skip connections between intermediate layers in biologically-motivated learning rules.
result Skip connections can match the performance of backpropagation and are robust to hyper-parameters.
We show that wealth processes in the block-shaped order book model of Obizhaeva/Wang converge to their counterparts in the reduced-form model proposed by Almgren/Chriss, as the resilience of the order book tends to infinity. As an application of this limit theorem, we explain how to reduce portfolio choice in highly-re…
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.
Spectral backpropagation optimizes implicit likelihoods without explicit assumptions.
problem Training systems without explicit goals and likelihood functions.
method Spectral backpropagation for implicit likelihood optimization.
result Identified two novel properties of GAN generators: aberrant outputs and quasi-disentangled factors.