We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis. We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “Multiplicative Recurrent Networks”
RACDNN improves saliency detection by iteratively refining attention to multiple scales.
problem Saliency detection struggles with objects of varying scales.
method Recurrent attentional convolutional-deconvolution network (RACDNN) using spatial transformer and recurrent units.
result RACDNN outperforms state-of-the-art methods on saliency detection datasets.
New models improve language generation by sharing intermediate states.
problem Language models struggle with past mistakes in sequence generation.
method Integrate second-order terms in hidden-state update, sharing intermediate states.
result Shared parametrization improves language modeling performance.
mLSTM improves sequence modeling with better autoregressive density estimation.
problem Improving autoregressive density estimation in sequence modeling.
method Introduces mLSTM, a recurrent neural network combining LSTM and multiplicative recurrent networks.
result mLSTM outperforms standard LSTM and its variants in character-level language modeling tasks.
New RNN model fuses sensor data from multiple stations.
problem Modeling distributed sensor networks for future behavior prediction.
method Multi-Encoder-Decoder RNN architecture with attention mechanism.
result Model improves prediction accuracy on real-world sensor datasets.
Scalable verifier for recurrent neural networks using polyhedral abstractions.
problem Certifying the correctness of recurrent neural networks.
method Combining sampling, optimization, and Fermat's theorem for polyhedral abstractions; gradient descent for refinement.
result Successfully verified challenging recurrent models in various domains.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Recurrent Neural Networks (RNNs) have long been recognized for their potential to model complex time series. However, it remains to be determined what optimization techniques and recurrent architectures can be used to best realize this potential. The experiments presented take a deep look into Hessian free optimization…
Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4mlcs.LG
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
Recurrent Neural Networks handle sequential data better than traditional networks.
problem Handling temporal context and gradient flow in training RNNs.
method Explains the structure and challenges of RNNs, including training complexities.
result RNNs improve performance in various sequential tasks.
GRNN predicts multiple medical concepts from patient discharge summaries.
problem Understanding medical concepts in patient discharge summaries.
method Grounded Recurrent Neural Network (GRNN) that ties labels to specific dimensions of the hidden state.
result GRNN outperforms baselines in predicting medical concepts from discharge summaries.
Improved sentiment analysis explanations using LRP for RNNs.
problem Creating understandable explanations for recurrent neural network predictions.
method Extending Layer-wise Relevance Propagation (LRP) to recurrent neural networks (RNNs), specifically to multiplicative connections in LSTMs and GRUs.
result Better explanation quality for sentiment analysis tasks using LRP compared to gradient-based methods.
PSRNNs combine RNN and PSR insights for system filtering and prediction.
problem Modeling dynamical systems efficiently and accurately.
method Combines insights from RNNs and PSRs using bilinear transfer functions and tensor decomposition.
result PSRNNs outperform other models in filtering and prediction tasks across multiple datasets.
Transformer models outperform recurrent ones in modeling hierarchical data.
problem Modeling hierarchical structure in data.
method Introducing Multiresolution Transformer Networks leveraging self-attention.
result Multiresolution Transformer Networks significantly outperform state-of-the-art models on query suggestion datasets.
Professor Forcing improves recurrent network training and sampling quality.
problem Training and sampling quality issues in recurrent networks.
method Adversarial domain adaptation to align training and sampling dynamics.
result Improves test likelihood and sample quality in various tasks.
Adaptive RNN using mixture layer for multi-pattern sequences.
problem Inadequate RNN performance on sequences with multiple patterns.
method Introducing a mixture layer to partition and store prototype vectors, enabling adaptive state updates.
result M-RNN outperforms traditional RNN in assimilating sequences with multiple patterns.
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamicscs.LG
Reverse engineered RNNs reveal line attractor dynamics for sentiment classification.
problem Understanding how recurrent neural networks solve sequential tasks like sentiment classification.
method Dynamical systems analysis to reverse engineer trained RNNs, identifying fixed points and linearized dynamics.
result Trained RNNs converge to low-dimensional line attractor dynamics, providing interpretable solutions.
RC reduces neural network redundancy and improves performance through independent BN layers.
problem Improving neural network performance and reducing redundancy.
method Recurrent convolution with independent batch normalization layers for different unrolling steps.
result The proposed method improves RC networks' performance and achieves cost-adjustable inference.
Do Cascades Recur?cs.SI
The study finds that many social media content cascades recur with bursts of popularity over time.
problem Understanding the recurrence of content cascades on social media.
method Large-scale analysis of cascades on Facebook over extended time periods, using a simple model to simulate recurrence.
result Content virality drives recurrence, but beyond a certain popularity, the rate of recurrence decreases.
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
HMRNN combines HMMs and neural networks for Alzheimer's disease forecasting.
problem Improving disease progression modeling with hidden states not fully known.
method Developed HMRNN combining HMMs and recurrent neural networks.
result HMRNN improves disease forecasting and offers novel clinical interpretation.
Full-capacity uRNNs improve performance over restricted-capacity ones.
problem Vanishing and exploding gradient issues in recurrent neural networks.
method Optimized full-capacity unitary recurrence matrices over all unitary matrices.
result Significantly improved performance compared to LSTMs and restricted-capacity uRNNs.
ASRNN adapts scales dynamically for better sequence modeling.
problem Fixed scales in multiscale RNNs don't match temporal patterns.
method Adaptively learns and adjusts scales based on temporal contexts.
result ASRNNs yield better performances with dynamical scaling.
High-order RNNs reduce speech recognition errors.
problem Vanishing gradients in RNNs.
method High-order RNNs with multiple connections from previous time steps.
result HORNNs reduce WER by 4.2% and 6.3% over RNNs.
Recurrent CNNs improve image classification in low light conditions.
problem Poor performance of CNNs in noisy images.
method Added recurrent connections to CNN layers to enhance robustness.
result gruCNNs outperform cCNNs in low signal-to-noise ratio images.
Deep learning predicts real-time parking occupancy using multiple data sources.
problem Predicting real-time parking occupancy in spatio-temporal networks.
method Graph-Convolutional Neural Networks (GCNN) for spatial relations, Recurrent Neural Networks (RNN) with Long-Short Term Memory (LSTM) for temporal features, multiple data sources.
result The model outperforms other methods with an average testing MAPE of 10.6%.
Mod-DeepESN improves echo state networks for complex, multi-scale tasks.
problem Efficiency in solving complex, multi-scale temporal tasks.
method Incorporates intrinsic plasticity into a modular deep echo state network architecture.
result Significantly outperforms state-of-the-art for time series prediction tasks.
SRNNs learn dynamics of physical systems from data.
problem Learning dynamics of complex, noisy Hamiltonian systems.
method SRNNs model Hamiltonian functions with neural networks, using symplectic integration and optimization.
result SRNNs reliably learn dynamics of complex and noisy Hamiltonian systems.
Theoretical analysis shows RNNs with various nonlinearities benefit from depth efficiency.
problem Theoretical understanding of RNNs' efficiency is limited.
method Extended analysis to RNNs with Rectified Linear Unit (ReLU) and other nonlinearities.
result Various nonlinear RNNs also benefit from depth efficiency.
Research develops a water quality prediction model using LSTM.
problem Global degradation of water resources and need for optimal water quality monitoring.
method Developed a multivariate water quality prediction model using LSTM and historical data.
result Multiple step LSTM model achieved RMSE of 0.227 mg/L.
Recurrent Neural Networks with more flexible memory: better predictions than rough volatilityq-fin.ST
Extended LSTMs improve volatility prediction by 20%.
problem Predicting asset price volatility with long memory.
method Extended LSTMs with multiple flexible timescales.
result Extended LSTMs outperform rough volatility predictions by 20%.
RED models efficiently estimate distributions of real-valued data.
problem Efficient modeling of real-valued data with conditional dependencies.
method Semiparametric modeling using recurrent neural networks (RNNs).
result RED models achieve lower held-out negative log-likelihood than other neural network approaches.
Paper compares training and sampling schemes for character-level RNNs.
problem Optimizing training and sampling schemes for character-level RNNs.
method Tested various training and sampling schemes on different datasets and architectures.
result The choice of training and sampling scheme affects stability, performance, and implementation effort.
The paper improves DFA for CNN and RNN training to match BP accuracy.
problem Low accuracy in CNN and RNN training using DFA.
method Divide network into modules, apply DFA within, use sparse backward weight, and incorporate dilated convolution and sparse matrix multiplication.
result Achieves BP-level accuracy in CNN and RNN training.
Unified GARCH-NN models improve financial volatility forecasting.
problem Improving financial volatility forecasting accuracy and efficiency.
method Embedding GARCH dynamics within recurrent neural networks (GRU and LSTM).
result Unified GARCH-NN models outperform classical GARCH and hybrid methods.
A new approach to learning in brain-like networks using adversarial algorithms.
problem Complex inter-dependencies in brain-like networks not compatible with conditional independence assumptions.
method Adversarial algorithm for learning models of perceptual processing.
result The approach can mimic known neural phenomena and yields testable hypotheses.
This paper improves RNN memory capacity for long sequences through learning associative memory update rules.
problem Challenges in RNNs remembering long sequences.
method Jointly learns memory update rule with task objective and uses multiple associative memories.
result Improves memory capacity for long sequence encoding.
3D-PRNN generates shapes from depth images using recurrent neural networks.
problem Representing 3D shapes from limited sensor data.
method Generative Recurrent Neural Network (3D-PRNN) with Gaussian Fields.
result 3D-PRNN synthesizes plausible shapes from primitives, outperforming nearest-neighbor methods.
Self-organized action hierarchy and compositionality learned by RNNs.
problem Improving RNN architectures for reinforcement learning.
method Multiple-timescale, stochastic RNN for RL.
result Network autonomously learns sub-goals and develops an action hierarchy.
Linear RC shows hierarchical temporal patterns in state signals.
problem Understanding hierarchical temporal representations in deep RNNs.
method Used linear recurrent units and frequency analysis on state signals.
result Linear RC reveals intrinsic hierarchical temporal structure.
Quaternion neural networks improve distant speech recognition.
problem Challenges in distant speech recognition due to noise and reverberation.
method Quaternion neural networks process multi-channel audio signals as quaternion entities, capturing internal and external dependencies.
result QLSTM outperforms real-valued LSTM on multi-channel distant speech recognition tasks.
Overview of structured data representation methods.
problem Structured data lacks vectorial form, complicating machine learning.
method Various approaches including kernel, distance, neural networks, and graph convolutional networks.
result New approaches like metric learning and recurrent decoder networks have emerged.
BRITS fills missing values in correlated time series data without specific assumptions.
problem Missing values in correlated time series data.
method Bidirectional Recurrent Neural Networks (RNN) for imputation without specific assumptions.
result BRITS outperforms state-of-the-art methods in imputation and classification/regression accuracies.
ParaRNN: An Interpretable and Parallelizable Recurrent Neural Network for Time-Dependent Datastat.ML
ParaRNN improves RNN interpretability and parallelizability for time-dependent data.
problem Limited interpretability and slow training of RNNs.
method Parallelized RNN with additive representation and recurrence features.
result ParaRNN achieves comparable performance to vanilla RNNs but with improved interpretability and efficiency.
Paper compares RNNs for short-term traffic flow prediction.
problem Accurate short-term traffic flow prediction for smart cities.
method Used three types of RNNs (simple RNN, GRU, LSTM) and compared with other models.
result Simple RNNs and GRU outperform LSTM in accuracy and training time.
SeqSleepNet tackles automatic sleep staging as a sequence-to-sequence problem.
problem Automatic sleep staging as a sequence-to-sequence classification problem.
method End-to-end hierarchical recurrent neural network (SeqSleepNet) with filterbank and attention-based recurrent layers.
result SeqSleepNet achieves high accuracy (87.1% overall accuracy, 83.3% macro F1-score, 0.815 Cohen's kappa) on a publicly available dataset.
Proposes a new RNN model for grouped sequential data with varying time intervals.
problem Implicitly models fixed time intervals between observations and lacks group-level effects.
method Mixed membership framework for RNN, learning group-level base parameter.
result Demonstrates dynamic topic modeling with evolving topic distributions over time.
Dynamic sparseness reduces neural network computation by selectively omitting parts of computations.
problem Reducing the computational and memory footprint of neural networks.
method Combining dynamic sparseness with block-wise matrix-vector multiplications to selectively omit parts of computations.
result The proposed method outperforms static sparseness and achieves similar perplexities at half the computational cost.