New LCM aggregator improves GNN performance and efficiency.
problem Graph neural networks' sensitivity to aggregation function choice.
method Learnable commutative monoid for graph aggregation.
result LCM aggregator achieves performance competitive with recurrent aggregators.
This work examines aggregation functions in Deep Set learning.
problem The sensitivity of Deep Set networks to aggregation function choices.
method Investigation of alternative aggregation functions, including learnable recurrent ones.
result Learnable aggregations improve performance, reduce hyper-parameter sensitivity, and generalize better.
RAM extends attention-based mechanism for existence determination.
problem Binary determination of object existence in cluttered images.
method Recurrent attention model (RAM) with k-maximum aggregation layer and new reward mechanism. result Significant efficiency and accuracy improvement over existing approaches.
RE-NET predicts future interactions in temporal knowledge graphs.
problem Predicting future facts in temporal knowledge graphs.
method Autoregressive architecture with recurrent event encoder and neighborhood aggregator.
result State-of-the-art performance on five public datasets.
We investigate the performance of features that can capture nonlinear recurrence dynamics embedded in the speech signal for the task of Speech Emotion Recognition (SER). Reconstruction of the phase space of each speech frame and the computation of its respective Recurrence Plot (RP) reveals complex structures which can…
This research improves transparency in RNN predictions of ICU mortality risk.
problem Lack of transparency in RNNs' healthcare predictions.
method Introduced Learned Binary Masks (LBM) and KernelSHAP for identifying EMR variables' contributions to RNN risk of mortality predictions.
result Attribution matrices show each input feature's contribution to RNN predictions, facilitating analysis of the model and its predictions.
News novelty predicts negative stock market returns.
problem Negative stock market returns due to increased news novelty.
method Quantified news novelty using entropy measure from recurrent neural network applied to a large news corpus.
result Entropy exposure carries a negative risk premium, indicating that assets positively correlated with entropy hedge aggregate news risk.
Adaptive prediction timing improves healthcare outcomes by predicting patient events at the right frequency.
problem Inconsistent prediction granularity in healthcare models.
method Introduces a novel approach using Bayesian recurrent models and a new aggregation method to adapt prediction frequency based on uncertainty.
result Adaptive prediction timing leads to improved predictive performance, especially in the critical first 12 hours of patient stay.
Memory-Augmented Recurrent Networks improve dialogue coherence by expanding conversation history storage.
problem Fixed-size vectors limit dialogue coherence; attention mechanisms are computationally expensive.
method Introduce Neural Turing Machines (NTMs) to provide flexible and permanent storage for dialogue history.
result Improved perplexity performance compared to existing baselines.
Paper introduces a novel traffic forecasting model using autoencoders and exogenous variables.
problem Traffic forecasting using aggregated data from vehicles and infrastructure.
method Recurrent Autoencoder with skip connections and exogenous variables for dynamic traffic data.
result Model predicts speed, volume, and traffic direction with exogenous variables like weather and time.
Recently a variety of methods have been developed to encode graphs into low-dimensional vectors that can be easily exploited by machine learning algorithms. The majority of these methods start by embedding the graph nodes into a low-dimensional vector space, followed by using some scheme to aggregate the node embedding…
We describe a deep learning approach for automated brain hemorrhage detection from computed tomography (CT) scans. Our model emulates the procedure followed by radiologists to analyse a 3D CT scan in real-world. Similar to radiologists, the model sifts through 2D cross-sectional slices while paying close attention to p…
Determining whether hypotensive patients in intensive care units (ICUs) should receive fluid bolus therapy (FBT) has been an extremely challenging task for intensive care physicians as the corresponding increase in blood pressure has been hard to predict. Our study utilized regression models and attention-based recurre…
RE enhances DL by learning model behavior, enabling iterative self-improvement.
problem Static data representations limit DL's potential for evolving models.
method RE uses multiple mappings of data through identical deep architectures, analyzing internal representations and performance signals.
result Models can gain insight from predecessors, leading to iterative self-improvement.
FMA-ETA predicts travel time using FFN with attention.
problem Estimating travel time from spatial-temporal data.
method FFN with Multi-factor self-attention mechanism.
result FMA-ETA outperforms state-of-the-art methods in prediction accuracy with faster inference.
The size of a website's active user base directly affects its value. Thus, it is important to monitor and influence a user's likelihood to return to a site. Essential to this is predicting when a user will return. Current state of the art approaches to solve this problem come in two flavors: (1) Recurrent Neural Networ…
FedGRU uses federated learning to predict traffic flow accurately while preserving user privacy.
problem Developing accurate traffic flow prediction while protecting user privacy.
method Federated Learning, Secure Parameter Aggregation, Joint Announcement Protocol, Ensemble Clustering.
result FedGRU achieves 90.96% higher prediction accuracy than advanced deep learning models.
Proposes a new binary classification model inspired by fluid phase separation.
problem Binary classification challenges.
method Discretization of nonlinear reaction-diffusion equation coupled with ODE, inspired by fluid dynamics.
result PSBC model achieves comparable performance to traditional methods on MNIST.
The pullback approach to global Finsler geometry is adopted. Three classes of recurrence in Finsler geometry are introduced and investigated: simple recurrence, Ricci recurrence and concircular recurrence. Each of these classes consists of four types of recurrence. The interrelationships between the different types of …
Many real-world sequential decision making problems are partially observable by nature, and the environment model is typically unknown. Consequently, there is great need for reinforcement learning methods that can tackle such problems given only a stream of incomplete and noisy observations. In this paper, we propose d…
Study on biharmonic hypersurfaces with specific recurrent operators in Euclidean space.
problem Characterizing biharmonic hypersurfaces with recurrent operators.
method Analysis of various recurrent operators and their impact on biharmonic hypersurfaces.
result Some well-known recurrent operators play a significant role in making biharmonic hypersurfaces minimal.
The aim of the present paper is to investigate new types of recurrence in Finsler geometry, namely, hyper-generalized recurrence and generalized conharmonic recurrence. The properties of such recurrences and their relations to other Finsler recurrences are studied.
To generalize the notion of recurrent manifold, there are various recurrent like conditions in the literature. In this paper we present a recurrent like structure, namely, \textit{super generalized recurrent manifold}, which generalizes both the hyper generalized recurrent manifold and weakly generalized recurrent mani…
Two special Finsler spaces have been introduced and investigated, namely Rh-recurrent Finsler space and consircularly recurrent Finsler space. The defining properties of these spaces are formulated in terms of the first curvature tensor of Cartan connection. The following three results constitute the main object of …
U-Time uses a fully convolutional network for sleep stage classification.
problem Challenges in tuning and optimizing recurrent neural networks for sleep data.
method U-Time is a fully feed-forward deep learning approach based on U-Net architecture.
result U-Time outperforms state-of-the-art models for sleep stage classification.
We parallelize backpropagation for deep learning models, achieving significant speedups.
problem Sequential dependency in backpropagation limits scalability on parallel systems.
method Reformulated backpropagation as a scan operation, using Blelloch scan algorithm.
result Up to 2.75x speedup on overall training time and 108x on backward pass.
MES-LSTM hybrid method improves multivariate time series forecasting and mortality modeling.
problem Challenges in applying hybrid forecast methods to multivariate data.
method Generalized multivariate extension of ES-RNN, utilizing vectorized implementation.
result MES-LSTM shows significant improvement over pure statistical and deep learning methods in forecast accuracy and prediction interval construction.
The object of the present paper is to obtain the characterization of a warped product semi-Riemannian manifold with a special type of recurrent like structure, called super generalized recurrent. As consequence of this result we also find out the necessary and sufficient conditions for a warped product manifold to sati…
The present paper deals with the proper existence of a generalized class of recurrent manifolds, namely, hyper-generalized recurrent manifolds. We have established the proper existence of various generalized notions of recurrent manifolds. For this purpose we have presented a metric and computed its curvature propertie…
We show that in dimension n>3 the class of simple conformally recurrent space-times coincides with the class of conformally recurrent pp-waves.
Interneurons improve learning in neural networks by accelerating convergence.
problem Rapid adaptation to changing input statistics in neural networks.
method Two mathematically tractable recurrent linear neural networks were compared: one with direct recurrent connections and the other with interneurons that mediate recurrent communication.
result The network with interneurons converges more quickly than the network with direct recurrent connections, scaling logarithmically with initialization spectrum.
Unified recurrent networks reveal differences in complexity levels of grammars.
problem Understanding the complexity and behavior of recurrent networks.
method Connecting recurrent networks with deterministic finite automata and formal grammars.
result Unified recurrent networks improve performance and match grammars from different complexity levels.
RESHAPE explains financial statement anomalies by aggregating explanations from AENNs.
problem Detecting and explaining accounting anomalies in financial audits is challenging.
method Proposes RESHAPE to explain model output on an aggregated attribute-level.
result RESHAPE provides more comprehensible explanations compared to existing methods.
Study predicts colorectal polyp recurrence using medical records and statistical models.
problem Identifying patient characteristics influencing colorectal polyp recurrence.
method Natural language processing for extracting polyp characteristics, Kaplan-Meier curves, Cox proportional hazards modeling, random survival forest models.
result Polyp size, number, location, and patient smoking status significantly influence recurrence risk.
It is proved that every concircularly recurrent manifold must be necessarily a recurrent manifold.
Proposes a new optimization-based method for aggregating sets in neural networks.
problem Limited representational power of existing aggregation methods.
method Equilibrium Aggregation: an optimization-based approach.
result Equilibrium Aggregation outperforms existing methods in various tasks.
Recurrent-DBN models dynamic relational data with interpretable latent structures.
problem Interpreting dynamic relational data with hidden structures.
method Recurrent Dirichlet Belief Network framework with hierarchical latent structures and efficient inference strategy.
result Recurrent-DBN discovers interpretable latent structures and improves link prediction.
In this paper we adopt the pullback approach to global Finsler geometry. We investigate horizontally recurrent Finsler connections. We prove that for each scalar (π)1-form A, there exists a unique horizontally recurrent Finsler connection whose h-recurrence form is A. This result generalizes the existence and u…
Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically, we prove stable recurrent neural networks are well approximated by feed-forward …
Transformer models outperform recurrent ones in modeling hierarchical data.
problem Modeling hierarchical structure in data.
method Introducing Multiresolution Transformer Networks leveraging self-attention.
result Multiresolution Transformer Networks significantly outperform state-of-the-art models on query suggestion datasets.
Lipschitz RNNs improve stability and performance in various tasks.
problem Improving stability and performance of RNNs.
method Introduced a Lipschitz recurrent unit with a linear and Lipschitz nonlinear component for stability analysis.
result Lipschitz RNNs outperform existing units on benchmark tasks.
Turbo-Aggregate reduces secure aggregation time from quadratic to nearly linear.
problem Quadratic overhead in secure model aggregation for federated learning.
method Multi-group circular strategy, additive secret sharing, and coding techniques.
result Achieves O(NlogN) overhead, compared to O(N2), for up to 50% user dropout. This paper analyzes the generalization risk of unrolled neural networks using Stein's Unbiased Risk Estimator.
problem Analyzing the generalization risk of unrolled neural networks and its relationship to network design and train sample size.
method Using Stein's Unbiased Risk Estimator (SURE), the paper analyzes the generalization risk with bias and variance components for recurrent unrolled networks, focusing on the degrees-of-freedom (DOF) component and the trace of the end-to-end network Jacobian.
result DOF is well-approximated by the weighted path sparsity of the network under incoherence conditions on the trained weights, and DOF increases with train sample size and converges to the generalization risk for both recurrent and non-recurrent schemes.
Code generation maps a program description to executable source code in a programming language. Existing approaches mainly rely on a recurrent neural network (RNN) as the decoder. However, we find that a program contains significantly more tokens than a natural language sentence, and thus it may be inappropriate for RN…
ENRNN uses eigenvalue normalization for short-term memory in RNNs.
problem Vanishing/exploding gradient problem and long-term dependency modeling.
method Eigenvalue normalization of recurrent matrix to simulate short-term memory.
result ENRNN outperforms existing RNN variants in experiments.
Bayesian units improve speech recognition with minimal parameters.
problem Improving speech recognition models with fewer parameters.
method Derived Bayesian recurrent units integrated into deep learning frameworks.
result Adding Bayesian units improves speech recognition performance.
Proposes a new Deep Recurrent Double Q-Learning model for Atari games.
problem Improving Deep Reinforcement Learning models for Atari games.
method Integrates Double Q-Learning and Recurrent Networks (LSTM, DRQN) for Atari games.
result Demonstrates improved performance in Atari games compared to existing models.
Bayesian approach improves neural network recurrence.
problem Improving neural network recurrence mechanisms.
method Introducing Bayesian recurrence relations and gates.
result Bayesian approach can perform as well as or better than conventional recurrent networks.