Memory-augmented neural networks improve machine translation performance.
problem Improving machine translation accuracy and flexibility.
method Evaluation of Neural Turing Machines and Differentiable Neural Computers for machine translation tasks.
result Memory-augmented neural networks perform similarly to attentional encoders on Vietnamese to English tasks but have lower BLEU scores on Romanian to English tasks.
AuGMEnT network struggles with long-term memory for hierarchical tasks.
problem Learning and memory in neural networks, especially hierarchical tasks.
method Introduced hybrid AuGMEnT with leaky and non-leaky memory units.
result Hybrid AuGMEnT solves hierarchical and distractor tasks.
New memory in neural networks mimics computer architectures.
problem Learning algorithms and complex tasks with neural networks.
method Introducing a new memory to store weights for a neural controller, similar to stored-program memory in computers.
result Neural Stored-program Memory enhances neural networks' adaptability and learning capabilities.
Memory augmented neural networks improve long-term dependency learning.
problem Vanishing gradients in RNNs for long-term dependency tasks.
method TARDIS model with wormhole connections to external memory.
result Memory helps propagate gradients effectively, improving learning.
Metalearned neural memory improves learning across various tasks.
problem Improving neural network adaptability and memory function.
method Augmenting neural networks with a metalearned external memory mechanism.
result The model achieves strong performance on diverse learning problems.
MEMGAN uses memory to improve anomaly detection by isolating abnormal data.
problem Weak guarantees for detecting anomalous data in classical algorithms.
method Memory-augmented Generative Adversarial Networks (MEMGAN) with a memory module.
result MEMGAN provides strong guarantees for anomaly detection with improved reconstruction.
MemNet optimizes neural architectures for memory efficiency.
problem Memory constraints in mobile devices limit the use of large neural networks.
method Augment-trim learning with memory consumption ranking score.
result MemNet finds architectures with 24.17% less memory usage compared to state-of-the-art methods.
ARMIN improves memory efficiency and lightness in neural networks.
problem Complex memory addressing and inefficient memory information exploitation in previous MANNs.
method ARMIN uses hidden state for automatic memory addressing and a novel RNN cell for memory integration.
result ARMIN achieves lower computational overhead and similar performances compared to vanilla LSTM.
Dual neural network improves treatment recommendations from medical history.
problem Improving treatment recommendations from patient medical history.
method Memory-augmented neural network with dual controllers.
result Dual controller write-protected memory-augmented neural network outperformed traditional methods.
This paper tackles quantizing memory-augmented neural networks, improving their performance and energy efficiency.
problem Quantizing memory-augmented neural networks (MANNs) poses unique challenges not seen in other network types.
method Identified memory addressing as the main issue, proposed a robust quantization method for MANNs.
result Achieved a 22x computation-energy gain with 8-bit fixed-point and binary quantization.
This paper successfully implements Neural Turing Machines for sequence learning tasks.
problem Implementing stable and high-performing Neural Turing Machines.
method Detailed description and successful implementation of NTMs, focusing on memory contents initialization.
result Memory contents initialized to small constant values converge 2 times faster.
Memory augmented neural networks improve active learning for one-shot predictions.
problem Scarcity and cost of labeled training data in deep architectures.
method Memory augmented neural networks and Class Margin Sampling (CMS) for reinforcement learning.
result The proposed method outperforms existing baselines in label predictions and reduces label requests.
LMNs enhance neural networks with memory to adapt online.
problem Online adaptation of neural networks for domain-relevant data.
method LMNs use label-based memory replacement and write to memory only for instances with non-zero loss.
result Significant accuracy gains on various tasks including word-modelling and few-shot learning.
LEMN improves memory networks for learning from streaming data.
problem Efficiently processing large data streams in memory-augmented neural networks.
method LEMN uses a RNN-based retention agent to learn and replace less important memory entries based on their importance and historical context.
result LEMN achieves significant improvements over existing methods in learning from streaming data.
Neural networks improve language comprehension by testing and refining hypotheses.
problem Improving language comprehension models through more sophisticated reasoning.
method Memory augmented neural networks with a hypothesis testing loop.
result Achieved state-of-the-art results on language comprehension benchmarks.
A new model learns preferences incrementally without personal data.
problem Incremental session-based recommendation without personal data.
method Memory Augmented Neural model (MAN) that combines a neural recommender with a nonparametric memory.
result MAN consistently outperforms existing methods in incremental session-based recommendation.
Neural Semantic Encoders improve natural language tasks with memory updates.
problem Natural language understanding and task performance improvements.
method Memory augmented neural network with variable sized encoding memory and read, compose, write operations.
result Achieved state-of-the-art performance on five natural language tasks.
GAMENet uses graph memory networks to recommend safe medication combinations.
problem Lack of personalized and safe medication combination recommendations considering patient history and drug interactions.
method Integrates drug-drug interactions knowledge graph into a memory module using graph convolutional networks and models longitudinal patient records.
result GAMENet outperformed state-of-the-art methods in effectiveness and safety, reducing DDI rate by 3.60%.
New method optimizes memory usage in neural networks, improving sequential learning.
problem Current memory models in neural networks waste memory and computation.
method Formulated an optimization problem to maximize information storage, introduced Cached Uniform Writing.
result Proved Cached Uniform Writing optimizes memory usage and outperforms other methods.
MAES architecture improves working memory task performance through multi-task and transfer learning.
problem Improving performance on complex working memory tasks.
method Memory-Augmented Encoder-Solver (MAES) architecture with dual recurrent neural network controllers and a shared memory module.
result MAES models achieve task-size generalization, handling inputs 50 times longer than training data.
New model adds persistent memory to self-attention layers for improved performance.
problem Improving transformer performance by removing feed-forward layers.
method Augmenting self-attention layers with persistent memory vectors.
result The model outperforms standard transformers on language modeling benchmarks.
Memory-Augmented Recurrent Networks improve dialogue coherence by expanding conversation history storage.
problem Fixed-size vectors limit dialogue coherence; attention mechanisms are computationally expensive.
method Introduce Neural Turing Machines (NTMs) to provide flexible and permanent storage for dialogue history.
result Improved perplexity performance compared to existing baselines.
A new method reduces memory requirements for Graph Transformers by sparsely training a network.
problem Quadratic memory complexity in Graph Transformers limits their scalability to large graphs.
method Spexphormer: trains a narrow network on augmented graph, then uses only active connections in a wider network.
result Spexphormer achieves good performance with drastically reduced memory requirements.
Accelerator boosts energy efficiency for MANNs on FPGAs.
problem Efficiently running MANNs on accelerators designed for other NNs.
method Data flow architecture, inference thresholding.
result Higher energy efficiency compared to NVIDIA GPU.
Memory-enhanced self-play leads to faster exploration and better performance.
problem Faster exploration and better performance in reinforcement learning environments.
method Memory augmentation in self-play to store and reuse experience.
result Memory-augmented self-play agents outperform standard self-play agents.
Paper shows how LSTM can remember long sequences by attending to persisted information.
problem LSTMs struggle with long sequences due to fading information and bias towards recent data.
method The paper introduces a mechanism that allows LSTMs to attend to information in memory based on how long it was persisted by the gating mechanism.
result The method improves LSTM's ability to process long sequences by retrieving information proportionally to its persistence in memory.
Global Memory Augmentation (GMAT) improves Transformer performance on long documents.
problem Large memory requirements of Transformer pairwise dot-product attention for long sequences.
method Integrates a dense global memory of length M into sparse Transformer blocks.
result Significant improvement on various tasks, including synthetic tasks, masked language modeling, and reading comprehension.
New memory architecture improves meta-learning performance in small datasets.
problem Meta-learning challenges with small datasets and memory interference.
method Introduced a Feature-Label Memory Network (FLMN) that splits memory into feature and label components.
result FLMN outperforms MANN in supervised one-shot classification tasks.
Memory-enhanced model predicts tennis shots based on player history.
problem Predicting shot location and type in tennis.
method Semi-supervised Generative Adversarial Network with neural memory modules.
result The model learns player-specific behavioral patterns from match data.
System builds Somali ASR for UN humanitarian efforts.
problem Developing ASR for under-resourced Somali language.
method Acoustic model training with annotated speech, neural architectures, language model data augmentation, acoustic data perturbation.
result Best system achieved 53.75% word error rate.
Paper proposes a new neural network for predicting machinery RUL.
problem Accurately estimating the remaining useful life of industrial machinery.
method Temporal Convolutional Memory Networks incorporating long and short term dependencies.
result Demonstrates superior performance compared to state-of-the-art algorithms.
A new memory replay mechanism improves reinforcement learning stability and speed.
problem Forgetting in reinforcement learning with continuous control.
method Augmented Memory Replay (AMR) that optimizes the replay of past experiences.
result AMR enhances stability and convergence speed of learning algorithms.
Transformers can perform well with less long-range memory.
problem The need for deep long-range memory in Transformers for language modeling.
method Performed interventions to show performance can be achieved with fewer long-range memories and by limiting attention range.
result Comparable performance can be achieved with 6X fewer long-range memories and better performance with limited attention range.
CLA improves investment accuracy by leveraging past knowledge.
problem Improving investment decisions through past knowledge integration.
method CLA uses an explicit memory structure and FFNN base model, incorporating change points and contextual similarity.
result CLA significantly outperforms FFNN base models in expected return forecasting.
DAM with MRL improves relational reasoning in MANNs.
problem Limited performance of associative memory networks on complex relational reasoning tasks.
method Distributed Associative Memory architecture with Memory Refreshing Loss.
result Enhanced relation reasoning performance of MANNs on long temporal sequence data.
Tree-SMU enables strong compositional generalization in neural networks.
problem Zero-shot generalization to novel compositions of concepts.
method Tree Stack Memory Units (Tree-SMU) with Stack Memory Units (SMU).
result Tree-SMU achieves strong empirical results on mathematical reasoning benchmarks.
This paper uses deep learning to improve memory prefetching.
problem Improving memory performance in software optimizations.
method Relating prefetching strategies to n-gram models and using recurrent neural networks.
result Neural networks consistently outperform traditional prefetching strategies in benchmark tests.
NHC learns scalable algorithmic solutions from diverse tasks.
problem Neural networks struggle to learn algorithmic strategies.
method Memory-augmented network with abstraction mechanism and evolutionary training.
result Reliable learning of robust and scalable algorithmic solutions.
CMT efficiently manages memory by inserting and querying memories in logarithmic time.
problem Managing large memory stores efficiently for quick access and updates.
method Designing a Contextual Memory Tree (CMT) that inserts and retrieves memories in logarithmic time.
result CMT improves classification algorithms and image-captioning tasks, demonstrating better computational efficiency.
Deep neural networks estimate long memory parameters efficiently.
problem Estimating long memory parameters in stochastic processes.
method Scale-invariant 1D Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) models trained with synthetic data.
result Neural models outperform conventional methods in precision, speed, consistency, and robustness.
A simplified neural network with fast memory improves performance on rare class tasks.
problem Neural networks struggle with rare class identification.
method Augmenting with a fast-learning non-parametric model storing recent activations and class labels.
result Improved performance on word-based language models and rare class tasks.
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
problem Memory models struggle in partially observable reinforcement learning environments.
method Introduces a novel memory model using the Hadamard product for efficient memory management and updates.
result Significantly outperforms state-of-the-art memory-based methods on challenging benchmarks.
LSTM-FCN improves time series classification with minimal model size increase.
problem Classifying time series sequences with high accuracy.
method Augmented fully convolutional networks with LSTM sub-modules and attention mechanism.
result LSTM-FCN achieves state-of-the-art performance.
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
LSTMs fail in financial tasks, CL improves performance.
problem Financial time-series analysis and inference with LSTMs.
method Continual Learning (CL) approach for financial decision making.
result CL outperforms LSTMs and FFNN in financial decision making.
Memory-Augmented Meta-Optimization improves cold-start recommendation.
problem Cold-start problem in recommender systems for new users or items.
method Memory-Augmented Meta-Optimization approach with personalized and task-specific memories.
result Significant improvement in cold-start recommendation performance on multiple datasets.
Algorithm optimizes millions of hyperparameters efficiently.
problem Training modern network architectures with millions of hyperparameters.
method Combines implicit function theorem with efficient inverse Hessian approximations for gradient-based optimization.
result Jointly tuning weights and hyperparameters is only a few times more costly than standard training.
SARN improves relational reasoning with less computation.
problem Efficiently perform relational reasoning with reduced computation.
method Introduces SARN, a sequential attention relational network.
result SARN achieves high accuracy on relational questions.