A new method selects PCA components based on residual memory, outperforming existing techniques.
problem Selecting the optimal number of components in PCA for data with long memory effects.
method Sequentially removes components, stopping when maximum memory accounted for.
result Our method outperforms existing techniques in computational efficiency and accuracy.
Memory-based models can learn to approximate Bayes-optimal predictors for non-stationary data.
problem Learning from non-stationary data with unobserved switching points.
method Memory-based neural models, including Transformers, LSTMs, and RNNs, trained to minimize log loss.
result Memory-based models can accurately approximate known Bayes-optimal algorithms and perform Bayesian inference over latent switching points.
Meta-learning improves adaptability across diverse tasks.
problem Building efficient strategies that adapt to new tasks.
method Memory-based meta-learning, Bayesian framework, state-machine of sufficient statistics.
result Meta-learned strategies are near-optimal and efficient.
SMT improves robotic long-horizon tasks by embedding and utilizing past observations.
problem Long-horizon tasks in partially observable environments require effective long-term memory.
method Scene Memory Transformer (SMT) embeds and uses attention to exploit spatio-temporal dependencies.
result SMT outperforms existing policies in visual navigation tasks.
Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt…
Paper analyzes convergence of memory-based distributed SGD with momentum.
problem High communication cost in distributed SGD for large models.
method Introduces transformation equation to analyze convergence of M-DSGD with momentum.
result Universal convergence analysis for M-DSGD with momentum for convex and non-convex problems.
New schemes improve lifelong learning by balancing old and new tasks.
problem Catastrophic forgetting in deep neural networks when learning multiple tasks.
method Unified optimization perspective of episodic memory based approaches, introducing MEGA-I and MEGA-II schemes.
result Significant improvement in lifelong learning benchmarks, reducing error by up to 18%.
Study shows significant differences in recommendation bias between model-based and memory-based algorithms.
problem Recommendation bias disparity across different algorithms and item categories.
method Examined bias disparity in a range of collaborative recommendation algorithms and item categories.
result Significant differences found between model-based and memory-based algorithms.
Study identifies key MRI features for predicting cognitive performance after mTBI.
problem Identify relevant diffusion MRI metrics for cognitive functions in mTBI patients.
method Proposes a novel feature selection method combining best-first search with genetic algorithm crossover.
result Achieves significantly more accurate predictions than other feature selection algorithms.
A method to improve sequential learning by keeping past data errors in check.
problem Challenges in sequential learning with Gaussian processes due to accumulating errors.
method Memory-based dual sparse variational Gaussian processes.
result Improves accuracy in inference and learning for various applications.
Efficient memory layer improves graph neural networks for graph classification and regression.
problem Efficiently learning node representations and graph coarsening for arbitrary graph topology.
method Introduces a memory layer for GNNs that learns node representations and graph coarsening, and two new networks: MemGNN and GMN.
result Proposed models achieve state-of-the-art results in graph classification and regression benchmarks.
A new memory-based fusion layer improves multi-modal deep learning performance.
problem Improving performance of multi-modal deep learning by addressing long-term dependencies.
method Introducing a Memory based Attentive Fusion (MBAF) layer that incorporates both current and long-term dependencies.
result The MBAF layer enhances fusion and improves performance across different modalities and networks.
Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently,…
MTRGL learns temporal correlations from multi-modal data for improved pair trading.
problem Discerning temporal correlations among financial entities.
method Combines time series data and discrete features into a temporal graph, using a memory-based temporal graph neural network.
result MTRGL outperforms traditional methods in temporal graph link prediction and pair trading.
The study shows how modular learning can adapt to new tasks.
problem Adapting to new tasks in an ever-changing environment.
method Task segmentation, modular learning, memory-based ensembling.
result The system demonstrates robustness to catastrophic forgetting and increasing positive transfer.
In this paper, a novel architecture of Recurrent Neural Network (RNN) is designed and experimented. The proposed RNN adopts a computational memory based on the concept of stigmergy. The basic principle of a Stigmergic Memory (SM) is that the activity of deposit/removal of a quantity in the SM stimulates the next activi…
GMED edits stored examples to improve continual learning.
problem Catastrophic forgetting in task-free continual learning.
method Gradient-based memory editing of stored examples.
result GMED-edited examples help prevent forgetting.
New framework learns policies for partially observable systems.
problem Learning policies in partially observable dynamical systems.
method Partially Observable Bilinear Actor-Critic framework.
result Algorithm can learn against optimal policies in certain cases.
Robots adapt to damage with a single policy and diagnosis.
problem Robotic failure due to damage during mission-critical tasks.
method Damage-aware control architecture using supervised learning for diagnosis and policy adaptation.
result Single-shot diagnosis and adaptation achieved with a single policy.
BeMF improves recommendation reliability in recommender systems.
problem Improving reliability in recommender systems beyond accuracy.
method Bernoulli Matrix Factorization (BeMF) for model-based collaborative filtering.
result BeMF selects more reliable predictions, improving recommendation quality.
Real-time fuel leakage detection framework MOCPD improves accuracy.
problem Early detection of fuel leakage to prevent hazards and losses.
method Memory-based Online Change Point Detection (MOCPD) framework.
result MOCPD outperforms baseline methods in detection accuracy.
We propose a primitive called PJOIN, for "predictive join," which combines and extends the operations JOIN and LINK, which Valiant proposed as the basis of a computational theory of cortex. We show that PJOIN can be implemented in Valiant's model. We also show that, using PJOIN, certain reasonably complex learning and …
NIPA aims to translate brain learning mechanisms into scalable Bayesian inference.
problem Scalable Bayesian inference for large-scale statistical machine learning problems.
method Neural-inspired algorithm combining model-based, model-free, and episodic-control modules.
result Advances Bayesian methods and facilitates their application to deep learning.
Agent learns directed exploration policies to improve performance in hard games.
problem Improving exploration in complex games.
method Episodic memory-based intrinsic reward, self-supervised inverse dynamics, UVFA framework.
result Doubles performance in hard exploration games, achieves non-zero rewards in Pitfall!.
Study extends cognitive modeling to natural images, revealing the importance of image representation.
problem Extending cognitive modeling to natural images and understanding human categorization.
method Conducted a large-scale study with over 500,000 human judgments. Used deep and shallow machine learning methods to represent images. Applied psychological models of categorization to natural images.
result Simple models with abstract prototypes outperform complex exemplar accounts when using expressive, data-driven image representations.
With the arrival of the big data era, more and more data are becoming readily available in various real-world applications and those data are usually highly heterogeneous. Taking computational medicine as an example, we have both Electronic Health Records (EHR) and medical images for each patient. For complicated disea…
Framework for continual object recognition in egocentric settings.
problem Recognizing objects in a cold-start, open-world setting with limited supervision.
method Memory-based incremental framework using time and space persistence, similarity, and active learning.
result Feasibility of open-world, generic object recognition with complete user supervision.
Given a system of equations in a "random" finitely generated subgroup of the braid group, we show how to find a small ordered list of elements in the subgroup, which contains a solution to the equations with a significant probability. Moreover, with a significant probability, the solution will be the first in the list.…
The Industrial Internet of Things drastically increases connectivity of devices in industrial applications. In addition to the benefits in efficiency, scalability and ease of use, this creates novel attack surfaces. Historically, industrial networks and protocols do not contain means of security, such as authentication…
Meta learning with information theory and Gaussian processes.
problem Few-shot learning problems.
method Information bottleneck, mutual information, variational approximations, Gaussian processes.
result Competitive accuracy on few-shot classification problems.
New method prevents deep learning forgetting past by remembering key examples.
problem Catastrophic forgetting in continual learning.
method Functional regularisation using Gaussian Process formulation.
result Achieves state-of-the-art performance on benchmarks.
Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to perform complex relational reasoning with the information they remember. Here, we first confirm our intuitions that standard memory architectur…
Combines k-NN and RVM for improved classification accuracy.
problem Improving k-NN's performance by considering relevancy.
method Integrates k-NN and RVM in kernel space, introduces a new stopping parameter.
result Significantly prunes irrelevant attributes and improves classification accuracy.
Self-referential meta learning avoids explicit optimization by modifying itself.
problem Dependency on human engineering in meta learning algorithms.
method Investigates self-referential meta learning systems that modify themselves without explicit optimization.
result Self-referential neural networks can improve their own modifications without explicit optimization.
Hybrid framework prevents forgetting in continual learning.
problem Avoiding forgetting in learning new tasks without forgetting old ones.
method Hybrid continual learning framework combining architecture growth and experience replay.
result Hybrid approach effectively avoids forgetting across multiple tasks.
Long Short-Term Memory (LSTM) is a recurrent neural network (RNN) architecture that has been designed to address the vanishing and exploding gradient problems of conventional RNNs. Unlike feedforward neural networks, RNNs have cyclic connections making them powerful for modeling sequences. They have been successfully u…
Hybrid model predicts stock prices using ML, DL, and NLP.
problem Improving prediction accuracy of stock price movement.
method Machine learning, deep learning, natural language processing, sentiment analysis.
result LSTM model outperforms traditional machine learning models.
Paper analyzes convergence of continual learning with adaptive methods.
problem Preventing catastrophic forgetting in sequential learning tasks.
method Adaptive method for nonconvex continual learning (NCCL) adjusts step sizes of previous and current tasks.
result Proposed adaptive method achieves same convergence rate as SGD when catastrophic forgetting is suppressed.
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
problem Memory models struggle in partially observable reinforcement learning environments.
method Introduces a novel memory model using the Hadamard product for efficient memory management and updates.
result Significantly outperforms state-of-the-art memory-based methods on challenging benchmarks.
Research benchmarks LLMs in medical domain to reduce hallucinations.
problem Hallucinations in medical LLMs can lead to incorrect information.
method Developed Med-HALT dataset and testing methods.
result Significant performance differences among LLMs identified.
Paper introduces a new framework to improve sample efficiency in POMDPs learning.
problem Challenges in off-policy evaluation for POMDPs, especially with hidden states.
method Exploits the metric structure of belief space to relax coverage assumptions.
result Unified analysis technique yields tighter error bounds and sample efficiency improvements.
MEM learns set functions from permutation-invariant data.
problem Learning from sets of instances with labels only on sets, not instances.
method Memory-based Exchangeable Model (MEM) with self-attention mechanism.
result Achieved 84.84% accuracy on lung cancer classification.
Real time application of deep learning algorithms is often hindered by high computational complexity and frequent memory accesses. Network pruning is a promising technique to solve this problem. However, pruning usually results in irregular network connections that not only demand extra representation efforts but also …
DCRNN improves LSTM for chaotic dynamical system forecasting.
problem Modeling chaotic dynamical systems with recurrent neural networks.
method DCRNN incorporates learnable skip-connections and a Lyapunov stability regularization term.
result DCRNN outperforms LSTM in 100 out of 100 experiments, reducing mean squared error by 80.0%.
Study investigates how simple speech sounds can form abstract categories.
problem How do abstract categories like phonemes emerge from speech exposure?
method Used modeling techniques to test Memory-Based Learning and Error-Correction Learning.
result Error-Correction Learning models can learn abstractions, identifying phone inventory and grouping.
This work improves reinforcement learning with sparse rewards by following diverse past trajectories.
problem Challenges in reinforcement learning with sparse rewards and myopic behavior.
method Proposes a trajectory-conditioned policy to learn from a memory buffer of diverse past trajectories.
result Significantly outperforms existing methods on complex tasks with local optima.
AI predicts dementia onset from emotional face evaluations.
problem Early detection of dementia in aging societies.
method Behavioral responses analysis and AI regression.
result Encouraging AI-based prediction results for MoCA scores.
Time series forecasting is ubiquitous in the modern world. Applications range from health care to astronomy, and include climate modelling, financial trading and monitoring of critical engineering equipment. To offer value over this range of activities, models must not only provide accurate forecasts, but also quantify…