A novel memory network improves interpretability and training for EHR analysis.
problem Limited interpretability and training difficulty in traditional memory networks for EHR analysis.
method Proposes a hybrid memory architecture with explicit and blurred memory blocks for improved interpretability and training efficiency.
result The model performs comparably to state-of-the-art methods and enables ready interpretation of results.
DRLViz interprets deep RL agent memory for better understanding.
problem Understanding complex deep RL agent memory.
method Visual analytics interface to reduce and interpret memory vectors.
result Experts can better understand and investigate agent decisions.
V-HMN integrates memory mechanisms for improved image recognition.
problem Limited interpretability and high data requirements of existing vision backbones.
method Brain-inspired hierarchical memory modules with iterative refinement.
result V-HMN achieves strong performance on image classification benchmarks.
A novel memory mechanism for reinforcement learning agents that stores past events in human-readable language.
problem Lack of interpretability in reinforcement learning agent's memory mechanisms.
method Uses CLIP to associate visual inputs with language tokens, then feeds these tokens to a pretrained language model.
result Significantly faster convergence on challenging continuous recognition tasks.
MEMGAN uses memory to improve anomaly detection by isolating abnormal data.
problem Weak guarantees for detecting anomalous data in classical algorithms.
method Memory-augmented Generative Adversarial Networks (MEMGAN) with a memory module.
result MEMGAN provides strong guarantees for anomaly detection with improved reconstruction.
FF layers in transformers are nearly as interpretable as sparse autoencoders.
problem Comparing interpretability of feature vectors in FF layers vs. sparse autoencoders.
method Revisited interpretability of FF layers as key-value memories using modern benchmarks.
result FF and SAE feature vectors are similarly interpretable, but FFs can be better in some aspects.
Generative diffusion models mimic biological memory networks, encoding associative dynamics in deep neural weights.
problem Understanding long-term memory mechanisms in neuroscience and AI.
method Interpreting generative diffusion models as energy-based models and comparing them to Hopfield networks.
result Generative diffusion models can encode associative dynamics of Hopfield networks in deep neural weights.
Efficient sparse modern Hopfield models are introduced for memory retrieval and learning tasks.
problem Efficient modern Hopfield models for memory retrieval and learning tasks.
method Nonparametric interpretation of Hopfield models as a regression problem.
result Sparse-structured modern Hopfield models with sub-quadratic complexity.
MAYA learns bee foraging decisions with limited memory.
problem Reproducing and predicting bees' foraging decisions with limited memory.
method Sequential imitation learning model based on multi-armed bandits, considering a temporal window τ of 7 trials.
result MAYA outperforms imitation baselines and classical models, providing interpretability and realistic trajectories.
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked …
Typical neural networks with external memory do not effectively separate capacity for episodic and working memory as is required for reasoning in humans. Applying knowledge gained from psychological studies, we designed a new model called Differentiable Working Memory (DWM) in order to specifically emulate human workin…
The paper studies scaling laws for associative memory mechanisms.
problem Understanding and optimizing learning and memorization processes.
method High-dimensional matrices of outer products of embeddings, relating to transformer models. Derived scaling laws with sample and parameter sizes. Extensive numerical experiments.
result Precise scaling laws and statistical efficiency of estimators.
The ability to look multiple times through a series of pose-adjusted glimpses is fundamental to human vision. This critical faculty allows us to understand highly complex visual scenes. Short term memory plays an integral role in aggregating the information obtained from these glimpses and informing our interpretation …
A new model for sequential memory using temporal predictive coding.
problem Forming accurate memory of sequential stimuli in the brain.
method Proposes a novel PC-based model called temporal predictive coding (tPC).
result Shows that tPC models can accurately memorize and retrieve sequential inputs.
Adaptive Bayesian learning agent for non-stationary bandits.
problem Non-stationary rewards in reinforcement learning.
method Dynamic memory and statistical hypothesis testing.
result Adapts to changing rewards with minimal regret.
A technique identifies memoryless algorithms approximating memory-dependent optimization methods.
problem Understanding how memory in optimization algorithms affects loss and generalization.
method Introducing a general technique to replace past iterates with the current one and adding a correction term.
result Lion does not have the same implicit anti-regularization as AdamW, explaining its better generalization performance.
Despite the huge success of Long Short-Term Memory networks, their applications in environmental sciences are scarce. We argue that one reason is the difficulty to interpret the internals of trained networks. In this study, we look at the application of LSTMs for rainfall-runoff forecasting, one of the central tasks in…
A new method for detecting anomalies in large, high-dimensional data streams using probabilistic forest models.
problem Challenges in detecting anomalies in large, high-dimensional data.
method Probabilistic Mondrian Pólya Forests for summarizing data and estimating underlying probability density.
result State-of-the-art performance with interpretable anomaly scores.
FactorMiner discovers financial alpha factors with low redundancy.
problem Finding novel financial alpha factors in a vast search space.
method Modular Skill Architecture and Experience Memory to distill and guide exploration.
result FactorMiner constructs a diverse library of high-quality factors with competitive performance.
How can we find patterns and anomalies in a tensor, or multi-dimensional array, in an efficient and directly interpretable way? How can we do this in an online environment, where a new tensor arrives each time step? Finding patterns and anomalies in a tensor is a crucial problem with many applications, including buildi…
Memory-based models can learn to approximate Bayes-optimal predictors for non-stationary data.
problem Learning from non-stationary data with unobserved switching points.
method Memory-based neural models, including Transformers, LSTMs, and RNNs, trained to minimize log loss.
result Memory-based models can accurately approximate known Bayes-optimal algorithms and perform Bayesian inference over latent switching points.
Study shows adding correlated features doesn't improve LSTM model interpretability for oil stocks.
problem Improving interpretability of LSTM models for predicting oil company stocks.
method Designed and trained Standard LSTM networks using various correlated datasets.
result Adding correlated features does not enhance LSTM model interpretability.
The usual derivation of the Fokker-Planck partial differential eqn. assumes the Chapman-Kolmogorov equation for a Markov process. Starting instead with an Ito stochastic differential equation we argue that finitely many states of memory are allowed in Kolmogorov's two pdes, K1 (the backward time pde) and K2 (the Fokker…
Two new algorithms speed up TreeSHAP computation for tree-based models.
problem Slow computation of SHAP values on tree-based models.
method Two new algorithms, Fast TreeSHAP v1 and v2, designed to improve computational efficiency.
result Fast TreeSHAP v2 is 2.5x faster than TreeSHAP, with slightly higher memory usage.
Tensor models decode human perception and memory using SPO triples.
problem Understanding implicit and explicit perception and memory in the brain.
method Tensor models with SPO triples, dual representations, and four layers.
result Semantic memory is crucial for explicit perception and declarative memories.
A novel online GP model captures long-term memory in sequential data.
problem Capturing long-term memory in sequential data online.
method Integrates HiPPO framework into interdomain GP, leveraging time-varying orthogonal projections as inducing variables.
result OHSVGP outperforms existing online GP methods in predictive performance, long-term memory preservation, and computational efficiency.
Multivariate time series forecasting is extensively studied throughout the years with ubiquitous applications in areas such as finance, traffic, environment, etc. Still, concerns have been raised on traditional methods for incapable of modeling complex patterns or dependencies lying in real word data. To address such c…
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
SSINNs learn Hamiltonian systems from data with interpretable, low-memory models.
problem Learning Hamiltonian dynamical systems from data efficiently and accurately.
method Combines fourth-order symplectic integration with sparse regression for a learned Hamiltonian.
result Outperforms state-of-the-art techniques in system prediction and energy conservation.
New linear models improve time series classification efficiency and interpretability.
problem Complex and inefficient classifiers limit interpretability and applicability to variable-length time series.
method Symbolic representations, multi-resolution, multi-domain, linear models.
result mtSS-SEQL+LR achieves similar accuracy to state-of-the-art methods but with lower time and memory usage.
Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems. RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features. In …
The choice of how to retain information about past gradients dramatically affects the convergence properties of state-of-the-art stochastic optimization methods, such as Heavy-ball, Nesterov's momentum, RMSprop and Adam. Building on this observation, we use stochastic differential equations (SDEs) to explicitly study t…
Exponential growth in Electronic Healthcare Records (EHR) has resulted in new opportunities and urgent needs for discovery of meaningful data-driven representations and patterns of diseases in Computational Phenotyping research. Deep Learning models have shown superior performance for robust prediction in computational…
RFFNet scales kernel methods to large datasets by learning kernel relevance.
problem Scaling kernel methods to large datasets while maintaining interpretability.
method Designs random Fourier features for ARD kernels and uses first-order stochastic optimization for learning kernel relevances.
result RFFNet achieves low prediction error and identifies relevant features, leading to more interpretable solutions.
METEOR learns efficient representations from multi-modal data streams.
problem Efficiently interpreting multi-modal information in complex environments.
method METEOR learns compact representations by sharing parameters within semantically meaningful groups and preserving domain-agnostic semantics.
result METEOR reduces memory usage by around 80% compared to conventional methods.
A brain-inspired spiking Transformer reduces energy consumption and enhances interpretability.
problem Energy inefficiency and lack of interpretability in Transformer models.
method Spiking STDP Transformer using spike-timing-dependent plasticity (STDP) for self-attention.
result Achieves 94.35% and 78.08% accuracy on CIFAR-10 and CIFAR-100 datasets respectively, with 88.47% energy reduction.
The study improves Bitcoin price prediction using hybrid machine learning and enhances interpretability.
problem Improving Bitcoin price prediction accuracy and interpretability.
method Hybrid machine learning algorithms (OLS, LASSO, LSTM, decision tree regressors) and preprocessing techniques for time-series data.
result Linear regression achieves the best performance in predicting Bitcoin prices.
Feature selection is an important challenge in machine learning. It plays a crucial role in the explainability of machine-driven decisions that are rapidly permeating throughout modern society. Unfortunately, the explosion in the size and dimensionality of real-world datasets poses a severe challenge to standard featur…
We introduce a novel description of the dynamics of the order book of financial markets as that of an effective colloidal Brownian particle embedded in fluid particles. The analysis of a comprehensive market data enables us to identify all motions of the fluid particles. Correlations between the motions of the Brownian…
Recurrent neural networks (RNNs) are powerful and effective for processing sequential data. However, RNNs are usually considered "black box" models whose internal structure and learned parameters are not interpretable. In this paper, we propose an interpretable RNN based on the sequential iterative soft-thresholding al…
Momentum ResNets improve ResNets' memory efficiency.
problem Memory inefficiency in deep residual neural networks (ResNets).
method Adding a momentum term to the forward rule of ResNets to make them invertible.
result Momentum ResNets can learn any linear mapping up to a multiplicative factor, improving memory efficiency.
Researchers develop a neural network that learns like humans, overcoming forgetting and structure issues.
problem Catastrophic forgetting and structure limitations in neural networks.
method Memory playback strategy and dynamic structure extension using conditional variational autoencoder (CVAE).
result The method effectively prevents forgetting and allows for dynamic network growth.
Study uses IMFs and neural networks to predict economic time series, enhancing interpretability.
problem Improving prediction accuracy and interpretability of economic time series.
method Intrinsic Mode Functions (IMFs) derived from economic time series, combined with DeepSHAP for interpretability.
result The last IMFs are most influential, and high-frequency IMFs introduce noise.
Muon outperforms GD in associative memory learning by balancing frequency components.
problem Training dynamics and scaling behavior of Muon in associative memory learning.
method Study of Muon in a linear associative memory model with softmax retrieval and hierarchical frequency spectrum over query-answer pairs.
result Muon achieves exponential speedup over GD in noiseless case and superior scaling efficiency in noisy case.
MAC Net is a compositional attention network designed for Visual Question Answering. We propose a modified MAC net architecture for Natural Language Question Answering. Question Answering typically requires Language Understanding and multi-step Reasoning. MAC net's unique architecture - the separation between memory an…
Intelligence emerges from stabilizing invariant cycles in memory.
problem Understanding the nature of intelligence and its emergence.
method Structural-dynamical account rooted in a topological closure law: \(\partial^2=0\).
result Memory-amortized inference (MAI) mechanism that implements SbS \(
ightarrow\) CCUP.
FinMem enhances LLM trading agents with layered memory and character design.
problem Developing purpose-driven LLM agents for financial decision-making.
method Integrates layered memory and character design modules into an LLM framework.
result Significantly enhanced trading performance in financial markets.
Improved breast cancer screening with a fast, memory-efficient model.
problem Classifying high-resolution breast cancer screening images.
method Extends globally-aware multiple instance classifier to handle image-level labels.
result Achieves AUC of 0.93 in classifying malignant findings, outperforming existing methods.