This paper successfully implements Neural Turing Machines for sequence learning tasks.
problem Implementing stable and high-performing Neural Turing Machines.
method Detailed description and successful implementation of NTMs, focusing on memory contents initialization.
result Memory contents initialized to small constant values converge 2 times faster.
DAM with MRL improves relational reasoning in MANNs.
problem Limited performance of associative memory networks on complex relational reasoning tasks.
method Distributed Associative Memory architecture with Memory Refreshing Loss.
result Enhanced relation reasoning performance of MANNs on long temporal sequence data.
Paper classifies brain signals using eigenvalues for 2D and 3D educational content questions.
problem Classifying brain signals for 2D and 3D educational content questions.
method Eigenvalues of covariance matrix used as features; KNN and SVM classifiers applied.
result No significant difference in learning, memory retention, and recall between 2D and 3D educational content.
HMN detects fake faces using memory networks and visual cues.
problem Detecting fake faces from unseen manipulation types.
method Hierarchical Memory Network (HMN) architecture.
result Superior performance in fake and fraudulent face detection.
Cascades of information-sharing are a primary mechanism by which content reaches its audience on social media, and an active line of research has studied how such cascades, which form as content is reshared from person to person, develop and subside. In this paper, we perform a large-scale analysis of cascades on Faceb…
Study on heat content for domains with fractal boundaries.
problem Analyzing short-time asymptotics of heat content for domains with fractal boundaries.
method Developing mathematical analysis on de Gennes' hypothesis and exploring fractal curvatures.
result Fractal curvatures and their scaling exponents may emerge in the short-time heat content asymptotics of domains with fractal boundaries.
A new approach uses deep learning to manage vehicular content efficiently.
problem Managing content replication and caching in vehicular networks efficiently.
method Data-driven, centralized approach using a Convolutional Neural Network (CNN).
result Effective strategies derived to modulate FC operation in space and adapt to mobility changes.
Efficient sparse attention reduces self-attention complexity and improves model performance.
problem Quadratic compute and memory requirements of self-attention for long sequences.
method Content-based sparse attention with dynamic routing module.
result Routing Transformer achieves state-of-the-art performance on various benchmarks.
A new approach to PCGML uses discriminative models to improve design control.
problem Fundamental tension between detailed training examples and return on investment.
method Discriminative models trained on positive and negative examples.
result Demonstrates new mode of control for learning-based generators.
Memory-Augmented Meta-Optimization improves cold-start recommendation.
problem Cold-start problem in recommender systems for new users or items.
method Memory-Augmented Meta-Optimization approach with personalized and task-specific memories.
result Significant improvement in cold-start recommendation performance on multiple datasets.
Derives an empirical capacity model for self-attention neural networks.
problem Theoretical capacity of large transformer models is not fully utilized by current optimization algorithms.
method Analyzes memory capacity of transformers using synthetic training data and common training algorithms.
result Derives an empirical capacity model (ECM) for a generic transformer.
Deep networks retain initial bias after training, affecting generalization.
problem Understanding how much initial bias in neural networks survives training.
method Introduced initialization memory to measure initial bias's survival.
result SGD can preserve initial bias, while Adam-family methods erase it.
Study heat content in sub-Riemannian manifolds, obtaining asymptotic expansion.
problem Heat content in sub-Riemannian manifolds with non-characteristic domains.
method Fourth-order asymptotic expansion, combining rough boundary temperature and stochastic completeness.
result Obtained a fourth-order asymptotic expansion for relative heat content.
New study shows personalized content recommendations can lead to polarization of user preferences.
problem Personalized content recommendations can alter user preferences, leading to polarization.
method Used a model of preference dynamics to explore how personalized content affects user preferences.
result Standard reward maximization algorithms achieve only constant regret in personalized recommendation environments.
Designing AI market for content creation
problem Balancing technological progress and individual incentives for content creation
method Dynamic Stackelberg game model
result Inducing greater reliance on AI-assisted creation
DEMUD-VIS detects novel image content and explains it visually.
problem Detecting and explaining novel image content in large datasets.
method Uses CNN for feature extraction, reconstruction error for novelty detection, and up-convolutional networks for image reconstruction.
result Demonstrates visual explanations of novel image content on diverse datasets.
HOPE improves SSMs for long-memory tasks with robust initialization and training.
problem Improving state-space models for long-memory tasks with robust initialization and training.
method Developed a new parameterization scheme called HOPE using Hankel operators and Markov parameters.
result HOPE improves SSMs' performance on Long-Range Arena tasks and demonstrates non-decaying memory.
PARMESAN learns from memory without parameters for fast, efficient continual learning.
problem Inflexibility in deep learning methods for continual learning.
method Transductive reasoning and memory search for parameter-free learning.
result 3-4 orders of magnitude faster than baselines, comparable performance.
One of the core tasks in multi-view learning is to capture relations among views. For sequential data, the relations not only span across views, but also extend throughout the view length to form long-term intra-view and inter-view interactions. In this paper, we present a new memory augmented neural network model that…
An associative memory is a framework of content-addressable memory that stores a collection of message vectors (or a dataset) over a neural network while enabling a neurally feasible mechanism to recover any message in the dataset from its noisy version. Designing an associative memory requires addressing two main task…
Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the sam…
Linear RNNs exhibit a bias towards shorter memory due to initialization variance.
problem Understanding the performance limitations of RNNs, especially linear ones.
method Kernel regime analysis to show equivalence to 1D-convolutional networks and analyze weightings.
result Linear RNNs with random initialization have a bias towards shorter memory periods.
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Dynamic memory prevents forgetting in continuous learning of medical images.
problem Catastrophic forgetting in machine learning models over time due to domain shifts.
method Dynamic memory to store and replay diverse training data subsets.
result Dynamic memory mitigates forgetting without knowing when shifts occur.
DA-LSTM adapts LSTM depth to non-uniform data, improving efficiency.
problem Non-uniform information distribution in sequential data cannot be accurately modeled by traditional LSTM.
method Developed DA-LSTM architecture that dynamically adjusts LSTM depth based on information distribution.
result DA-LSTM reduces computation resource usage and convergence time by 41.78% and 46.01% respectively.
Spaced repetition is a technique for efficient memorization which uses repeated, spaced review of content to improve long-term retention. Can we find the optimal reviewing schedule to maximize the benefits of spaced repetition? In this paper, we introduce a novel, flexible representation of spaced repetition using the …
We propose to study market efficiency from a computational viewpoint. Borrowing from theoretical computer science, we define a market to be \emph{efficient with respect to resources S} (e.g., time, memory) if no strategy using resources S can make a profit. As a first step, we consider memory-m strategies whose a…
Proposes D2D-LSTM for predicting mobile social network content diffusion paths.
problem Lack of accurate content popularity prediction considering time and location in mobile social networks.
method D2D-LSTM, a deep neural network combining user social features and files features.
result Significantly improved prediction accuracy (up to 85.858%) and faster convergence (less than 100 steps).
A framework for multi-agent learning improves coordination through a memory-driven communication protocol.
problem Coordination and synchronisation in multi-agent systems with limited observations.
method A memory-driven communication protocol learned concurrently with individual policies during training.
result Superior performance in small-scale systems with up to six agents, demonstrating improved coordination.
Study shows how heat leaks from material sets in low diffusivity scenarios.
problem Understanding heat leakage from material sets in low diffusivity limits.
method Generalized leading-order asymptotics for time-dependent diffusion processes.
result Diffusive transport out of a material set is proportional to the surface area of the set boundary.
New initialization techniques improve the performance and speed of EMI sensor-based object discrimination.
problem Improving the performance and speed of EMI sensor-based object discrimination.
method Proposed and evaluated new initialization techniques for MI-ACE.
result Comparison of initialization approaches shows improved performance and speed.
We present an LDA approach to entity disambiguation. Each topic is associated with a Wikipedia article and topics generate either content words or entity mentions. Training such models is challenging because of the topic and vocabulary size, both in the millions. We tackle these problems using a novel distributed infer…
Anti-phishing aims to detect phishing content/documents in a pool of textual data. This is an important problem in cybersecurity that can help to guard users from fraudulent information. Natural language processing (NLP) offers a natural solution for this problem as it is capable of analyzing the textual content to per…
In recommender systems, cold-start issues are situations where no previous events, e.g. ratings, are known for certain users or items. In this paper, we focus on the item cold-start problem. Both content information (e.g. item attributes) and initial user ratings are valuable for seizing users' preferences on a new ite…
Memory-augmented neural networks (MANNs) refer to a class of neural network models equipped with external memory (such as neural Turing machines and memory networks). These neural networks outperform conventional recurrent neural networks (RNNs) in terms of learning long-term dependency, allowing them to solve intrigui…
A new model decomposes equity returns and volatilities into memory components.
problem Understanding long-term equity dynamics and volatility patterns.
method Proposes a multivariate generalization of the variance ratio to decompose long-horizon equity dynamics.
result Identifies a five-factor model capturing persistent, antipersistent, and multi-scale memory in returns and volatility.
In earlier work, carrying out numerical simulations of the Ricci flow of families of rotationally symmetric geometries on S3, we have found strong support for the contention that (at least in the rotationally symmetric case) the Ricci flow for a ``critical'' initial geometry - one which is at the transition point bet…
System identifies language of transliterated text.
problem Users struggle to understand non-native language transliterated text.
method Feature extraction of phonetic syllables using LSTM network.
result System accurately identifies language of transliterated text.
Intelligence emerges from stabilizing invariant cycles in memory.
problem Understanding the nature of intelligence and its emergence.
method Structural-dynamical account rooted in a topological closure law: \(\partial^2=0\).
result Memory-amortized inference (MAI) mechanism that implements SbS \(
ightarrow\) CCUP.
Financial frequency combs emerge from macroeconomic long-range memory.
problem Financial economy's long-run cyclic structure
method Incommensurate fractional-order financial model
result Frequency comb structure in steady-state spectrum
Transformers recall from long distributions with statistical guarantees.
problem Designing Transformers that can recall from arbitrarily long, distributional contexts.
method Recast associative memory as probability measures, decomposing the task into recall and prediction.
result A shallow measure-theoretic Transformer learns the recall-and-predict map under spectral assumptions.
A deep historical LSTM model improves tennis shot recognition from RGB videos.
problem Improving action recognition from RGB video input for sports analysis.
method Convolutional Neural Network (CNN) and Weighted Long Short-Term Memory (LSTM) for 3D tennis shot recognition.
result The method achieves better performance than state-of-the-art baselines for tennis shot recognition.
A new active learning method for skewed data sets.
problem Severe class imbalance and small initial training data in sequential active learning.
method HAL: a hybrid active learning algorithm balancing labeled and unlabeled data.
result HAL makes better choices for labeling points than strong baselines.
Memory-augmented neural networks improve machine translation performance.
problem Improving machine translation accuracy and flexibility.
method Evaluation of Neural Turing Machines and Differentiable Neural Computers for machine translation tasks.
result Memory-augmented neural networks perform similarly to attentional encoders on Vietnamese to English tasks but have lower BLEU scores on Romanian to English tasks.
In this paper we outline initial concepts for an immune inspired algorithm to evaluate price time series data. The proposed solution evolves a short term pool of trackers dynamically through a process of proliferation and mutation, with each member attempting to map to trends in price movements. Successful trackers fee…
In recent years, content recommendation systems in large websites (or \emph{content providers}) capture an increased focus. While the type of content varies, e.g.\ movies, articles, music, advertisements, etc., the high level problem remains the same. Based on knowledge obtained so far on the user, recommend the most d…
On many social networking web sites such as Facebook and Twitter, resharing or reposting functionality allows users to share others' content with their own friends or followers. As content is reshared from user to user, large cascades of reshares can form. While a growing body of research has focused on analyzing and c…
Machine learning improves learning and memory retention by optimizing study sessions.
problem Improving learning and memory retention methods for factual material.
method Large-scale randomized controlled trial with machine learning optimization of study sessions.
result Study sessions optimized with machine learning lead to 67% longer retention and 50% higher return rate.