arXiv research

Memory-efficient learning for large-scale imaging systems.

The paper studies scaling laws for associative memory mechanisms.

This study examines memory effects in S&P500 market correlations using Langevin models.

Linear memory stores associations up to a logarithmic scale, but listwise retrieval can handle a quadratic scale.

Analyzed a generalized voter model with power-law herding intensity, revealing anomalous diffusion and long-range memory.

Extended LSTM improves language modeling performance.

We investigate the probability distribution of the volatility return intervals $τ$ for the Chinese stock market. We rescale both the probability distribution $P_{q}(τ)$ and the volatility return intervals $τ$ as $P_{q}(τ)=1/\barτ f(τ/\barτ)$ to obtain a uniform scaling curve for different threshold value $q$. The scali…

Improved recurrent neural networks learn long-term dependencies through multi-scale memory.

Efficiently scales continuous kernels with sparse Fourier domain learning.

Novel algorithm speeds up log-determinant estimation for large matrices.

Improved PQM for pattern classification on quantum computers.

SDCOR clusters massive datasets efficiently, detecting outliers with low memory usage.

Enhances sequence memory capacity in neural networks.

Online method learns sparse models efficiently in large scale settings.

Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can handle long sequences but suffers from the vanishing and exploding gradient probl…

Matrix Factorization (MF) on large scale matrices is computationally as well as memory intensive task. Alternative convergence techniques are needed when the size of the input matrix is higher than the available memory on a Central Processing Unit (CPU) and Graphical Processing Unit (GPU). While alternating least squar…

We examine the scaling regime for the detrended fluctuation analysis (DFA) - the most popular method used to detect the presence of long memory in data and the fractal structure of time series. First, the scaling range for DFA is studied for uncorrelated data as a function of length $L$ of time series and regression li…

Study finds long memory in some emerging Asian stocks but not in developed markets.

We propose a generic algorithmic building block to accelerate training of machine learning models on heterogeneous compute systems. Our scheme allows to efficiently employ compute accelerators such as GPUs and FPGAs for the training of large-scale machine learning models, when the training data exceeds their memory cap…

Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.

Paper proposes a GPU-based system for training massive deep learning models in ads systems.

We perform return interval analysis of 1-min {\em{realized volatility}} defined by the sum of absolute high-frequency intraday returns for the Shanghai Stock Exchange Composite Index (SSEC) and 22 constituent stocks of SSEC. The scaling behavior and memory effect of the return intervals between successive realized vola…

Muon outperforms GD in associative memory learning by balancing frequency components.

We extend Kirman's model by introducing variable event time scale. The proposed flexible time scale is equivalent to the variable trading activity observed in financial markets. Stochastic version of the extended Kirman's agent based model is compared to the non-linear stochastic models of long-range memory in financia…

We investigate scaling and memory effects in return intervals between price volatilities above a certain threshold $q$ for the Japanese stock market using daily and intraday data sets. We find that the distribution of return intervals can be approximated by a scaling function that depends only on the ratio between the …

Embedding representations power machine intelligence in many applications, including recommendation systems, but they are space intensive -- potentially occupying hundreds of gigabytes in large-scale settings. To help manage this outsized memory consumption, we explore mixed dimension embeddings, an embedding layer arc…

State-of-the-art models are now trained with billions of parameters, reaching hardware limits in terms of memory consumption. This has created a recent demand for memory-efficient optimizers. To this end, we investigate the limits and performance tradeoffs of memory-efficient adaptively preconditioned gradient methods.…

Kernel-based K-means clustering has gained popularity due to its simplicity and the power of its implicit non-linear representation of the data. A dominant concern is the memory requirement since memory scales as the square of the number of data points. We provide a new analysis of a class of approximate kernel methods…

We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in t…

Gravitational wave memory increases faster than Brownian motion in early universe and astrophysical sources.

Transformers can store facts efficiently using associative memories.

Muon spectral optimizer outperforms SGD in associative memory tasks.

N-BEATS(P) efficiently forecasts millions of time series with reduced memory and time.

A new algorithm speeds up CP decomposition for large tensors.

Memory networks are neural networks with an explicit memory component that can be both read and written to by the network. The memory is often addressed in a soft way using a softmax function, making end-to-end training with backpropagation possible. However, this is not computationally scalable for applications which …

Kernel methods provide a principled way to perform non linear, nonparametric learning. They rely on solid functional analytic foundations and enjoy optimal statistical properties. However, at least in their basic form, they have limited applicability in large scale scenarios because of stringent computational requireme…

Kernel clustering algorithm improved for large datasets using incomplete Cholesky factorization.

Study shows memory needs grow with task sequence length in continual learning.

FDS tackles long horizon hyperparameter optimization issues.

Extended LSTMs improve volatility prediction by 20%.

Product Kanerva Machines dynamically combine smaller models for better memory organization.

Extends RCNNs to handle hypercomplex-valued data.

Deep networks retain initial bias after training, affecting generalization.

We consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, which is difficult to solve with existing QA methods due to their lack of scalability. To tackle this problem, we propose a novel end-to…

A new L-BFGS method tackles large-scale optimization with fewer evaluations.

DS-FACTO optimizes factorization machines for large-scale datasets.

XLA compiler extension improves memory efficiency for machine learning.

DVNet efficiently segments large neurovascular datasets using skip connections.