A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Express improves causal attention guarantees for language models.
The mathematical model of a linear system with the short memory about own stochastic behavior is proposed. It is assumed that the system is under a continual influence of independent stochastic impulses. In a short memory approximation the expression of the stochastic process is found. An application of the model propo…
LEM efficiently models long-term sequences with gradients.
Iterative learning to infer approaches have become popular solvers for inverse problems. However, their memory requirements during training grow linearly with model depth, limiting in practice model expressiveness. In this work, we propose an iterative inverse model with constant memory that relies on invertible networ…
State-of-the-art models are now trained with billions of parameters, reaching hardware limits in terms of memory consumption. This has created a recent demand for memory-efficient optimizers. To this end, we investigate the limits and performance tradeoffs of memory-efficient adaptively preconditioned gradient methods.…
In this paper, a taxonomy for memory networks is proposed based on their memory organization. The taxonomy includes all the popular memory networks: vanilla recurrent neural network (RNN), long short term memory (LSTM ), neural stack and neural Turing machine and their variants. The taxonomy puts all these networks und…
Implicit models can match or exceed explicit models with more test-time compute.
The paper introduces a new stochastic volatility model with long-term memory and jumps.
Study reveals Transformer's expressive power and mechanisms.
We propose a general method to obtain approximation of the first passage time distribution for the birth-death processes. We rely on the general properties of birth-death processes, Keilson's theorem and the concept of Riemann sum to obtain closed-form expressions. We apply the method to the three selected birth-death …
ActNN reduces neural network training memory by 2-bit quantization.
We study the problem of learning associative memory -- a system which is able to retrieve a remembered pattern based on its distorted or incomplete version. Attractor networks provide a sound model of associative memory: patterns are stored as attractors of the network dynamics and associative retrieval is performed by…
The article introduces a new set of Polish word embeddings, built using KGR10 corpus, which contains more than 4 billion words. These embeddings are evaluated in the problem of recognition of temporal expressions (timexes) for the Polish language. We described the process of KGR10 corpus creation and a new approach to …
We introduce a Vasicek-type short rate model which has two additional parameters representing memory effect. This model presents better results in yield curve fitting than the classical Vasicek model. We derive closed-form expressions for the prices of bonds and bond options. Though the model is non-Markov, there exist…
We analyse perception and memory, using mathematical models for knowledge graphs and tensors, to gain insights into the corresponding functionalities of the human mind. Our discussion is based on the concept of propositional sentences consisting of \textit{subject-predicate-object} (SPO) triples for expressing elementa…
We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate. MAPO is applicable to deterministic environments with discrete actions, such as structured prediction and combinatorial optimization ta…
Transformers can store facts efficiently using associative memories.
Recurrent neural networks (RNN) are at the core of modern automatic speech recognition (ASR) systems. In particular, long-short term memory (LSTM) recurrent neural networks have achieved state-of-the-art results in many speech recognition tasks, due to their efficient representation of long and short term dependencies …
Spot option prices, forwards and options on forwards relevant for the commodity markets are computed when the underlying process S is modelled as an exponential of a process ξ with memory as e.g. a Lévy semi-stationary process. Moreover a risk premium \r{ho} representing storage costs, illiquidity, convenience yield or…
Analyzed a generalized voter model with power-law herding intensity, revealing anomalous diffusion and long-range memory.
Paper introduces a new framework to improve sample efficiency in POMDPs learning.
Bayesian neural networks can be partially stochastic without losing predictive power.
We propose Deep Closed-Form Subspace Clustering (DCFSC), a new embarrassingly simple model for subspace clustering with learning non-linear mapping. Compared with the previous deep subspace clustering (DSC) techniques, our DCFSC does not have any parameters at all for the self-expressive layer. Instead, DCFSC utilizes …
IVF k-means algorithm improves performance on large sparse data sets.
The paper introduces a new short rate model with memory components.
Improved neural ODEs learn adaptable flows.
Improves hypergraph link prediction by breaking symmetry.
Enhances reward specification in RL with a novel language-based approach.
We consider the general problem of modeling temporal data with long-range dependencies, wherein new observations are fully or partially predictable based on temporally-distant, past observations. A sufficiently powerful temporal model should separate predictable elements of the sequence from unpredictable elements, exp…
Regression learns Mori-Zwanzig operators for dynamical systems.
The paper tackles scalability issues in Graph Representation Learning.
In this paper we introduce a novel online time series forecasting model we refer to as the pM-GP filter. We show that our model is equivalent to Gaussian process regression, with the advantage that both online forecasting and online learning of the hyper-parameters have a constant (rather than cubic) time complexity an…
Computational methods that predict differential gene expression from histone modification signals are highly desirable for understanding how histone modifications control the functional heterogeneity of cells through influencing differential gene regulation. Recent studies either failed to capture combinatorial effects…
Introduces Exponentially Weighted Signature for better path representation.
Recent empirical studies suggest that the volatility of an underlying price process may have correlations that decay slowly under certain market conditions. In this paper, the volatility is modeled as a stationary process with long-range correlation properties in order to capture such a situation, and we consider Europ…
New RNN model handles long-term dependencies in irregularly-sampled time series.
A new method for estimating SW from streaming data.
Enhanced Neural ODEs outperform traditional models in image classification and video prediction.
In biological research machine learning algorithms are part of nearly every analytical process. They are used to identify new insights into biological phenomena, interpret data, provide molecular diagnosis for diseases and develop personalized medicine that will enable future treatments of diseases. In this paper we (1…
PLUMAGE improves large model training efficiency and stability.
DS-FACTO optimizes factorization machines for large-scale datasets.
New hypergraph method improves scRNA-seq clustering.
New explanation of reservoir computing using random projections.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
Word2Vec is a widely used algorithm for extracting low-dimensional vector representations of words. It generated considerable excitement in the machine learning and natural language processing (NLP) communities recently due to its exceptional performance in many NLP applications such as named entity recognition, sentim…
Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown breakthrough success in natural language processing (NLP), achieving state-of-the-art results in domains such as language modeling and machine t…
We analyze generalization in deep learning models using random matrix theory.
Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized by the temporal scaling of correlation and mutual information. In this paper, we study the mutual information of recurrent neural networks …