arXiv research

HGNet improves GNNs' ability to handle long-range interactions in graphs.

Novel neural network layer improves long-range interactions in point clouds.

Study finds phase transition in context-sensitive language model with short-range interactions.

We propose a new method for assessing agents' influence in financial network structures, which takes into consideration the intensity of interactions. A distinctive feature of this approach is that it considers not only direct interactions of agents of the first level and indirect interactions of the second level, but …

Graph neural networks struggle to propagate long-range information, causing over-squashing.

DRew dynamically rewires message passing to improve long-range tasks.

Self-attention prefers sparse functions of input sequences, reducing sample complexity.

We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference. The method is to apply the Random Batch Method (RBM) for interacting particle systems proposed by Jin et al to the …

A new graph neural network framework captures long-range interactions efficiently.

With the daily and minutely data of the German DAX and Chinese indices, we investigate how the return-volatility correlation originates in financial dynamics. Based on a retarded volatility model, we may eliminate or generate the return-volatility correlation of the time series, while other characteristics, such as the…

A multi-scale model predicts atomic-scale properties using both local and long-range information.

To know the statistical distribution of a variable is an important problem in management of resources. Distributions of the power law type are observed in many real systems. However power law distributions have an infinite variance and thus can not be used as a standard distribution. Normally professionals in the area …

Learning representations that accurately capture long-range dependencies in sequential inputs -- including text, audio, and genomic data -- is a key problem in deep learning. Feed-forward convolutional models capture only feature interactions within finite receptive fields while recurrent architectures can be slow and …

Complex systems are composed of mutually interacting components and the output values of these components are usually long-range cross-correlated. We propose a method to characterize the joint multifractal nature of such long-range cross correlations based on wavelet analysis, termed multifractal cross wavelet analysis…

A dynamic herding model with interactions of trading volumes is introduced. At time $t$, an agent trades with a probability, which depends on the ratio of the total trading volume at time $t-1$ to its own trading volume at its last trade. The price return is determined by the volume imbalance and number of trades. The …

HC-GNN tackles long-range graph information and high-order neighbourhoods.

Mutually interacting components form complex systems and the outputs of these components are usually long-range cross-correlated. Using wavelet leaders, we propose a method of characterizing the joint multifractal nature of these long-range cross correlations, a method we call joint multifractal analysis based on wavel…

Recurrent Neural Networks (RNNS) are now widely used on sequence generation tasks due to their ability to learn long-range dependencies and to generate sequences of arbitrary length. However, their left-to-right generation procedure only allows a limited control from a potential user which makes them unsuitable for int…

Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data. Long Range Dependence (LRD) --- referring to long-range correlations decaying as a power law rather than exponentially w.r.t. distance --- demands a different set of tools for modeling the unde…

Axial-LOB predicts stock prices from LOB data using attention layers.

We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single realization. Asymptotic properties of maximum entropy microcanonical and macrocanonical processes and the…

W-Transformers use wavelets to improve time series forecasting.

In dialogues, an utterance is a chain of consecutive sentences produced by one speaker which ranges from a short sentence to a thousand-word post. When studying dialogues at the utterance level, it is not uncommon that an utterance would serve multiple functions. For instance, "Thank you. It works great." expresses bot…

Develops a nonlocal PINN framework using PDDO for better solution of PDEs with sharp gradients.

An average instantaneous cross-correlation function is introduced to quantify the interaction of the financial market of a specific time. Based on the daily data of the American and Chinese stock markets, memory effect of the average instantaneous cross-correlations is investigated over different price return time inte…

This paper looks into the analysis of the long-range auto-correlations and cross-correlations in bond market. Based on Detrended Moving Average (DMA) method, empirical results present a clear evidence of long-range persistence that exists in one year scale. The degree of long-range correlation related to maturities has…

Transformers improve time series modeling by capturing long-range dependencies.

Research on long-range memory in financial and social systems using various models.

Networks provide a powerful formalism for modeling complex systems by using a model of pairwise interactions. But much of the structure within these systems involves interactions that take place among more than two nodes at once; for example, communication within a group rather than person-to person, collaboration amon…

A new method uses burst and inter-burst duration to test long-range memory in financial markets.

We introduce a Maximum Entropy model able to capture the statistics of melodies in music. The model can be used to generate new melodies that emulate the style of the musical corpus which was used to train it. Instead of using the $n-$body interactions of $(n-1)-$order Markov models, traditionally used in automatic mus…

Spacetimeformer learns spatiotemporal relationships from data alone.

Transformers can perform well with less long-range memory.

The origin of the long-range memory in the non-equilibrium systems is still an open problem as the phenomenon can be reproduced using models based on Markov processes. In these cases a notion of spurious memory is introduced. A good example of Markov processes with spurious memory is stochastic process driven by a non-…

This econophysics work studies the long-range Ising model of a finite system with $N$ spins and the exchange interaction $\frac{J}{N}$ and the external field $H$ as a modely for homogeneous credit portfolio of assets with default probability $P_{d}$ and default correlation $ρ_{d}$. Based on the discussion on the $(J,H)…

A theory explaining how deep learning works is yet to be developed. Previous work suggests that deep learning performs a coarse graining, similar in spirit to the renormalization group (RG). This idea has been explored in the setting of a local (nearest neighbor interactions) Ising spin lattice. We extend the discussio…

Improved GNN handles long-range dependencies in multi-relational graphs.

Introduces Spectral Attention for better long-range time series forecasting.

We investigate how simultaneously recorded long-range power-law correlated multi-variate signals cross-correlate. To this end we introduce a two-component ARFIMA stochastic process and a two-component FIARCH process to generate coupled fractal signals with long-range power-law correlations which are at the same time lo…

TIMeSynC combines financial service interactions for intent prediction.

Matrix Product States (MPS), also known as Tensor Train (TT) decomposition in mathematics, has been proposed originally for describing an (especially one-dimensional) quantum system, and recently has found applications in various applications such as compressing high-dimensional data, supervised kernel linear classifie…

New principle reveals how neural networks learn complex interactions.

Improved Mamba model for long-range sequence tasks.

TSLANet improves time series models by capturing long-term and short-term interactions.

Novel graph neural network combines random walks with local message passing.

Collective phenomena with universal properties have been observed in many complex systems with a large number of components. Here we present a microscopic model of the emergence of scaling behavior in such systems, where the interaction dynamics between individual components is mediated by a global variable making the …

Fractal analysis is carried out on the stock market indices of seven European countries and the US. We find evidence of long range dependence in the log return series of the Mibtel (Italy) and the PX Glob (Czech Republic). Long range dependence implies that predictable patterns in the log returns do not dissipate quick…

A new sampler tackles critical phenomena by leveraging scale invariance.