Improved GNN handles long-range dependencies in multi-relational graphs.
problem Vanishing gradients in GNNs for multi-relational graphs.
method Proposes a Gated Graph Neural Network with improved long-range dependency handling.
result Outperforms popular GNN models in synthetic tasks.
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
Fractal analysis is carried out on the stock market indices of seven European countries and the US. We find evidence of long range dependence in the log return series of the Mibtel (Italy) and the PX Glob (Czech Republic). Long range dependence implies that predictable patterns in the log returns do not dissipate quick…
Novel graph neural network combines random walks with local message passing.
problem Graph neural networks struggle with long-range dependencies.
method Combines random walks with local message passing in a novel architecture.
result Significant performance improvements on graph benchmarks.
This paper presents empirical evidence using recently developed techniques in econophysics suggesting that the degree of long-range dependence in interest rates depends on the conduct of monetary policy. We study the term structure of interest rates for the US and find evidence that global Hurst exponents change dramat…
This paper studies the dynamics of Brazilian interest rates for short-term maturities. The paper employs developed techniques in the econophysics literature and tests for long-range dependence in the term structure of these interest rates for the last decade. Empirical results suggest that the degree of long-range depe…
IGNN captures long-range graph dependencies using fixed-point equations.
problem Limited GNN ability to capture long-range graph dependencies.
method Fixed-point equilibrium equations involving implicitly defined state vectors, leveraging Perron-Frobenius theory and projected gradient descent.
result IGNN consistently captures long-range dependencies and outperforms state-of-the-art GNNs.
Regularizes RNNs to handle long-range dependencies and multiple time scales.
problem Identifying nonlinear dynamical systems with varying time scales and long-range dependencies.
method A simple regularization scheme for vanilla RNNs with ReLU activation.
result Regularized RNNs can solve long-range dependency problems and express slow time scales.
This study uses moving average cluster entropy to analyze financial market dynamics.
problem Understanding long-range dependence in financial markets.
method Moving average cluster entropy approach applied to ARFIMA and FBM processes.
result Long-range positive correlation in financial markets is linked to the cluster entropy behavior.
New model incorporates long-range dependence in mortality rates for better valuation and risk management.
problem Lack of appropriate models for valuing and managing mortality securities with long-range dependence.
method Proposes a novel class of Volterra mortality models that incorporate LRD, derived in closed-form solution.
result Models provide flexibility and tractability for valuing and hedging mortality-related products.
DRew dynamically rewires message passing to improve long-range tasks.
problem Over-squashing in MPNNs due to local message passing.
method Layer-dependent rewiring and delay mechanism for gradual densification.
result DRew outperforms graph Transformers and multi-hop MPNNs on long-range tasks.
Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring long-range dependence, either in a given data source or as represented in a trained deep model, r…
AR-Net models time-series with interpretable coefficients and scalability.
problem Modeling time-series with long-range dependencies and interpretability.
method Feed-forward neural network approach to AR-process dynamics.
result AR-Net learns identical AR-coefficients as Classic-AR and scales to long-range dependencies.
Improved Mamba model for long-range sequence tasks.
problem Mamba's poor performance on long-range sequential tasks.
method Proposed B2S6, combining block-wise selective dynamics and channel-specific bias.
result Empirically, B2S6 outperforms S4 and S4D on LRA tasks.
New neural model processes 2D data with long-range dependencies efficiently.
problem Limited receptive field of convolutions for complex 2D tasks.
method Proposes Matrix Shuffle-Exchange network with O(logn) layers and O(n2logn) complexity. result Exceeds convolutional and graph neural network baselines in long-range dependency modeling.
Paper proposes an active learning method for surgical workflow recognition using long-range temporal dependency.
problem Challenges in automatic surgical workflow recognition due to lack of large-scale labelled datasets.
method NL-RCNet with non-local block for capturing long-range temporal dependency and intra-clip dependency score for selection.
result Our approach outperforms state-of-the-art methods by selecting only 50% of samples for training.
Develops a bi-variate stochastic framework to model mortality and interest rates with long-range dependence.
problem Captures long-range dependence and instantaneous correlation in mortality and interest rates.
method Mixed fractional Brownian motions, analytical solutions, risk-neutral measure, sequential parameter estimation.
result Explicit pricing of zero-coupon bonds and extreme mortality bonds, practical implications for pricing and risk management.
In this work, we develop a novel regularizer to improve the learning of long-range dependency of sequence data. Applied on language modelling, our regularizer expresses the inductive bias that sequence variables should have high mutual information even though the model might not see abundant observations for complex lo…
In this paper, we present the results of Monte Carlo simulations for two popular techniques of long-range correlations detection - classical and modified rescaled range analyses. A focus is put on an effect of different distributional properties on an ability of the methods to efficiently distinguish between short and …
Rough Transformers improve efficiency for medical time-series data.
problem Efficiently modeling irregularly sampled, long-range time-series data.
method Introducing Rough Transformers, a Transformer variant with continuous-time representations and multi-view signature attention.
result Rough Transformers outperform vanilla Transformers while using less computational resources.
We investigate how simultaneously recorded long-range power-law correlated multi-variate signals cross-correlate. To this end we introduce a two-component ARFIMA stochastic process and a two-component FIARCH process to generate coupled fractal signals with long-range power-law correlations which are at the same time lo…
Study finds long-range dependence in financial markets, but deep generative models struggle to replicate it.
problem Long-range dependence in financial markets and challenges of deep generative models.
method Empirical analysis of financial data from three sectors, including LRD through various statistical methods and deep learning models.
result Deep generative models can reproduce stylized features but fail to capture long-range dependence structures.
Cointegration helps insurers understand long-range mortality patterns.
problem Insurers struggle to detect long-range dependence in their mortality data.
method Cointegration techniques applied to mixed fractional Brownian motion (mfBm) to capture long-range dependence.
result Cointegration brings long-range dependence information from national mortality data to insurers' models.
We review ideas on temporal dependences and recurrences in discrete time series from several areas of natural and social sciences. We revisit existing studies and redefine the relevant observables in the language of copulas (joint laws of the ranks). We propose that copulas provide an appropriate mathematical framework…
Transformer model improves source code summarization.
problem Generating readable summaries of source code.
method Transformer model with self-attention mechanism for code representation.
result Transformer model outperforms state-of-the-art techniques.
Investigates RI strategies for life insurers with LRD mortality rates.
problem Effect of long-range dependent mortality rates on RI strategies.
method Volterra mortality model, compound Poisson process, open-loop equilibrium mean-variance criterion.
result Explicit equilibrium RI controls derived and uniqueness studied.
Study on SA with heavy-tailed and LRD noise, establishing finite-time bounds.
problem Analyzing stochastic approximation under heavy-tailed and LRD noise.
method Noise-averaging argument to regularize impact of non-classical noise.
result Established first finite-time moment bounds for SA under heavy-tailed and LRD noise.
Forecaster uses graph Transformers to forecast spatial and time-dependent data.
problem Complex spatial and temporal dependencies in data.
method Graph Transformer architecture with sparsification for spatial and temporal dependencies.
result Forecaster significantly outperforms state-of-the-art baselines in taxi demand forecasting.
TFiLM expands convolutional models' receptive field with minimal overhead.
problem Capturing long-range dependencies in sequential data.
method A novel architectural component using a recurrent neural network to modulate convolutional model activations.
result TFiLM significantly improves learning speed and accuracy on various tasks.
Combines PCA and message passing for better graph node embeddings.
problem Challenges of learning long-range dependencies in graph data.
method PCAPass: PCA + message passing for unsupervised node embeddings.
result Empirically shows competitive performance on node classification benchmarks.
Self-attention prefers sparse functions of input sequences, reducing sample complexity.
problem Understanding the inductive biases of self-attention in modeling long-range dependencies.
method Theoretical analysis and synthetic experiments to probe sample complexity of learning sparse functions with Transformers.
result Bounded-norm Transformer networks can represent sparse functions of the input sequence with logarithmic sample complexity.
Improved BN-S model for crude oil price analysis using machine learning.
problem Limitations of the BN-S model, especially lack of long-range dependence.
method Refined BN-S model with machine learning algorithms.
result The refined model is more efficient and has fewer parameters.
Analysis of long-range dependence in financial time series was one of the initial steps of econophysics into the domain of mainstream finance and financial economics in the 1990s. Since then, many different financial series have been analyzed using the methods standardly used outside of finance to deliver some importan…
In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based language model designed to directly capture the global semantic meaning relating words in a document via latent topics. Because of their sequential nature, RNNs are good at capturing the local structure of a word sequence - both semantic and syn…
A new algorithm for restless bandits handles long-range dependencies.
problem Generalization of linear bandits with time-dependent parameters.
method LinMix-UCB algorithm with Berbee's coupling lemma.
result Sub-linear regret of $\mathcal{O}\left(\sqrt{d n\mathrm{polylog}(n) }
ight)$.
We consider the estimation of large covariance and precision matrices from high-dimensional sub-Gaussian or heavier-tailed observations with slowly decaying temporal dependence. The temporal dependence is allowed to be long-range so with longer memory than those considered in the current literature. We show that severa…
We propose a multiresolution Gaussian process to capture long-range, non-Markovian dependencies while allowing for abrupt changes. The multiresolution GP hierarchically couples a collection of smooth GPs, each defined over an element of a random nested partition. Long-range dependencies are captured by the top-level GP…
Proposes continuous graph neural networks to capture long-range dependencies.
problem Capturing long-range dependencies in graph data.
method Defines continuous dynamics for graph neural networks using diffusion-based methods.
result Proposed continuous graph neural networks are effective and deeper networks can capture long-range dependencies.
Approximates derivative pricing under fractional stochastic volatility.
problem Derivative pricing under fractional stochastic volatility model.
method Approximate expression derived from deterministic functions and fractional Ornstein-Uhlenbeck process.
result Numerical simulations show the feasibility and effect of long-range dependencies on derivative prices.
Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data. Long Range Dependence (LRD) --- referring to long-range correlations decaying as a power law rather than exponentially w.r.t. distance --- demands a different set of tools for modeling the unde…
This paper examines multifractal dynamics in cryptocurrencies using two methodologies.
problem Understanding the multifractal nature of cryptocurrencies and their stochastic processes.
method Two alternative multi-scaling methodologies applied to 84 cryptocurrencies.
result Cryptocurrencies exhibit different degrees of long-range dependence and stochastic processes.
In this paper we extend the theory of option pricing to take into account and explain the empirical evidence for asset prices such as non-Gaussian returns, long-range dependence, volatility clustering, non-Gaussian copula dependence, as well as theoretical issues such as asymmetric information and the presence of limit…
Geom-GCN improves graph neural networks by preserving structural information and capturing long-range dependencies.
problem Weaknesses in MPNNs' aggregators: loss of structural information and lack of long-range dependencies.
method Proposes a geometric aggregation scheme with three modules: node embedding, structural neighborhood, and bi-level aggregation.
result Achieved state-of-the-art performance on various graph datasets.
Analyzed a generalized voter model with power-law herding intensity, revealing anomalous diffusion and long-range memory.
problem Anomalous diffusion and long-range memory in a generalized voter model.
method Derived analytical expressions for moments and first passage time distribution, confirmed numerically.
result The model exhibits long-range memory indicators despite being a Markov model.
Graph neural networks struggle to propagate long-range information, causing over-squashing.
problem Graph neural networks struggle to propagate long-range information.
method Identified over-squashing as the bottleneck in GNNs, demonstrated on various models.
result Breaking the bottleneck improves GNNs' performance on long-range problems.
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
We present the mixture-of-parents maximum entropy Markov model (MoP-MEMM), a class of directed graphical models extending MEMMs. The MoP-MEMM allows tractable incorporation of long-range dependencies between nodes by restricting the conditional distribution of each node to be a mixture of distributions given the parent…
Transformers improve time series modeling by capturing long-range dependencies.
problem Capturing long-range dependencies in time series data.
method Summarized and reviewed adaptations of Transformers for time series analysis.
result Transformers enhance time series forecasting, anomaly detection, and classification.