In this work, we develop a novel regularizer to improve the learning of long-range dependency of sequence data. Applied on language modelling, our regularizer expresses the inductive bias that sequence variables should have high mutual information even though the model might not see abundant observations for complex lo…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Long-Range Information”
Novel graph neural network combines random walks with local message passing.
problem Graph neural networks struggle with long-range dependencies.
method Combines random walks with local message passing in a novel architecture.
result Significant performance improvements on graph benchmarks.
Graph neural networks struggle to propagate long-range information, causing over-squashing.
problem Graph neural networks struggle to propagate long-range information.
method Identified over-squashing as the bottleneck in GNNs, demonstrated on various models.
result Breaking the bottleneck improves GNNs' performance on long-range problems.
HC-GNN tackles long-range graph information and high-order neighbourhoods.
problem Costly encoding of long-range information and failure to encode high-order neighbourhoods.
method Hierarchical structure with multi-level super graphs and innovative intra- and inter-level propagation.
result HC-GNN efficiently captures long-range interactions and incorporates meso- and macro-level semantics.
Novel neural network layer improves long-range interactions in point clouds.
problem Efficiently treating long-range interactions in point clouds.
method Long-range convolutional (LRC)-layer that incorporates Fourier transforms.
result Global all-to-all convolution operation performed in nearly-linear time.
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
HGNet improves GNNs' ability to handle long-range interactions in graphs.
problem Insufficiency of GNNs in capturing long-range interactions.
method Introduces hierarchical message passing models with multi-resolution graph representations.
result HGNet outperforms conventional GNNs in molecular property prediction.
Graph algorithms are key tools in many fields of science and technology. Some of these algorithms depend on propagating information between distant nodes in a graph. Recently, there have been a number of deep learning architectures proposed to learn on undirected graphs. However, most of these architectures aggregate i…
This study uses moving average cluster entropy to analyze financial market dynamics.
problem Understanding long-range dependence in financial markets.
method Moving average cluster entropy approach applied to ARFIMA and FBM processes.
result Long-range positive correlation in financial markets is linked to the cluster entropy behavior.
In this paper we extend the theory of option pricing to take into account and explain the empirical evidence for asset prices such as non-Gaussian returns, long-range dependence, volatility clustering, non-Gaussian copula dependence, as well as theoretical issues such as asymmetric information and the presence of limit…
Cointegration helps insurers understand long-range mortality patterns.
problem Insurers struggle to detect long-range dependence in their mortality data.
method Cointegration techniques applied to mixed fractional Brownian motion (mfBm) to capture long-range dependence.
result Cointegration brings long-range dependence information from national mortality data to insurers' models.
This paper looks into the analysis of the long-range auto-correlations and cross-correlations in bond market. Based on Detrended Moving Average (DMA) method, empirical results present a clear evidence of long-range persistence that exists in one year scale. The degree of long-range correlation related to maturities has…
Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data. Long Range Dependence (LRD) --- referring to long-range correlations decaying as a power law rather than exponentially w.r.t. distance --- demands a different set of tools for modeling the unde…
Combines PCA and message passing for better graph node embeddings.
problem Challenges of learning long-range dependencies in graph data.
method PCAPass: PCA + message passing for unsupervised node embeddings.
result Empirically shows competitive performance on node classification benchmarks.
Research on long-range memory in financial and social systems using various models.
problem Understanding the nature of long-range memory in socioeconomic systems.
method Various Markov processes including point processes, stochastic differential equations, and agent-based models.
result New estimators of self-similarity and long-range memory for non-Gaussian systems are needed.
A new method uses burst and inter-burst duration to test long-range memory in financial markets.
problem Varying results from long-range memory estimators in financial markets.
method Burst and inter-burst duration statistical analysis of limit order book data.
result The new method provides a more reliable evaluation of the Hurst exponent.
Transformers can perform well with less long-range memory.
problem The need for deep long-range memory in Transformers for language modeling.
method Performed interventions to show performance can be achieved with fewer long-range memories and by limiting attention range.
result Comparable performance can be achieved with 6X fewer long-range memories and better performance with limited attention range.
Paper proposes an active learning method for surgical workflow recognition using long-range temporal dependency.
problem Challenges in automatic surgical workflow recognition due to lack of large-scale labelled datasets.
method NL-RCNet with non-local block for capturing long-range temporal dependency and intra-clip dependency score for selection.
result Our approach outperforms state-of-the-art methods by selecting only 50% of samples for training.
A multi-scale model predicts atomic-scale properties using both local and long-range information.
problem Inability of machine-learning schemes to capture long-range physical effects.
method Combines local and non-local information in a multipole expansion framework.
result Demonstrates the ability to model electrostatics, polarization, and dispersion.
This paper investigates the presence of long memory in corporate bond and stock indices of six European Union countries from July 1998 to February 2015. We compute the Hurst exponent by means of the DFA method and using a sliding window in order to measure long range dependence. We detect that Hurst exponents behave di…
Predicts solar dynamics with diffusion models, improving long-range dependencies.
problem Predicting solar dynamics with limited observable data.
method Multiscale inference scheme for diffusion models.
result Improved long-range predictions with reduced bias.
Improved GNN handles long-range dependencies in multi-relational graphs.
problem Vanishing gradients in GNNs for multi-relational graphs.
method Proposes a Gated Graph Neural Network with improved long-range dependency handling.
result Outperforms popular GNN models in synthetic tasks.
A method to generate long-range human actions by leveraging graph convolutional networks and self-attention.
problem Generating long-range skeleton-based human actions is challenging due to small frame deviations.
method Proposes a variant of GCNs with self-attention to adaptively sparsify action graphs and capture structure information.
result Extensive experiments show superior performance compared to existing methods on human action datasets.
We investigate how simultaneously recorded long-range power-law correlated multi-variate signals cross-correlate. To this end we introduce a two-component ARFIMA stochastic process and a two-component FIARCH process to generate coupled fractal signals with long-range power-law correlations which are at the same time lo…
Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. …
Improved Mamba model for long-range sequence tasks.
problem Mamba's poor performance on long-range sequential tasks.
method Proposed B2S6, combining block-wise selective dynamics and channel-specific bias.
result Empirically, B2S6 outperforms S4 and S4D on LRA tasks.
DRew dynamically rewires message passing to improve long-range tasks.
problem Over-squashing in MPNNs due to local message passing.
method Layer-dependent rewiring and delay mechanism for gradual densification.
result DRew outperforms graph Transformers and multi-hop MPNNs on long-range tasks.
Fractal analysis is carried out on the stock market indices of seven European countries and the US. We find evidence of long range dependence in the log return series of the Mibtel (Italy) and the PX Glob (Czech Republic). Long range dependence implies that predictable patterns in the log returns do not dissipate quick…
Long-range dependence in Interest Rates and Monetary Policyphysics.data-an
This paper studies the dynamics of Brazilian interest rates for short-term maturities. The paper employs developed techniques in the econophysics literature and tests for long-range dependence in the term structure of these interest rates for the last decade. Empirical results suggest that the degree of long-range depe…
This paper presents empirical evidence using recently developed techniques in econophysics suggesting that the degree of long-range dependence in interest rates depends on the conduct of monetary policy. We study the term structure of interest rates for the US and find evidence that global Hurst exponents change dramat…
A new method extracts events and their arguments efficiently from text.
problem Efficiently extract event information from texts with long-range dependencies and associations.
method Graph Convolutional Networks with shortest dependency paths to capture syntactic relationships.
result Significant improvement over state-of-the-art methods.
We present the Compressive Transformer, an attentive sequence model which compresses past memories for long-range sequence learning. We find the Compressive Transformer obtains state-of-the-art language modelling results in the WikiText-103 and Enwik8 benchmarks, achieving 17.1 ppl and 0.97 bpc respectively. We also fi…
We introduce a new method for detection of long-range cross-correlations and multifractality - multifractal height cross-correlation analysis (MF-HXA) - based on scaling of qth order covariances. MF-HXA is a bivariate generalization of the height-height correlation analysis of Barabasi & Vicsek [Barabasi, A.L., Vicsek,…
IGNN captures long-range graph dependencies using fixed-point equations.
problem Limited GNN ability to capture long-range graph dependencies.
method Fixed-point equilibrium equations involving implicitly defined state vectors, leveraging Perron-Frobenius theory and projected gradient descent.
result IGNN consistently captures long-range dependencies and outperforms state-of-the-art GNNs.
In the sentence classification task, context formed from sentences adjacent to the sentence being classified can provide important information for classification. This context is, however, often ignored. Where methods do make use of context, only small amounts are considered, making it difficult to scale. We present a …
We propose a new method (implemented in an R-program) to simulate long-range daily stock-price data. The program reproduces various stylized facts much better than various parametric models from the extended GARCH-family. In particular, the empirically observed changes in unconditional variance are truthfully mirrored …
Long-range correlation in financial time series reflects the complex dynamics of the stock markets driven by algorithms and human decisions. Our analysis exploits ultra-high frequency order book data from NASDAQ Nordic over a period of three years to numerically estimate the power-law scaling exponents using detrended …
Volterra mortality model: Actuarial valuation and risk management with long-range dependenceq-fin.MF
New model incorporates long-range dependence in mortality rates for better valuation and risk management.
problem Lack of appropriate models for valuing and managing mortality securities with long-range dependence.
method Proposes a novel class of Volterra mortality models that incorporate LRD, derived in closed-form solution.
result Models provide flexibility and tractability for valuing and hedging mortality-related products.
Mobility datasets are fundamental for evaluating algorithms pertaining to geographic information systems and facilitating experimental reproducibility. But privacy implications restrict sharing such datasets, as even aggregated location-data is vulnerable to membership inference attacks. Current synthetic mobility data…
Spacetimeformer learns spatiotemporal relationships from data alone.
problem Forecasting multivariate time series with distinct spatial relationships.
method Transformers with dynamic graph connections learning interactions between space, time, and value.
result Competitive results on various time series prediction benchmarks.
Proving that next-token prediction makes language models generate coherent long documents.
problem Understanding why language models generate coherent documents despite focusing on next-token prediction.
method Proving the power of next-token prediction in learning longer-range structure using Recurrent Neural Networks (RNN).
result Optimizing next-token prediction in RNNs yields a model that closely approximates the training distribution, even for long-range coherence.
Price graphs: Utilizing the structural information of financial time series for stock predictionq-fin.ST
Proposes a graph-based approach for better stock prediction.
problem Long-range dependencies and chaotic property in stock prediction.
method Transforms time series into graphs, extracting structural information to resolve issues.
result Obtains the best performance among state-of-the-art benchmarks and highest cumulative profits in trading simulations.
Earlier we proposed the stochastic point process model, which reproduces a variety of self-affine time series exhibiting power spectral density S(f) scaling as power of the frequency f and derived a stochastic differential equation with the same long range memory properties. Here we present a stochastic differential eq…
Hierarchical decoupling improves sample efficiency for complex robots.
problem Learning long-range behaviors on complex robots.
method Two-part policy: low-level imitation and high-level transfer, with KL regularization.
result Hierarchical transfer significantly improves zero-shot high-level transfer and stabilizes learning.
Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring long-range dependence, either in a given data source or as represented in a trained deep model, r…
Anomalous diffusion and long-range memory in the scaled voter modelcond-mat.stat-mech
Analyzed a generalized voter model with power-law herding intensity, revealing anomalous diffusion and long-range memory.
problem Anomalous diffusion and long-range memory in a generalized voter model.
method Derived analytical expressions for moments and first passage time distribution, confirmed numerically.
result The model exhibits long-range memory indicators despite being a Markov model.
New neural model processes 2D data with long-range dependencies efficiently.
problem Limited receptive field of convolutions for complex 2D tasks.
method Proposes Matrix Shuffle-Exchange network with layers and complexity.
result Exceeds convolutional and graph neural network baselines in long-range dependency modeling.
Develops a bi-variate stochastic framework to model mortality and interest rates with long-range dependence.
problem Captures long-range dependence and instantaneous correlation in mortality and interest rates.
method Mixed fractional Brownian motions, analytical solutions, risk-neutral measure, sequential parameter estimation.
result Explicit pricing of zero-coupon bonds and extreme mortality bonds, practical implications for pricing and risk management.