This paper uses SPk languages to explore LDD characteristics and multi-element dependencies.
problem Understanding and modeling Long Distance Dependencies (LDDs) with multi-element interactions.
method Generated datasets with various properties using Strictly k-Piecewise languages, analyzed using mutual information.
result The number of interacting elements in a dependency is a key characteristic of LDDs.
We consider the task of detecting regulatory elements in the human genome directly from raw DNA. Past work has focused on small snippets of DNA, making it difficult to model long-distance dependencies that arise from DNA's 3-dimensional conformation. In order to study long-distance dependencies, we develop and release …
Analyzes dependencies in sequential datasets to improve deep neural architectures.
problem Improving deep recurrent neural architectures by understanding long distance dependencies.
method Detailed analysis of dependency decay curves in various datasets, testing factors affecting decay, generating synthesized datasets.
result Factors influencing dependency decay curves (number of unique symbols, dataset size, interacting symbols, distance between symbols) can inform optimal hyper-parameters.
DRew dynamically rewires message passing to improve long-range tasks.
problem Over-squashing in MPNNs due to local message passing.
method Layer-dependent rewiring and delay mechanism for gradual densification.
result DRew outperforms graph Transformers and multi-hop MPNNs on long-range tasks.
Paper shows how LSTM can remember long sequences by attending to persisted information.
problem LSTMs struggle with long sequences due to fading information and bias towards recent data.
method The paper introduces a mechanism that allows LSTMs to attend to information in memory based on how long it was persisted by the gating mechanism.
result The method improves LSTM's ability to process long sequences by retrieving information proportionally to its persistence in memory.
Physics: Similar long-distance properties can mask vastly different short-distance metrics.
problem Classifying homogeneous metrics on group manifolds by long-distance properties.
method Apply universality concept to geometry, focusing on metrics on Lie groups.
result Many metrics on low-dimensional Lie groups have similar long-distance properties despite differing short-distance properties.
HighwayGraph models long-distance node relations in GNNs with improved performance.
problem Limited-layer information propagation in GNNs hinders long-distance node relation modeling.
method Proposes two solutions: implicit and explicit modeling of long-distance node relations using shallow GNN architectures and a self-training framework.
result HighwayGraph achieves consistent and significant improvements over four GNNs on three benchmark datasets.
Combines PCA and message passing for better graph node embeddings.
problem Challenges of learning long-range dependencies in graph data.
method PCAPass: PCA + message passing for unsupervised node embeddings.
result Empirically shows competitive performance on node classification benchmarks.
The study examines Hawkes processes and their long-term behavior.
problem Understanding the long-term behavior of Hawkes processes.
method Proving functional limit theorems under various conditions on the dispersion of child events.
result Functional limit theorems hold for Hawkes processes with different levels of child event dispersion.
Estimates intrinsic dimension of data for GANs.
problem Estimating intrinsic dimension of high-dimensional data.
method Uses Wasserstein distances for estimation.
result Provides sample complexity bounds for GANs.
Estimates LRD in sequential data, improving RNNs.
problem Quantifying LRD in sequential data for better RNNs.
method Principled estimation procedure based on LRD theory for real-valued time series.
result Estimates LRD reliably in user behavior and Wikipedia article writing.
Deep RNNs excel at capturing long-term dependencies in sequential data.
problem Lack of a formal measure for RNNs' long-term memory capacity.
method Introduced a measure called Start-End separation rank to quantify RNNs' ability to model long-term dependencies.
result Deep RNNs support Start-End separation ranks that are combinatorially higher than shallow ones.
This paper studies mutual information in sequence models and finds Transformers excel in capturing long-range dependencies.
problem Understanding the expressive power of sequence models in capturing temporal dependencies.
method Theoretical and empirical analysis of linear and nonlinear RNNs, including Transformers.
result Transformers can capture long-range mutual information more efficiently than RNNs.
Paper proposes Gini distance statistics for estimating feature-label dependence.
problem Identifying statistical dependence between features and categorical labels.
method Generalized Gini distance in RKHS for feature-label dependence estimation.
result Gini distance statistics converge faster and have tighter error bounds than distance covariance.
Machine learning aids in discovering and optimizing quantum protocols for long-distance communication.
problem Designing and optimizing quantum communication protocols for long distances.
method Projective simulation, a learning agent combining reinforcement learning and decision making.
result Machine learning identifies and improves quantum protocols like teleportation and entanglement purification.
Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.
problem Understanding long-term behavior of finite-particle systems in relation to their mean-field limits.
method Developed uniform-in-time propagation-of-chaos results for continuous-time SVGD using cutoff strategies and finite-dimensional theories.
result Uniform-in-time propagation-of-chaos bounds in various metrics, including Langevin kernel Stein discrepancy, Wasserstein-1, and Wasserstein-2 distances.
Improves retrieval accuracy for hierarchical documents, especially for distant matches.
problem Limited expressive power of dual encoder models in hierarchical retrieval.
method Proves feasibility of DEs for HR, introduces pretrain-finetune recipe to improve long-distance retrieval.
result Pretrain-finetune boosts recall on long-distance pairs from 19% to 76%.
Finite Goeritz groups for links with long bridge decompositions.
problem Characterizing Goeritz groups of links.
method Proving finite Goeritz groups for links with specific bridge decompositions.
result Finite Goeritz groups for links with distance at least 6.
We consider a point cloud Xn:={x1,…,xn} uniformly distributed on the flat torus Td:=Rd/Zd, and construct a geometric graph on the cloud by connecting points that are within distance ε of each other. We let P(Xn) be the space of probability …
Novel methods robustify Gromov-Wasserstein distance for cross-domain alignment.
problem Robustifying Gromov-Wasserstein distance for cross-domain alignment.
method Three novel techniques derived from robust statistics to improve GW and its variants.
result Empirical validation shows superior resilience to contamination.
Paper explores using memory models to improve idiomatic language translation.
problem Translation of idiomatic language poses challenges to machine translation systems.
method Uses memory augmented models to enhance neural machine translation of idioms.
result Improves translation of idiomatic language by bridging long-distance dependencies.
Investigates long-range dependencies in language and music.
problem Measuring long-range dependence in sequence data.
method Statistical framework based on long memory stochastic processes.
result Testable implications about long memory in data and learned representations.
BlockEcho method improves imputation of block-wise missing data.
problem Block-wise missing data reduces interpolation capability and predictive power.
method Integrates Matrix Factorization (MF) within Generative Adversarial Networks (GAN) to retain long-distance inter-element relationships.
result Superior performance on public datasets across three domains, especially at higher missing rates.
The following working document summarizes our work on the clustering of financial time series. It was written for a workshop on information geometry and its application for image and signal processing. This workshop brought several experts in pure and applied mathematics together with applied researchers from medical i…
We present a case-study demonstrating the usefulness of Bayesian hierarchical mixture modelling for investigating cognitive processes. In sentence comprehension, it is widely assumed that the distance between linguistic co-dependents affects the latency of dependency resolution: the longer the distance, the longer the …
We develop the distance dependent Chinese restaurant process (CRP), a flexible class of distributions over partitions that allows for non-exchangeability. This class can be used to model many kinds of dependencies between data in infinite clustering models, including dependencies across time or space. We examine the pr…
PBO methods improve RNN performance in learning long-term dependencies.
problem Training RNNs to learn long-term dependencies is challenging.
method Population-based global optimisation (PBO) techniques, including evolution strategies and particle swarm optimisation.
result PBO methods lead to performance improvements in RNNs for volatility forecasting.
Novel graph model forecasts urban traffic with reduced spatial complexity.
problem Challenges in traffic forecasting due to spatio-temporal complexity, especially in urban environments.
method MW-TGC network model that combines spatial and temporal dependencies using multi-weighted adjacency matrices and graph convolution operations.
result MW-TGC network outperforms other models in urban-core and urban-mix sites, reducing variance in heterogeneous environments.
Paper provides GOT convergence guarantees for sub-gamma distributions and dependent samples.
problem Estimating GOT distance under general settings.
method Gaussian-smoothed optimal transport (GOT) framework, sub-gamma distributions, dependent samples, kernel MMD distances.
result Convergence guarantees for GOT distance under more general settings.
We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance…
Procedure tests if unknown Markov chain matches a reference chain.
problem Testing if an unknown Markov chain matches a reference chain.
method An efficient procedure based on a single long state sequence.
result Nearly matching upper and lower sample complexity bounds for total variation distance.
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
New empirical process bounds reveal trade-off between dependence and complexity in nonparametric learning.
problem Understanding generalization in nonparametric learning with temporal dependencies.
method Developed bounds on expected supremum of empirical processes under β/ρ-mixing assumptions. result Achieved rates similar to i.i.d. setting under long-range dependence with complex function classes.
Researchers modify dp distance to handle long, thin splines.
problem Maintaining stability in convergence metrics with scalar curvature approaching positivity.
method Introducing and analyzing a modified dp distance to handle persistent splines. result The modified dp distance provides a stable estimate, useful for geometric stability. A new RNN model tackles long-time dependencies with fast, invertible, and memory-efficient hidden states.
problem Challenges in processing sequential inputs with long-time dependencies in RNNs.
method A novel RNN architecture based on a Hamiltonian system of oscillators.
result The proposed RNN mitigates exploding and vanishing gradient problems, providing state-of-the-art performance.
Fractal analysis is carried out on the stock market indices of seven European countries and the US. We find evidence of long range dependence in the log return series of the Mibtel (Italy) and the PX Glob (Czech Republic). Long range dependence implies that predictable patterns in the log returns do not dissipate quick…
A new regularizer boosts long-range dependency in sequence data.
problem Improving long-range dependency in sequence data models.
method Developed a mutual information regularizer to enhance sequence learning.
result The approach increases mutual information and likelihood on holdout data.
Combines CNN and Transformer for financial time series forecasting.
problem Forecasting financial time series, especially stock prices, is challenging due to short-term and long-term dependencies.
method Uses CNN for short-term dependencies and Transformer for long-term dependencies.
result Demonstrated superior performance in forecasting stock price changes compared to traditional methods.
New network distance based on Laplacian flow captures structure.
problem Measuring similarity between network objects.
method Introducing Laplacian flow to define a new diffusion distance.
result Demonstrated utility and advantage over existing distances.
Improved GNN handles long-range dependencies in multi-relational graphs.
problem Vanishing gradients in GNNs for multi-relational graphs.
method Proposes a Gated Graph Neural Network with improved long-range dependency handling.
result Outperforms popular GNN models in synthetic tasks.
QLSTM improves speech recognition by considering internal quaternion dependencies.
problem Weak internal dependencies in traditional RNNs for speech recognition.
method Proposes QLSTM, a quaternion-based LSTM that considers both external and internal dependencies.
result QLSTM achieves better performance with up to 2.8 times fewer parameters.
FDBM models use fractional Brownian motion to model complex stochastic processes.
problem Capturing memory effects and long-range dependencies in stochastic processes.
method Developed a generative diffusion bridge framework using a Markovian approximation of fractional Brownian motion.
result FDBM outperforms standard models in predicting future states and unpaired data translation.
A new recurrent unit alleviates vanishing gradients for long-term dependencies.
problem Vanishing gradients in recurrent neural networks make long-term dependencies hard to model.
method Proposes a new NRU architecture that avoids saturating activation functions and gates.
result Demonstrates superior performance across various tasks with and without long-term dependencies.
Study finds long-range dependence in financial markets, but deep generative models struggle to replicate it.
problem Long-range dependence in financial markets and challenges of deep generative models.
method Empirical analysis of financial data from three sectors, including LRD through various statistical methods and deep learning models.
result Deep generative models can reproduce stylized features but fail to capture long-range dependence structures.
We prove that every Teichmuller geodesic of a finite type surface contains a string of intersecting long, thick and dominant segments, such that the distance between consecutive segments is bounded. This is key to obtaining some results about Teichmuller geodesics which mimic those for hyperbolic geodesics. These resul…
Geography effect is investigated for the Chinese stock market including the Shanghai and Shenzhen stock markets, based on the daily data of individual stocks. The Shanghai city and the Guangdong province can be identified in the stock geographical sector. By investigating a geographical correlation on a geographical pa…
This paper presents empirical evidence using recently developed techniques in econophysics suggesting that the degree of long-range dependence in interest rates depends on the conduct of monetary policy. We study the term structure of interest rates for the US and find evidence that global Hurst exponents change dramat…
Neural M3 model adapts to diverse user behaviors over short and long timeframes.
problem Adapting to diverse user behaviors over short and long timeframes.
method Neural Multi-temporal-range Mixture Model (M3) combining short-term and long-term models with a learned gating mechanism.
result M3 consistently outperforms state-of-the-art sequential recommendation methods.