MinimalRNN simplifies RNNs for better interpretability and efficiency.
problem Improving interpretability and efficiency of RNNs.
method MinimalRNN uses a simplified structure with minimal updates, leading to efficient learning and testing.
result MinimalRNN learns disentangled RNN states and captures longer range dependencies.
PAGTN improves molecular property prediction by leveraging longer-range graph dependencies.
problem Local aggregation in GCNs misses higher-order graph properties.
method PAGTN uses path features and global attention layers to capture longer-range dependencies.
result PAGTN outperforms GCNs on various molecular property prediction datasets.
Combines PCA and message passing for better graph node embeddings.
problem Challenges of learning long-range dependencies in graph data.
method PCAPass: PCA + message passing for unsupervised node embeddings.
result Empirically shows competitive performance on node classification benchmarks.
We consider the estimation of large covariance and precision matrices from high-dimensional sub-Gaussian or heavier-tailed observations with slowly decaying temporal dependence. The temporal dependence is allowed to be long-range so with longer memory than those considered in the current literature. We show that severa…
Within its traditional range of perversity parameters, intersection cohomology is a topological invariant of pseudomanifolds. This is no longer true once one allows superperversities, in which case intersection cohomology may depend on the choice of the stratification by which it is defined. Topological invariance also…
Inspired by the recent literature on aggregation theory, we aim at relating the long range correlation of the stocks return volatility to the heterogeneity of the investors' expectations about the level of the future volatility. Based on a semi-parametric model of investors' anticipations, we make the connection betwee…
SALT models combine ARHMM and SLDS for efficient, interpretable time-series analysis.
problem Efficient modeling of systems with time-varying dynamics and long-range dependencies.
method Switching autoregressive low-rank tensor models parameterized with a low-rank factorization.
result SALT models provide a balance of interpretability and efficiency, outperforming ARHMMs and SLDSs.
Study finds strong long-range correlations in financial markets, especially over longer time scales.
problem Understanding long-range correlations in limit order book markets.
method Ultra-high frequency order book data from NASDAQ Nordic, detrended fluctuation analysis (DFA).
result Strong evidence of long-range correlation in inter-event durations, becoming stronger over longer time scales.
This paper develops a two-dimensional structural framework for valuing credit default swaps and corporate bonds in the presence of default contagion. Modelling the values of related firms as correlated geometric Brownian motions with exponential default barriers, analytical formulae are obtained for both credit default…
Transformer-XL extends language models' context length without disrupting temporal coherence.
problem Fixed-length context limitation in Transformers.
method Segment-level recurrence and novel positional encoding.
result 80% longer dependency, 450% longer than vanilla Transformers.
Introduces recency bias to improve time-series forecasting.
problem Lack of recency bias in standard Transformer attention for time-series data.
method Reweights attention scores with a smooth heavy-tailed decay to emphasize nearby observations.
result Recency-biased attention consistently improves sequential modeling and achieves competitive performance on time-series forecasting benchmarks.
Sparse Attentive Backtracking selectively backpropagates long-term dependencies in recurrent networks.
problem Difficulty in learning long-term dependencies in BPTT due to computational impracticality and biased gradient estimates.
method Sparse Attentive Backtracking learns an attention mechanism over past hidden states and selectively backpropagates through high-weight paths.
result Model learns long-term dependencies with fewer backpropagation steps, addressing biased gradient issues.
The paper explains why estimating a history-dependent policy can reduce MSE in reinforcement learning.
problem Understanding why history-dependent policies can improve MSE in off-policy evaluation.
method The paper derives a bias-variance decomposition of MSE for various OPE estimators, showing how history-dependent policies can decrease variance and increase bias.
result History-dependent policies can decrease the variance of importance sampling estimators, leading to lower MSE.
We present a case-study demonstrating the usefulness of Bayesian hierarchical mixture modelling for investigating cognitive processes. In sentence comprehension, it is widely assumed that the distance between linguistic co-dependents affects the latency of dependency resolution: the longer the distance, the longer the …
BigBird improves transformer performance on NLP tasks with longer sequences.
problem Quadratic dependency on sequence length in transformer models.
method Sparse attention mechanism reducing quadratic dependency to linear.
result Significant improvement in performance on NLP tasks like QA and summarization.
The paper examines how insurers manage risks and liquidity in a dynamic market.
problem Model uncertainty in insurance pricing and competitive equilibrium.
method Analyzes insurers' robustness preferences and optimization strategies for underwriting and liquidity management.
result Robust insurance pricing leads to higher premiums and equity valuations compared to a benchmark.
Proving that next-token prediction makes language models generate coherent long documents.
problem Understanding why language models generate coherent documents despite focusing on next-token prediction.
method Proving the power of next-token prediction in learning longer-range structure using Recurrent Neural Networks (RNN).
result Optimizing next-token prediction in RNNs yields a model that closely approximates the training distribution, even for long-range coherence.
Deep learning outperforms Black-Scholes in Brazilian Petrobras option pricing.
problem Improving option pricing accuracy for Petrobras stocks.
method Trained deep residual networks using a custom loss function with historical data.
result Deep learning achieved a 64.3% reduction in mean absolute error compared to Black-Scholes.
Reasoning models generate differently based on problem difficulty, not just length.
problem Understanding how reasoning models handle different problem difficulties.
method Examined hidden-state trajectories across competitive programming, mathematics, and Boolean satisfiability.
result Corrected trajectory geometry shows difficulty-dependent differences in reasoning models, with stronger effects in the code domain.
Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflect…
This paper examines the volatility and covariance dynamics of cash and futures contracts that underlie the Optimal Hedge Ratio (OHR) across different hedging time horizons. We examine whether hedge ratios calculated over a short term hedging horizon can be scaled and successfully applied to longer term horizons. We als…
Transformers learn chain-of-thought reasoning for longer problems, proving length generalization.
problem Challenging problems require deeper reasoning, but how do models generalize this to longer tasks?
method Theoretical analysis of transformers on synthetic state-tracking tasks, proving length generalization through attention concentration.
result Transformers can learn chain-of-thought reasoning for longer problems, proving length generalization.
SWD approximates GP for faster regression.
problem Costly GP matrix inversion limits large data analysis.
method SWD replaces matrix inversion with diagonalization for grid and short-range correlations.
result SWD-GP outperforms existing methods in predictive accuracy.
The paper extends Gromov's non-squeezing theorem to deformed symplectic forms.
problem Extending Gromov's non-squeezing theorem to deformed symplectic forms.
method Trap idea for holomorphic curves analogous to dynamical systems.
result The classical Gromov argument breaks down for deformed forms.
Scales attention for long contexts in LLMs.
problem Development of attention mechanisms for long context inference.
method Scale-invariant total attention and sparsity conditions, with a position-dependent transformation of logits.
result Scale-invariant attention scheme improves validation loss and long-context retrieval.
Study finds rough volatility models underperform in SPX option pricing.
problem Inconsistency of rough volatility models with SPX option prices.
method Empirical study using SPX options data, comparing rough and Markovian models.
result Rough volatility models with H∈(0,1/2) are inconsistent with SPX smiles, especially at short maturities. In this paper, we present the results of Monte Carlo simulations for two popular techniques of long-range correlations detection - classical and modified rescaled range analyses. A focus is put on an effect of different distributional properties on an ability of the methods to efficiently distinguish between short and …
Motivated by how transaction amount constrain trading volume and price volatility in stock market, we, in this paper, study the relation between volume and price if amount of transaction is given. We find that accumulative trading volume gradually emerges a kurtosis near the price mean value over a trading price range …
Fractal analysis is carried out on the stock market indices of seven European countries and the US. We find evidence of long range dependence in the log return series of the Mibtel (Italy) and the PX Glob (Czech Republic). Long range dependence implies that predictable patterns in the log returns do not dissipate quick…
Solves kernel dimension reduction while making features interpretable.
problem Making kernel dimension reduction methods interpretable.
method Projects onto a subspace before kernel feature mapping, using ISM for optimization.
result Extends ISM's theoretical guarantees to a family of kernels, enabling broader applicability.
News sentiment in U.S. economic newspapers has become more persistent over 45 years.
problem Understanding the temporal dynamics of U.S. economic news sentiment over time.
method Daily economic news sentiment index from 1980-2025, analyzed using sentiment indexes.
result News sentiment states have become more persistent, with longer residence times in optimistic or pessimistic regimes.
A new GP interpolation method for better predictive distributions in ranges of interest.
problem Improving predictive distributions in specific ranges of interest.
method Relaxed Gaussian process interpolation, relaxing interpolation constraints outside ranges of interest.
result Better predictive distributions in ranges of interest, especially in non-stationary cases.
A new regularizer boosts long-range dependency in sequence data.
problem Improving long-range dependency in sequence data models.
method Developed a mutual information regularizer to enhance sequence learning.
result The approach increases mutual information and likelihood on holdout data.
Paper explores learning patterns in binary sequences, finding no method consistently outperforms others.
problem Learning patterns in infinite binary sequences.
method Various learning methods are compared, finding no method consistently outperforms others.
result No learning method consistently outperforms others in predicting binary sequences.
Neural M3 model adapts to diverse user behaviors over short and long timeframes.
problem Adapting to diverse user behaviors over short and long timeframes.
method Neural Multi-temporal-range Mixture Model (M3) combining short-term and long-term models with a learned gating mechanism.
result M3 consistently outperforms state-of-the-art sequential recommendation methods.
Improved GNN handles long-range dependencies in multi-relational graphs.
problem Vanishing gradients in GNNs for multi-relational graphs.
method Proposes a Gated Graph Neural Network with improved long-range dependency handling.
result Outperforms popular GNN models in synthetic tasks.
A new GNN module learns geometric scattering features for better graph classification and feature exploration.
problem Learning long-range graph relations and extracting meaningful features from graphs.
method Proposes a learnable geometric scattering (LEGS) module in graph neural networks (GNNs), incorporating wavelet filters.
result LEGS-based GNNs outperform existing methods in graph classification and feature extraction tasks.
S4 model improves long sequence modeling efficiency.
problem Handling long-range dependencies efficiently in convolutional models.
method S4 model uses global convolution with decaying kernel weights.
result SGConv model achieves similar performance to S4 with improved efficiency.
New perspective on SGD reveals short-range memory effects in deep learning.
problem Understanding the efficacy of stochastic gradient descent (SGD) in deep learning.
method Proposed that SGD is a discretization of an SDE driven by fractional Brownian motion (FBM).
result SGD stays longer in flat minima, favoring generalization.
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
We study the probability distribution of stock returns at mesoscopic time lags (return horizons) ranging from about an hour to about a month. While at shorter microscopic time lags the distribution has power-law tails, for mesoscopic times the bulk of the distribution (more than 99% of the probability) follows an expon…
Bird sounds possess distinctive spectral structure which may exhibit small shifts in spectrum depending on the bird species and environmental conditions. In this paper, we propose using convolutional recurrent neural networks on the task of automated bird audio detection in real-life environments. In the proposed metho…
This paper presents empirical evidence using recently developed techniques in econophysics suggesting that the degree of long-range dependence in interest rates depends on the conduct of monetary policy. We study the term structure of interest rates for the US and find evidence that global Hurst exponents change dramat…
IGNN captures long-range graph dependencies using fixed-point equations.
problem Limited GNN ability to capture long-range graph dependencies.
method Fixed-point equilibrium equations involving implicitly defined state vectors, leveraging Perron-Frobenius theory and projected gradient descent.
result IGNN consistently captures long-range dependencies and outperforms state-of-the-art GNNs.
Novel graph neural network combines random walks with local message passing.
problem Graph neural networks struggle with long-range dependencies.
method Combines random walks with local message passing in a novel architecture.
result Significant performance improvements on graph benchmarks.
Last year, at least 30,000 scientific papers used the Kohn-Sham scheme of density functional theory to solve electronic structure problems in a wide variety of scientific fields, ranging from materials science to biochemistry to astrophysics. Machine learning holds the promise of learning the kinetic energy functional …
CauSTream forecasts streamflow by integrating causal graphs for better interpretability.
problem Streamflow forecasting lacks interpretability and generalization due to fixed causal models.
method CauSTream learns causal graphs for meteorological forcings and routing dependencies.
result CauSTream outperforms existing methods, especially at longer forecast windows.
This paper studies the dynamics of Brazilian interest rates for short-term maturities. The paper employs developed techniques in the econophysics literature and tests for long-range dependence in the term structure of these interest rates for the last decade. Empirical results suggest that the degree of long-range depe…