arXiv research

A new method learns complex dynamical systems from data efficiently.

The recent discovered spatial-temporal information processing capability of bio-inspired Spiking neural networks (SNN) has enabled some interesting models and applications. However designing large-scale and high-performance model is yet a challenge due to the lack of robust training algorithms. A bio-plausible SNN mode…

SSMs combined with neural networks match Transformers in dynamic token selection.

DLFM models complex systems with uncertainty, outperforming traditional methods.

New metric compares noisy neural trajectories using optimal transport.

The theoretical explanation for deep neural network (DNN) is still an open problem. In this paper DNN is considered as a discrete-time dynamical system due to its layered structure. The complexity provided by the nonlinearity in the dynamics is analyzed in terms of topological entropy and chaos characterized by Lyapuno…

Neural EKF improves structural dynamics prediction.

Transformer models can approximate smooth functions with prompts, enhancing LLMs' dynamic capabilities.

This paper considers the computational power of constant size, dynamic Bayesian networks. Although discrete dynamic Bayesian networks are no more powerful than hidden Markov models, dynamic Bayesian networks with continuous random variables and discrete children of continuous parents are capable of performing Turing-co…

Proposes a framework to identify and correct model-form errors in nonlinear systems.

Study of deep neural networks using finite-time Lyapunov exponents.

Method learns dynamics of slow variables from stochastic data.

Study uses machine learning to predict nonlinear seismic brace behavior.

Model infers latent variables in sparse coding models using Langevin dynamics.

Recursive KalmanNet generalizes well in noisy, out-of-distribution scenarios.

To deal with various datasets over different complexity, this paper presents an self-adaptive learning model that combines the proposed Dynamic Connected Neural Decision Networks (DNDN) and a new pruning method--Dynamic Soft Pruning (DSP). DNDN is a combination of random forests and deep neural networks that enjoys bot…

Temporal-difference (TD) networks are a class of predictive state representations that use well-established TD methods to learn models of partially observable dynamical systems. Previous research with TD networks has dealt only with dynamical systems with finite sets of observations and actions. We present an algorithm…

Deep RL optimizes sensor placement in digital twins for dynamic data acquisition.

The paper analyzes the performance of delay-based reservoir computing using eigenvalue analysis.

New method predicts language model scaling risks with limited data.

CODE learns ODE dynamics from sparse data, outperforming neural and kernel methods.

Proposes a new model for better speech segmentation.

It was recently shown that neural ordinary differential equation models cannot solve fundamental and seemingly straightforward tasks even with high-capacity vector field representations. This paper introduces two other fundamental tasks to the set that baseline methods cannot solve, and proposes mixtures of stochastic …

The paper proposes a method to identify causal structure in complex dynamical systems.

PASTIS method selects simple models from noisy data.

This paper proposes a method for modeling event sequences with ambiguous timestamps, a time-discounting convolution. Unlike in ordinary time series, time intervals are not constant, small time-shifts have no significant effect, and inputting timestamps or time durations into a model is not effective. The criteria that …

Inspired by the success of deep learning techniques in the physical and chemical sciences, we apply a modification of an autoencoder type deep neural network to the task of dimension reduction of molecular dynamics data. We can show that our time-lagged autoencoder reliably finds low-dimensional embeddings for high-dim…

Time-lagged autoencoders (TAEs) have been proposed as a deep learning regression-based approach to the discovery of slow modes in dynamical systems. However, a rigorous analysis of nonlinear TAEs remains lacking. In this work, we discuss the capabilities and limitations of TAEs through both theoretical and numerical an…

Dynamic models learn from sparse, interacting sub-systems.

Experts predict significant adoption of decentralized finance by 2034, with traditional finance adapting.

We propose a simple discrete time semi-supervised graph embedding approach to link prediction in dynamic networks. The learned embedding reflects information from both the temporal and cross-sectional network structures, which is performed by defining the loss function as a weighted sum of the supervised loss from past…

Measuring entity relatedness is a fundamental task for many natural language processing and information retrieval applications. Prior work often studies entity relatedness in static settings and an unsupervised manner. However, entities in real-world are often involved in many different relationships, consequently enti…

DREAM model improves computational efficiency for non-linear effects in relational event models.

AER dynamically adjusts entropy regularization for better LLM reinforcement learning.

Learning workable representations of dynamical systems is becoming an increasingly important problem in a number of application areas. By leveraging recent work connecting deep neural networks to systems of differential equations, we propose \emph{variational integrator networks}, a class of neural network architecture…

This work learns effective dynamics from short-term data of stochastic systems.

A model is presented of the market dynamics to emphasis the effects of increasing returns to scale, including the description of the born and death of the adaptive producers. The evolution of market structure and its behavior with the technological shocks are discussed. Its dynamics is in good agreement with some empir…

Transformers can learn mixture of linear models efficiently.

Neural Processes combine the strengths of neural networks and Gaussian processes to achieve both flexible learning and fast prediction in stochastic processes. However, a large class of problems comprises underlying temporal dependency structures in a sequence of stochastic processes that Neural Processes (NP) do not e…

Transformers achieve near-optimal dynamic regret in non-stationary reinforcement learning.

Understanding biological network dynamics is a fundamental issue in various scientific and engineering fields. Network theory is capable of revealing the relationship between elements and their propagation; however, for complex collective motions, the network properties often transiently and complexly change. A fundame…

New method learns credit prices offline without interaction.

Price dynamics is analyzed in terms of a model which includes the possibility of effective forces due to trend followers or trend adverse strategies. The method is tested on the data of a minority-majority model and indeed it is capable of reconstructing the prevailing traders' strategies in a given time interval. Then…

New cooperative dynamics enhances retrieval performance in neural networks.

Gaussian processes for dynamical systems with Koopman equivariance.

Oddnet detects anomalies in dynamic networks using time series methods.

In conventional ODE modelling coefficients of an equation driving the system state forward in time are estimated. However, for many complex systems it is practically impossible to determine the equations or interactions governing the underlying dynamics. In these settings, parametric ODE model cannot be formulated. Her…

Method learns latent dynamics of complex systems from noisy data.