A new algorithm for deep Q-learning with robustness to state transition uncertainty.
problem Model uncertainty in state transitions for non-tabular, continuous state spaces.
method Distributionally robust approach using worst-case transition ball and dualized Bellman operator with Sinkhorn distance.
result Optimal policy found through solving non-linear Bellman equation with neural network parameterization.
A Longitudinal Attribute-Conditioned Neural Network (LANTERN) framework for modeling health-state transition probabilities in irregular longitudinal data.
problem Estimating long-term care transition probabilities in irregular longitudinal health data.
method A neural network that learns from individual health history, incorporates time elapsed, and conditions on demographic and socioeconomic attributes.
result Improves severe disability discrimination and maintains strong calibration.
Study identifies transitions between traffic modes on Cologne motorways.
problem Understanding transitions between different traffic modes.
method Constructed state transition network, identified dominant states using PageRank algorithm.
result Identified seasonal dependence in traffic modes.
Manually authoring transition animations for a complete locomotion system can be a tedious and time-consuming task, especially for large games that allow complex and constrained locomotion movements, where the number of transitions grows exponentially with the number of states. In this paper, we present a novel approac…
Improved simulation of phase transitions using hierarchical autoregressive networks.
problem Simulating phase transitions in complex systems.
method Hierarchical Autoregressive Neural (HAN) network sampling algorithm.
result Significant improvement in statistical uncertainty compared to the Wolff cluster algorithm.
New method for state inference in state-space models with unknown dynamics.
problem State inference in state-space models with computationally expensive and undefined dynamics.
method Estimate state transition dynamics using a multi-output Gaussian process and Bayesian Neural Network as a surrogate model.
result Significant improvement in accuracy for state inference and prediction in non-stationary user models.
This study uses persistent homology to analyze complex transitional networks from time series data.
problem Lack of effective tools to summarize complex topology in transitional networks.
method Persistent homology from topological data analysis applied to coarse-grained state-space networks (CGSSN).
result CGSSN improves dynamic state detection and noise robustness compared to other methods.
We study the crash dynamics of the Warsaw Stock Exchange (WSE) by using the Minimal Spanning Tree (MST) networks. We find the transition of the complex network during its evolution from a (hierarchical) power law MST network, representing the stable state of WSE before the recent worldwide financial crash, to a superst…
Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, it is difficult to understand what exactly they learn. Second, they tend to work poorly on sequences requiring long-term memorization, despite having this capacity in principle. We aim to address both…
CNN detects phase transitions in Potts models without prior knowledge.
problem Detecting phase transitions in q-state Potts models using deep learning. method Trained a deep CNN on Ising model spin configurations and temperatures, then tested on Potts model images.
result Deep CNN accurately detects phase transitions in Potts models, including high- and low-temperature regions.
Model improves robustness of neural network sequences without transition failures.
problem Learning and generating complex sequences of motor primitives without interference.
method Inspired by thalamocortical circuit, uses specific module for motif transitions.
result Improved robustness of sequence generation with no transition failures.
A new method uses deep learning to efficiently sample rare transitions for estimating committor functions.
problem Efficiently sampling rare transitions to estimate committor functions in high-dimensional problems.
method DASTR (Deep Adaptive Sampling on Transition Paths) method using deep generative models.
result Significantly improved accuracy in approximating committor functions through efficient sampling.
We present a representation for describing transition models in complex uncertain domains using relational rules. For any action, a rule selects a set of relevant objects and computes a distribution over properties of just those objects in the resulting state given their properties in the previous state. An iterative g…
We introduce Dynamic Planning Networks (DPN), a novel architecture for deep reinforcement learning, that combines model-based and model-free aspects for online planning. Our architecture learns to dynamically construct plans using a learned state-transition model by selecting and traversing between simulated states and…
Develops M2 model for next-basket recommendation considering user preferences, item popularity, and transition patterns.
problem Next-basket recommendation problem considering user preferences, item popularity, and transition patterns.
method Mixed model with preferences, popularities, and transitions (M2) using ed-Trans for transition patterns among items.
result Significantly outperforms state-of-the-art methods on all datasets in all tasks, with up to 22.1% improvement.
Supervised learning with a deep convolutional neural network is used to identify the QCD equation of state (EoS) employed in relativistic hydrodynamic simulations of heavy-ion collisions from the simulated final-state particle spectra ρ(pT,Φ). High-level correlations of ρ(pT,Φ) learned by the neural network act a…
Machine learning predicts synchronization transitions in unknown systems.
problem Predicting synchronization transitions in systems with unknown equations.
method Developed a 'parameter-aware' machine learning scheme using reservoir computing or echo state networks.
result Machine learning accurately predicts synchronization transitions, including hysteresis loops.
Deep networks learn features suddenly, akin to a phase transition.
problem Understanding sudden feature learning in deep networks.
method Adaptive kernel approach applied to two teacher-student models.
result Feature learning and Grokking are analogous to a first-order phase transition.
Neural models learn continuous-time Markov chain transition rates from data.
problem Learning transition rates for complex stochastic systems.
method Neural networks to model nonlinear transition rates from observed data.
result Neural models outperform traditional methods in accuracy.
Proposes a new model to analyze mortgage delinquency transitions.
problem Analyzing mortgage delinquency transitions in a flexible yet identifiable way.
method Combines structured additive predictor with neural network for complex interactions, orthogonalising components for identifiability.
result The semi-structured model provides modest gains in discrimination compared to a structured model, especially in the early prediction spans.
This paper studies the dynamic generator model for spatial-temporal processes such as dynamic textures and action sequences in video data. In this model, each time frame of the video sequence is generated by a generator model, which is a non-linear transformation of a latent state vector, where the non-linear transform…
In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes. In particular, we aim for numerical schemes for the committor function, the central object of transition path theory, which satisfies a high-dimensional Fokker-Planck equation. By…
Percolation on complex networks has been used to study computer viruses, epidemics, and other casual processes. Here, we present conditions for the existence of a network specific, observation dependent, phase transition in the updated posterior of node states resulting from actively monitoring the network. Since tradi…
Enhances inference of spreading processes using neural-network priors.
problem Estimating initial states of graph processes from partial observations.
method Bayesian framework with single-layer perceptron neural network for initial states; hybrid BP-AMP algorithm.
result Model exhibits first-order phase transitions, creating a statistical-to-computational gap.
Generative Stochastic Networks (GSNs) have been recently introduced as an alternative to traditional probabilistic modeling: instead of parametrizing the data distribution directly, one parametrizes a transition operator for a Markov chain whose stationary distribution is an estimator of the data generating distributio…
We find numerical and empirical evidence for dynamical, structural and topological phase transitions on the (German) Frankfurt Stock Exchange (FSE) in the temporal vicinity of the worldwide financial crash. Using the Minimal Spanning Tree (MST) technique, a particularly useful canonical tool of the graph theory, two tr…
We study layered neural networks of rectified linear units (ReLU) in a modelling framework for stochastic training processes. The comparison with sigmoidal activation functions is in the center of interest. We compute typical learning curves for shallow networks with K hidden units in matching student teacher scenarios…
Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets…
Deep neural networks near edge of chaos show universal scaling laws.
problem Understanding the behavior of deep neural networks near critical points.
method Analogy to absorbing phase transitions in statistical mechanics, deterministic propagation dynamics, mean-field and directed percolation universality classes.
result Deep neural networks exhibit universal scaling laws near the edge of chaos.
Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL. On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori. However, in complex environments where transition models need t…
The stability of money value is an important requisite for a functioning economy, yet it critically depends on the actions of participants in the market themselves. Here we model the value of money as a dynamical variable that results from trading between agents. The basic trading scenario can be recast into an Ising t…
Federated learning interprets temporal dynamics across clients with graph attention.
problem Interpreting temporal patterns across decentralized, heterogeneous systems with nonlinear dynamics.
method Graph Attention Network for learning state transition models over latent states communicated between clients.
result First interpretable characterization of cross-client temporal interdependencies in decentralized nonlinear systems.
Linear recurrent networks explain reinforcement learning performance in partially observable settings.
problem Understanding why linear recurrent networks work in reinforcement learning with partial observability.
method Constructed and studied two linear filters for HMMs and action-controlled HMMs.
result Linear filters serve as sufficient statistics and reduce state ambiguity, explaining empirical reinforcement learning success.
Proposes a new method for GNNs that avoids iterative node state convergence.
problem Iterative computation of node states in GNNs is inefficient and requires many epochs.
method Constrained optimization in the Lagrangian framework to learn transition function and node states simultaneously.
result The proposed method compares favorably with existing models on various benchmarks.
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics. This idea a…
Neural Physicist learns physical dynamics from images.
problem Learning meaningful physical state representations and accurate state transitions from image sequences.
method Neural Physicist uses VAE for state extraction, NP for parameters, and SSM for dynamics.
result Achieves long-term predictions and identifies system degrees of freedom.
SA-GFN corrects biases in GFlowNets due to graph symmetries.
problem Systematic biases in state transition probability computations.
method Incorporates symmetry corrections into the learning process through reward scaling.
result Eliminates need for explicit state transition computations.
This research improves dynamical systems understanding by identifying latent states and their nonlinear transitions.
problem Previous work on dynamical systems could not identify nonlinear transition dynamics, leading to unreliable predictions.
method Proposes a state-space modeling framework using variational auto-encoders to identify latent states and their nonlinear transition functions.
result Demonstrates high accuracy in recovering latent state dynamics and future prediction accuracy.
We employ unsupervised machine learning techniques to learn latent parameters which best describe states of the two-dimensional Ising model and the three-dimensional XY model. These methods range from principal component analysis to artificial neural network based variational autoencoders. The states are sampled using …
The paper characterizes brain states and transitions using functional MRI data.
problem Characterizing the dynamic reconfiguration of neural systems in brain states.
method Bayesian model-based characterization of latent brain states and posterior predictive discrepancy using the latent block model.
result The model detects transitions between latent brain states and identifies distinctive community patterns in task-fMRI data.
New model of vague knowledge without strict partitions or transitivity.
problem Standard economic models of information fail to capture real-world vague knowledge.
method Relaxing assumptions of transitivity and partition structure to formalize vague knowledge.
result Vague knowledge can distinguish some states but not partition the state space.
TMTF improves time series visualization by separating dynamic regimes.
problem Misleading global transition matrix in time series analysis.
method Temporal chunking, local transition matrices, and image assembly.
result Temporal segmentation reveals distinct transition dynamics.
Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential energy landscapes in molecular science. The number of local minima and transition sta…
A framework combining HSMM and survival analysis for lifecycle-oriented mobility analysis.
problem Understanding individual metro usage dynamics over multi-year horizons.
method A state-based lifecycle modeling framework integrating HSMM and discrete-time survival analysis.
result Identification of interpretable mobility states, transition dynamics, and state-dependent exit and re-entry processes.
Develops methods to simulate rare transitions in molecular systems.
problem Rare transitions between metastable states in molecular systems are difficult to study due to limited data.
method Two novel methods: chain-based and midpoint-based approaches.
result Demonstrates effectiveness of methods in both data-rich and data-scarce scenarios.
New STH distance finds patterns in event timeseries without resampling.
problem Lack of efficient analysis methods for event and state timeseries.
method Define STE-ts, propose STH, leveraging both time and state duration.
result Improved precision and computation time compared to resampled metrics.
A new method for ILO with transition model disparity using an intermediary policy.
problem Learning tasks from expert observations with different transition dynamics.
method Training an intermediary policy to match the state transitions of the expert dataset.
result Our method outperforms existing ILO approaches with transition model mismatch.
Machine learning detects tipping points in complex systems.
problem Detecting abrupt shifts in complex dynamical systems.
method Equilibrium-informed neural networks (EINNs) trained on candidate equilibrium states.
result EINNs can identify critical thresholds in nonlinear systems.