A Longitudinal Attribute-Conditioned Neural Network (LANTERN) framework for modeling health-state transition probabilities in irregular longitudinal data.
problem Estimating long-term care transition probabilities in irregular longitudinal health data.
method A neural network that learns from individual health history, incorporates time elapsed, and conditions on demographic and socioeconomic attributes.
result Improves severe disability discrimination and maintains strong calibration.
A new algorithm for deep Q-learning with robustness to state transition uncertainty.
problem Model uncertainty in state transitions for non-tabular, continuous state spaces.
method Distributionally robust approach using worst-case transition ball and dualized Bellman operator with Sinkhorn distance.
result Optimal policy found through solving non-linear Bellman equation with neural network parameterization.
This research improves dynamical systems understanding by identifying latent states and their nonlinear transitions.
problem Previous work on dynamical systems could not identify nonlinear transition dynamics, leading to unreliable predictions.
method Proposes a state-space modeling framework using variational auto-encoders to identify latent states and their nonlinear transition functions.
result Demonstrates high accuracy in recovering latent state dynamics and future prediction accuracy.
New model of vague knowledge without strict partitions or transitivity.
problem Standard economic models of information fail to capture real-world vague knowledge.
method Relaxing assumptions of transitivity and partition structure to formalize vague knowledge.
result Vague knowledge can distinguish some states but not partition the state space.
Study identifies transitions between traffic modes on Cologne motorways.
problem Understanding transitions between different traffic modes.
method Constructed state transition network, identified dominant states using PageRank algorithm.
result Identified seasonal dependence in traffic modes.
A framework combining HSMM and survival analysis for lifecycle-oriented mobility analysis.
problem Understanding individual metro usage dynamics over multi-year horizons.
method A state-based lifecycle modeling framework integrating HSMM and discrete-time survival analysis.
result Identification of interpretable mobility states, transition dynamics, and state-dependent exit and re-entry processes.
Develops methods to simulate rare transitions in molecular systems.
problem Rare transitions between metastable states in molecular systems are difficult to study due to limited data.
method Two novel methods: chain-based and midpoint-based approaches.
result Demonstrates effectiveness of methods in both data-rich and data-scarce scenarios.
New STH distance finds patterns in event timeseries without resampling.
problem Lack of efficient analysis methods for event and state timeseries.
method Define STE-ts, propose STH, leveraging both time and state duration.
result Improved precision and computation time compared to resampled metrics.
A new method for ILO with transition model disparity using an intermediary policy.
problem Learning tasks from expert observations with different transition dynamics.
method Training an intermediary policy to match the state transitions of the expert dataset.
result Our method outperforms existing ILO approaches with transition model mismatch.
Develops a flexible model for regime transitions in time series data.
problem Nonlinear and context-dependent regime transitions in time series data.
method Semi-parametric state-space model with learned transition functions.
result Improved recovery of nonlinear transition dynamics and earlier detection of regime changes.
The transition amplitudes between coherent states on a coherent state manifold are expressed in terms of the embedding of the coherent state manifold into a projective Hilbert space. Consequences for the dimension of projective Hilbert space and a simple geometric interpretation of Calabi's diastasis follows.
Proposes a new model for time series that considers smooth transitions between states.
problem Models assume instantaneous transitions between discrete states, ignoring gradual changes.
method Dynamical Wasserstein Barycentric (DWB) model that estimates system state and pure state distributions over time.
result Accurately learns pure state distributions and improves state estimation for transition periods.
We search for digital biomarkers from Parkinson's Disease by observing approximate repetitive patterns matching hypothesized step and stride periodic cycles. These observations were modeled as a cycle of hidden states with randomness allowing deviation from a canonical pattern of transitions and emissions, under the hy…
New method for state inference in state-space models with unknown dynamics.
problem State inference in state-space models with computationally expensive and undefined dynamics.
method Estimate state transition dynamics using a multi-output Gaussian process and Bayesian Neural Network as a surrogate model.
result Significant improvement in accuracy for state inference and prediction in non-stationary user models.
Study shows depth improves generalization in deep learning models.
problem Understanding why and when depth improves generalization in deep learning.
method Implementation-agnostic state-transition model to analyze depth and generalization.
result Identifies geometric and semigroup mechanisms that keep entropy contribution saturated or polynomial, clarifying depth's statistical advantage.
CNN detects phase transitions in Potts models without prior knowledge.
problem Detecting phase transitions in q-state Potts models using deep learning. method Trained a deep CNN on Ising model spin configurations and temperatures, then tested on Potts model images.
result Deep CNN accurately detects phase transitions in Potts models, including high- and low-temperature regions.
We present a representation for describing transition models in complex uncertain domains using relational rules. For any action, a rule selects a set of relevant objects and computes a distribution over properties of just those objects in the resulting state given their properties in the previous state. An iterative g…
A multi-task GP model tracks time-varying transition probabilities between two states.
problem Tracking time-varying transition probabilities between 'moves' and 'pauses' states.
method Kernel-based multi-task Gaussian Process model with time-variability and constraints.
result Enforces constraints while learning transition probabilities.
We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between "nearby" states. This is accomplished by defining a similarity function on the state space and scaling transition probabilities by pai…
The idea of forward rates stems from interest rate theory. It has natural connotations to transition rates in multi-state models. The generalization from the forward mortality rate in a survival model to multi-state models is non-trivial and several definitions have been proposed. We establish a theoretical framework f…
New algorithm reduces reinforcement learning regret for linear MDPs with unknown transitions.
problem Adversarial linear mixture MDPs with bandit feedback and unknown transition.
method Proposes a new algorithm with a least square estimator and self-normalized concentration.
result Achieves improved regret bound with high probability.
A novel multi-resolution Gaussian process model for efficient time traversal.
problem Inference for long sequences with fast and slow transitions is difficult.
method A novel Gaussian process state-space architecture composed of multiple components, each trained on a different resolution.
result The combined model allows efficient inference for arbitrarily long sequences with complex dynamics.
Manually authoring transition animations for a complete locomotion system can be a tedious and time-consuming task, especially for large games that allow complex and constrained locomotion movements, where the number of transitions grows exponentially with the number of states. In this paper, we present a novel approac…
State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference \emph{and learning} (i.e. state estimation and system identification) in nonlinear nonparametric state-space models. We place a Gauss…
Paper introduces a new value function for state transitions and optimal policy learning.
problem Learning optimal policies from state transitions and actions.
method Develops a forward dynamics model to maximize a novel value function Q(s,s′). result Demonstrates benefits in value function transfer, redundant action spaces, and off-policy learning.
New issue found in value-based reinforcement learning for stochastic environments.
problem Value-based reinforcement learning struggles with stochastic state transitions.
method Demonstrated using a multiobjective Markov Decision Process (MOMDP).
result Approaches may converge to Pareto-dominated solutions instead of optimal ones.
This paper identifies and estimates the label noise transition matrix without ground truth labels.
problem Learning with noisy labels and identifying the noise transition matrix.
method Building on Kruskal's identifiability results, the paper characterizes the identifiability of the label noise transition matrix for the generic case at the instance level.
result The necessity of multiple noisy labels in identifying the noise transition matrix for the generic case at the instance level.
New state-only IL algorithm tackles MDP transition mismatch.
problem Transition dynamics mismatch between expert and imitator MDPs.
method Adversarial state-only IL with two subproblems solved iteratively.
result Effective performance improvement in transition dynamics mismatch scenarios.
A new RL paradigm reduces state-action-value function approximation inefficiency.
problem Challenges in state-action-value function approximation for RL.
method State Action Separable Reinforcement Learning (sasRL) decouples action space from value function learning.
result sasRL achieves up to 75% better performance than state-of-the-art MDP-based RL algorithms.
A critical and challenging problem in reinforcement learning is how to learn the state-action value function from the experience replay buffer and simultaneously keep sample efficiency and faster convergence to a high quality solution. In prior works, transitions are uniformly sampled at random from the replay buffer o…
The study of the critical dynamics in complex systems is always interesting yet challenging. Here, we choose financial market as an example of a complex system, and do a comparative analyses of two stock markets - the S&P 500 (USA) and Nikkei 225 (JPN). Our analyses are based on the evolution of crosscorrelation struct…
Change-point analysis is a flexible and computationally tractable tool for the analysis of times series data from systems that transition between discrete states and whose observables are corrupted by noise. The change-point algorithm is used to identify the time indices (change points) at which the system transitions …
Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, it is difficult to understand what exactly they learn. Second, they tend to work poorly on sequences requiring long-term memorization, despite having this capacity in principle. We aim to address both…
PQR estimates reward functions from actions and states without assuming state-only rewards.
problem Estimating reward functions from actions and states without state-only assumptions.
method Deep learning approach that sequentially estimates policy, Q-function, and reward.
result PQR uniquely recovers true reward with known transitions and bounds error with unknown transitions.
SCPO learns robust policies without modeling disturbance, improving real-world task performance.
problem Poor performance of reinforcement learning in real-world tasks due to disturbance in transition dynamics.
method State-conservative policy optimization (SCPO) that reduces disturbance to state space and approximates it with a gradient-based regularizer.
result SCPO learns robust policies without prior knowledge of disturbance or simulators, improving performance in robot control tasks.
GDM models time series with smoother transitions and interpretable states.
problem Capturing smooth, variable-speed transitions and stochastic mixtures of states.
method Introduces a continuous relaxation of discrete states and a Gumbel noise model.
result Models real-world datasets more faithfully with smoother dynamics and interpretable states.
Combines deep state space models with diffusion models for better forecasting and capturing latent dynamics
problem Forecasting and capturing latent dynamics in time series
method DDSSM: Diffusion-driven state space model
result Empirically outperforms state-of-the-art deep SSM
We show that reinforcement learning agents that learn by surprise (surprisal) get stuck at abrupt environmental transition boundaries because these transitions are difficult to learn. We propose a counter-intuitive solution that we call Mutual Information Minimising Exploration (MIME) where an agent learns a latent rep…
We study the problem of predicting rare critical transition events for a class of slow-fast nonlinear dynamical systems. The state of the system of interest is described by a slow process, whereas a faster process drives its evolution and induces critical transitions. By taking advantage of recent advances in reservoir…
TempLe learns transition templates for efficient multi-task RL.
problem Efficiently transferring knowledge across different RL tasks with varying state/action spaces.
method Generates transition dynamics templates to abstract similarities between tasks.
result Achieves significantly lower sample complexity than single-task or multi-task methods.
We find numerical and empirical evidence for dynamical, structural and topological phase transitions on the (German) Frankfurt Stock Exchange (FSE) in the temporal vicinity of the worldwide financial crash. Using the Minimal Spanning Tree (MST) technique, a particularly useful canonical tool of the graph theory, two tr…
Improved simulation of phase transitions using hierarchical autoregressive networks.
problem Simulating phase transitions in complex systems.
method Hierarchical Autoregressive Neural (HAN) network sampling algorithm.
result Significant improvement in statistical uncertainty compared to the Wolff cluster algorithm.
Paper introduces OMD for ordered state transitions in SSMs.
problem Modeling ordered latent states in dynamic systems.
method Ordered Matrix Dirichlet (OMD) prior over ordered stochastic matrices.
result OMD models recover interpretable ordered latent structure without sacrificing predictive performance.
We introduce Dynamic Planning Networks (DPN), a novel architecture for deep reinforcement learning, that combines model-based and model-free aspects for online planning. Our architecture learns to dynamically construct plans using a learned state-transition model by selecting and traversing between simulated states and…
The committor function is a central object of study in understanding transitions between metastable states in complex systems. However, computing the committor function for realistic systems at low temperatures is a challenging task, due to the curse of dimensionality and the scarcity of transition data. In this paper,…
A new method uses deep learning to efficiently sample rare transitions for estimating committor functions.
problem Efficiently sampling rare transitions to estimate committor functions in high-dimensional problems.
method DASTR (Deep Adaptive Sampling on Transition Paths) method using deep generative models.
result Significantly improved accuracy in approximating committor functions through efficient sampling.
This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…
New methods use machine learning to simulate rare transitions in molecular systems.
problem Simulating rare transitions between metastable states in molecular dynamics.
method Generative models and reinforcement learning for importance sampling.
result Efficiently generated transition paths linking metastable states.