arXiv research

New issue found in value-based reinforcement learning for stochastic environments.

The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independe…

The paper models market crashes as phase transitions, finding dynamic transitions offer better predictions.

In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative f…

Study models forest transitions with deep learning for parameter estimation.

A novel approach models rating transitions using Lie groups and Deep Learning.

New neural processes use stacked Markov operators to improve flexibility.

Proposes a method for approximating transition densities of SDEs driven by gamma processes.

Neural models learn continuous-time Markov chain transition rates from data.

Develops a new framework for conditional independence.

New model for pairwise comparisons without stochastic transitivity.

Develops CLTs for Markov chain transition probabilities and policies.

There are various parametric models for analyzing pairwise comparison data, including the Bradley-Terry-Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this work, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are requir…

Algorithm samples constrained stochastic differential equations.

The interaction between transitivity and sparsity, two common features in empirical networks, implies that there are local regions of large sparse networks that are dense. We call this the blessing of transitivity and it has consequences for both modeling and inference. Extant research suggests that statistical inferen…

Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

Paper introduces TtT, market-implied transition time, from greenium term structure.

Method uses neural networks for high-dimensional committor function calculations.

Enhances machine learning for complex systems by embedding transition manifolds.

Curiosity-Critic improves world model training by focusing on cumulative prediction error.

New algorithm learns mixtures of subcubes efficiently, with applications to decision trees.

Generative Stochastic Networks (GSNs) have been recently introduced as an alternative to traditional probabilistic modeling: instead of parametrizing the data distribution directly, one parametrizes a transition operator for a Markov chain whose stationary distribution is an estimator of the data generating distributio…

Paper analyzes LPSA algorithm for constrained optimization, revealing phase transitions and bias-variance trade-offs.

Proposes a method to learn a transition operator for generating samples.

GDM models time series with smoother transitions and interpretable states.

Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.

Estimates bisimulation metrics from sample streams, not full transition models.

This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…

Contrastive learning estimates transition kernels for continuous-time stochastic processes.

Persistent entropy detects phase transitions in complex systems.

Modeling poverty transitions in India over 54 years, showing rising but persistent poverty.

Study on bandit problems with switching constraints, revealing phase transitions in regret.

NSFs learn SDE transition laws for efficient sampling.

Novel approach detects early warning indicators in complex systems.

Proposes a new stochastic method to calibrate climate risks in financial models.

A novel multi-resolution Gaussian process model for efficient time traversal.

Develops a machine learning framework for computing most probable paths in stochastic systems.

We analyze the European transition economies and show that time series for most of major indices exhibit (i) power-law correlations in their values, power-law correlations in their magnitudes, and (iii) asymmetric probability distribution. We propose a stochastic model that can generate time series with all the previou…

PQR estimates reward functions from actions and states without assuming state-only rewards.

Paper develops a diagnostic test for detecting convergence in SGD with constant step size.

Algorithm minimizes regret in dueling bandits with contextualized utilities.

Study predicts crypto-currency price collapses using standard deviation.

We discuss the geometric foundation behind the use of stochastic processes in the frame bundle of a smooth manifold to build stochastic models with applications in statistical analysis of non-linear data. The transition densities for the projection to the manifold of Brownian motions developed in the frame bundle lead …

New insights show stochastic initialization prevents token clustering in deep Transformers.

Optimizes bus schedules to improve on-time performance.

Study MMD for critical transitions in fast-slow systems, showing it's a good binary classifier.

New method extracts stochastic laws from data, including Lévy noise.