New model captures state-dependent variability in partially observed systems.
problem Structured stochasticity not captured by constant-variance models.
method State-coupled stochastic volatility framework with particle expectation-maximization.
result Model consistently reduces recovery bias under partial observation.
Method estimates observation functions in state-space models without supervision.
problem Unsupervised learning of non-invertible observation functions in nonlinear state-space models.
method Nonparametric generalized moment method using constrained regression.
result Estimates function space of identifiability from state process.
Intelligent agents can cope with sensory-rich environments by learning task-agnostic state abstractions. In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP). Our m…
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
Adversarial attacks on probabilistic state-space models affect latent state and policy decisions.
problem Robust reinforcement learning under adversarial observability.
method Analyzing adversarial attacks on linear probabilistic state-space models.
result Demonstrating the influence of adversarial observations on latent state and policy decisions.
Sketch Tomography improves quantum state estimation accuracy.
problem Efficiently estimating quantum states, especially MPS states.
method Hybridizes classical shadow protocol with tensor train ansatz.
result Proven convergence with quadratic sample complexity.
Improves LSTM performance by initializing states via manifold learning.
problem Improving LSTM performance through better initialization.
method Learning an intrinsic data manifold to initialize LSTM internal states.
result Improved LSTM performance through consistent initialization.
In nonlinear state-space models, sequential learning about the hidden state can proceed by particle filtering when the density of the observation conditional on the state is available analytically (e.g. Gordon et al., 1993). This condition need not hold in complex environments, such as the incomplete-information equili…
Helps visually impaired users make better decisions by adjusting their observations.
problem Systematic biases in users' perception and processing of visual information.
method Synthesizes new observations based on true observations to correct user biases.
result Significant improvement in task performance for users with various biases.
Enhances RL in partially observable, noisy environments by uncovering causal states.
problem Making decisions based on incomplete and noisy observations in partially observable Markov decision processes (P2OMDPs). method Causal State Representation under Asynchronous Diffusion Model (CaDiff) framework, incorporating a novel asynchronous diffusion model (ADM) and a new bisimulation metric.
result Enhances returns by at least 14.18% compared to baselines on Roboschool tasks.
SOAD model improves data assimilation for nonlinear systems.
problem Challenges in classical data assimilation with high nonlinearity.
method State-Observation Augmented Diffusion (SOAD) model for data-driven assimilation.
result SOAD model matches true posterior distribution under mild assumptions.
We analyze how an observer synchronizes to the internal state of a finite-state information source, using the epsilon-machine causal representation. Here, we treat the case of exact synchronization, when it is possible for the observer to synchronize completely after a finite number of observations. The more difficult …
Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…
New BED method handles online inference for partially observed dynamical systems.
problem Optimizing data collection for partially observable, partially online dynamical systems.
method Derived estimators of expected information gain and its gradient for SSMs, using nested particle filters.
result Successfully handles both partial observability and online inference in realistic models.
New algorithm improves reinforcement learning from partial observations.
problem Inferior performance of algorithms in real-world reinforcement learning due to partial observability.
method Representation-based approach to POMDPs, leading to a tractable algorithm.
result Empirically demonstrates superior performance with partial observations.
This paper tackles belief-state selection in simulators with latent states.
problem Selecting among approximate belief-state samplers for simulators with latent variables.
method Reduces belief-state selection to conditional distribution selection, develops algorithms and analyses.
result Different formulations of belief-state selection have varying guarantees under different roll-out methods.
This paper addresses the problem of filtering with a state-space model. Standard approaches for filtering assume that a probabilistic model for observations (i.e. the observation model) is given explicitly or at least parametrically. We consider a setting where this assumption is not satisfied; we assume that the knowl…
Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically consistent estimates of PSR parameters can be learned effi…
Proposes neural delay differential equations for stable system identification with partially observed states.
problem Learning stable models for systems with partial or delayed observations.
method Augments states with history, uses neural delay differential equations, and ensures stability through time delay analysis.
result The approach ensures stability of learned models for partially observed systems.
A new method for state estimation in state-space models using incomplete data.
problem State estimation in nonlinear state-space models with incomplete observations.
method Statistical analysis of incomplete observations, score function, observed information matrices, EM-gradient-particle filtering.
result Maximum likelihood estimation of state-vector with explicit form of observed information matrix.
We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states. Under certain identifiability assumptions, we demonstrate how to estimate a mapping from the observations to latent states inductively through a sequence of regression and clustering steps -- where p…
Efficient RL in large POMDPs with latent determinism and embeddings.
problem Efficient reinforcement learning in large-scale POMDPs with latent states and observations.
method Conditional Hilbert space embeddings, linear optimal Q-function, deterministic latent transitions, gap assumption. result Computationally and statistically efficient algorithm for exact optimal policy.
The paper tackles restless bandits with limited observation, proposing a method to analyze and approximate their optimal strategies.
problem Restless bandits with limited observation.
method General probabilistic model, PCL analysis, and approximation process.
result The proposed method can transform the problem into a finite-state problem, enabling the use of existing algorithms.
This paper provides an analysis of the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data) in the context of reinforcement learning with partial observability. Our theoretical analysis formally characterizes that while potentially increasin…
New method improves DRL robustness against adversarial state observations.
problem Adversarial attacks on deep reinforcement learning agents observing state data.
method State-adversarial Markov decision process (SA-MDP) and policy regularization.
result Significant improvement in robustness of DRL algorithms under adversarial attacks.
Robust RL with learned optimal adversary improves agent performance under adversarial state observations.
problem Ensuring reinforcement learning agents' robustness against adversarial perturbations of state observations.
method Proposed a framework of alternating training with learned adversaries (ATLA) to find optimal adversarial policies and enhance agent robustness.
result ATLA achieves state-of-the-art performance under strong adversaries in continuous control environments.
A new method for state estimation on complex networks.
problem Reconstructing latent dynamics from multivariate time-series on topological cell complexes.
method Topology-aware state space framework derived from stochastic partial differential equations, with state evolution following heat-like topological diffusion.
result The proposed method successfully recovers latent states and topological structures in real-world networks.
Recurrent networks learn beliefs from history in partially observable environments.
problem Learning optimal policies in partially observable environments.
method Trained recurrent neural networks to approximate value functions, measuring mutual information between hidden states and beliefs.
result Recurrent networks' hidden states correlate with beliefs of relevant state variables, improving expected return.
Algorithm improves imitation learning from visual data in partially observable environments.
problem Imitation learning from visual observations with missing expert actions and partial observability.
method Theoretical analysis and Latent Adversarial Imitation from Observations algorithm combining adversarial and latent representations.
result Latent Adversarial Imitation from Observations achieves state-of-the-art performance in high-dimensional robotic tasks.
New framework for reinforcement learning with sporadic state observations.
problem Partial observability in reinforcement learning.
method Action-Triggered Sporadically Traceable Markov Decision Processes (ATST-MDPs).
result Optimistic algorithm achieving regret bound for episodic learning.
Enhanced feedback model improves sample-efficiency in POMDPs.
problem Exponential hardness of learning in POMDPs.
method Multiple observations in hindsight feedback model.
result Sample-efficient learning possible for new subclasses of POMDPs.
Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage b…
The paper develops a method to learn navigation costs from expert demonstrations in partially observable environments.
problem Learning navigation costs from expert demonstrations in partially observable environments.
method Develops a cost function representation composed of a probabilistic occupancy encoder and a cost encoder, optimized by differentiating the error between demonstrated controls and a control policy computed from the cost encoder.
result The method outperforms baseline IRL algorithms in robot navigation tasks, improving both training and test-time efficiency.
EnSF uses image inpainting to handle partial observations in data assimilation.
problem Data assimilation challenges with partial observations.
method EnSF integrates image inpainting with diffusion models to predict unobserved states.
result EnSF successfully tracks SQG dynamics with partial observations.
The Kalman filter (KF) is used in a variety of applications for computing the posterior distribution of latent states in a state space model. The model requires a linear relationship between states and observations. Extensions to the Kalman filter have been proposed that incorporate linear approximations to nonlinear m…
A new metric detects non-Markovian states in partially observable environments.
problem Learning state representations in partially observable environments.
method Introducing the λ-discrepancy metric to detect non-Markovian states. result The λ-discrepancy is zero for Markov processes and non-zero for partially observable environments. A Hawkes process with state-dependent factor models order flows in limit order books.
problem Modeling order flows in limit order books for better market prediction.
method A Hawkes process with a state-dependent factor for conditional intensity estimation.
result State-dependent formulations improve the fit of LOB models to financial data.
We solve a broad class of sequential decision-making problems with partially observed states.
problem Sequential decision-making under uncertainty with partially observed states.
method Modeling as a partially observed Markov decision process (POMDP) and separating state and modulation process.
result The approach allows for specialized approximate solution procedures.
Paper addresses state estimation in sensor networks with intermittent data.
problem State estimation in sensor networks with packet dropouts and corrupted observations.
method Bayesian variational inference with a dual-mask generative model.
result The method effectively identifies system states and noise parameters.
The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…
New algorithm infers trajectories from partial observations using optimal transport.
problem Inferring trajectories from partial observations of coupled systems.
method Extends MFL algorithm to latent SDEs using observable state space models and partial observations.
result Experiments show significant outperformance over latent-free baseline.
Latent-EnSF improves data assimilation for high-dimensional systems with sparse observations.
problem Challenges in high-dimensional, nonlinear Bayesian filtering with sparse observations.
method A novel data assimilation method using latent representations and a coupled VAE for efficient state encoding and reconstruction.
result Latent-EnSF outperforms traditional methods in accuracy, convergence, and efficiency for complex systems.
We extend a recent synchronization analysis of exact finite-state sources to nonexact sources for which synchronization occurs only asymptotically. Although the proof methods are quite different, the primary results remain the same. We find that an observer's average uncertainty in the source state vanishes exponential…
A new method for analyzing high-dimensional time-series data using deep neural networks.
problem Challenges in modeling high-dimensional time-series data with explicit state and observation processes.
method Deep Direct Discriminative Decoders (D4) for high-dimensional observation processes.
result D4 outperforms traditional SSMs and RNNs in various time-series data applications.
PINNs solve neuronal parameter and state estimation problems with limited data.
problem Estimating parameters and hidden state variables from noisy partial data in multiscale neuronal models.
method Physics-informed neural networks (PINNs) for joint state and parameter estimation.
result PINNs deliver robust and accurate parameter inference and state reconstruction, even with limited data.
A three-state model based on the Potts model is proposed to simulate financial markets. The three states are assigned to "buy", "sell" and "inactive" states. The model shows the main stylized facts observed in the financial market: fat-tailed distributions of returns and long time correlations in the absolute returns. …
This paper proposes a new estimation algorithm for the parameters of an HMM as to best account for the observed data. In this model, in addition to the observation sequence, we have \emph{partial} and \emph{noisy} access to the hidden state sequence as side information. This access can be seen as "partial labeling" of …
CVRL tackles complex visual observations in reinforcement learning.
problem Complex visual observations in natural environments.
method Contrastive Variational Reinforcement Learning (CVRL) learns a contrastive variational model by maximizing mutual information between latent states and observations.
result CVRL achieves comparable performance with state-of-the-art model-based DRL methods and significantly outperforms them on tasks with complex observations.