Paper introduces a new framework to improve sample efficiency in POMDPs learning.
problem Challenges in off-policy evaluation for POMDPs, especially with hidden states.
method Exploits the metric structure of belief space to relax coverage assumptions.
result Unified analysis technique yields tighter error bounds and sample efficiency improvements.
NBF combines deep learning with classical filtering for better belief tracking.
problem Maintaining distributions over hidden states in partially observable systems.
method Trains neural networks to map beliefs to fixed-length vectors, updating them with incoming observations and dynamics.
result NBF efficiently tracks shifting, multimodal beliefs without particle impoverishment.
FORBES learns flexible belief states for POMDPs using normalizing flows.
problem Accurately modeling belief states in POMDPs for high-dimensional, continuous spaces.
method Integrates normalizing flows into variational inference for continuous belief state learning.
result FORBES learns flexible belief states that enable multi-modal predictions and high-quality reconstructions.
A method for eliciting expert beliefs using preferential questions and normalizing flows.
problem Eliciting high-dimensional probability distributions from noisy judgments.
method Normalizing flows based on preferential questions with a novel functional prior.
result The method allows for the inference of arbitrarily flexible densities from preferential judgments.
A framework for cost of belief revision in uncertain agents.
problem Cost of revising beliefs in uncertain agents.
method Axiomatic framework for transport-based belief costs, postulates P0 and P1.
result Cost metric is conformally reweighted by Fisher information, leading to a cost floor diverging at certainty.
Generalized belief propagation converges to optimal solutions on graphs with motifs.
problem Understanding belief propagation on loopy graphs.
method Study of generalized belief propagation on graphs with motifs.
result Generalized belief propagation converges to the global optimum of the Bethe free energy.
Study risk sharing with Lambda VaR under diverse beliefs.
problem Risk sharing among agents with different beliefs.
method Use Lambda Value-at-Risk as preference, analyze under heterogeneous beliefs.
result Explicit formulas for risk sharing under various belief scenarios.
Unified kernel for prediction markets reduces belief variance forecast error.
problem Lack of standardized tools for quoting and hedging belief risk in prediction markets.
method Logit jump-diffusion model with risk-neutral drift, calibration pipeline, and coherent derivative layer.
result Model reduces forecast error compared to diffusion-only and probability-space baselines.
Belief propagation recovers backpropagation results.
problem Connection between backpropagation and belief propagation poorly understood.
method Converted backpropagation input to belief propagation input and showed results.
result Backpropagation is a special case of belief propagation.
New framework analyzes belief evolution in social networks.
problem Analyzing belief evolution in social networks.
method Proposes a new theoretical framework using Markov chain theory for horizontal and vertical transmission.
result Homophily-based networks do not converge to a single set of beliefs.
We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, …
Study links cognitive effort to thermodynamic principles, optimizing decision-making.
problem Understanding the energy cost of decision-making in living systems.
method Applied Landauer's principle to analyze belief dynamics in a two-armed bandit game.
result Cognitive energy cost correlates with optimal decision-making outcomes.
We propose an original particle-based implementation of the Loopy Belief Propagation (LPB) algorithm for pairwise Markov Random Fields (MRF) on a continuous state space. The algorithm constructs adaptively efficient proposal distributions approximating the local beliefs at each note of the MRF. This is achieved by cons…
New method approximates POMDPs with PB-MDPs, providing error bounds and practical algorithms.
problem Difficulty in solving POMDPs with continuous or hybrid state and observation spaces.
method Bounding particle filtering error and adapting MDP algorithms to POMDPs.
result General theory and practical algorithms for POMDPs with no direct dependence on state and observation space sizes.
Improved error correction using neural networks and belief propagation.
problem Inference in factor graphs with loops or poor approximations.
method Hybrid model combining FG-GNN and belief propagation.
result Hybrid model outperforms belief propagation in error correction tasks.
UWM-JEPA predicts future scenarios in belief space, improving accuracy in partially observed environments.
problem Predicting future scenarios in partially observed environments with uncertainty.
method Introduces UWM-JEPA, a JEPA world model with a density-matrix latent and learned unitary predictor.
result UWM-JEPA achieves 0.77 accuracy on a hidden-velocity indicator task, outperforming LSTM-JEPA.
This paper presents a solution for persistent monitoring of real-world stochastic phenomena, where the underlying covariance structure changes sharply across time, using a small number of mobile robot sensors. We propose an adaptive solution for the problem where stochastic real-world dynamics are modeled as a Gaussian…
This thesis investigates belief propagation's performance in graphical models with loops.
problem Belief propagation's performance and convergence guarantees in models with loops are uncertain.
method Investigates how model parameters affect belief propagation's performance, convergence, and approximation quality.
result Model parameters influence the number of fixed points, convergence properties, and approximation quality of belief propagation.
This work explores a social learning problem with agents having nonidentical noise variances and mismatched beliefs. We consider an N-agent binary hypothesis test in which each agent sequentially makes a decision based not only on a private observation, but also on preceding agents' decisions. In addition, the agents…
Model captures decision-making under bounded rationality with prior beliefs and market feedback.
problem Bounded rationality in decision-making with limited processing abilities.
method Maximum entropy principle applied to Quantal Response Statistical Equilibrium framework.
result Prior beliefs influence decision-making, altering the outcome of market feedback.
Belief Propagation algorithms are instruments used broadly to solve graphical model optimization and statistical inference problems. In the general case of a loopy Graphical Model, Belief Propagation is a heuristic which is quite successful in practice, even though its empirical success, typically, lacks theoretical gu…
By elaborating on the notion of linear belief functions (Dempster 1990; Liu 1996), we propose an elementary approach to knowledge representation for expert systems using linear belief functions. We show how to use basic matrices to represent market information and financial knowledge, including complete ignorance, stat…
This paper optimizes reinsurance contracts with belief differences between insurer and reinsurer.
problem Dynamic reinsurance design with heterogeneous beliefs under mean-variance framework.
method Modeling surplus process, applying partitioned domain optimization, solving HJB system.
result Optimal reinsurance contracts with belief heterogeneity are more complex than standard contracts.
Deep belief networks are a powerful way to model complex probability distributions. However, learning the structure of a belief network, particularly one with hidden units, is difficult. The Indian buffet process has been used as a nonparametric Bayesian prior on the directed structure of a belief network with a single…
In this paper I empirically investigate prediction markets for binary options. Advocates of prediction markets have suggested that asset prices are consistent estimators of the "true" probability of a state of the world being realized. I test whether the market reaches a "consensus." I find little evidence for converge…
New algorithm reduces communication in distributed learning by sharing compressed beliefs.
problem Efficiently learning from private data in a distributed setting with large hypothesis sets.
method Proposes a belief update rule for distributed cooperative learning with compressed (sparse or quantized) beliefs.
result Beliefs converge almost surely to optimal hypotheses with a linear concentration rate.
Generalizes information theory to evolving belief.
problem Measuring change in belief over time.
method Derives a general theory of information from first principles.
result Recover all information measures and interprets entropy as expected gain.
We propose a projected gradient dynamical system as a model for a bargaining scheme for an asset for which the two interested agents have personal valuations which do not initially coincide. The personal valuations are formed using subjective beliefs concerning the future states of the world and the reservation prices …
New α-BP algorithm improves belief propagation for graphs with loops.
problem Uncertainty in belief propagation for graphs with loops.
method Derive α-BP algorithm motivated by minimizing α-divergence. result Proves convergence conditions for α-BP. New group testing method uses Belief Propagation for accurate screening.
problem Efficiently identifying infected samples in large groups with minimal tests.
method Belief Propagation algorithm for inference in group testing schemes.
result Significantly increased accuracy of infection identification with fewer tests.
We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Montúfar and Ay,…
Belief propagation (BP) can do exact inference in loop-free graphs, but its performance could be poor in graphs with loops, and the understanding of its solution is limited. This work gives an interpretable belief propagation rule that is actually minimization of a localized α-divergence. We term this algorithm as $α…
Regularization methods, specifically those which directly alter weights like L1 and L2, are an integral part of many learning algorithms. Both the regularizers mentioned above are formulated by assuming certain priors in the parameter space and these assumptions, in some cases, induce sparsity in the parameter sp…
The paper explains stock market predictability through a model of heterogeneous beliefs.
problem Understanding and predicting stock market behavior based on news and investor beliefs.
method A discrete-time model of heterogeneous beliefs where some agents receive noisy signals about asset fundamentals.
result Momentum and reversal in stock prices arise from investors' incorrect beliefs about signal accuracy and fundamental values.
A number of problems in statistical physics and computer science can be expressed as the computation of marginal probabilities over a Markov random field. Belief propagation, an iterative message-passing algorithm, computes exactly such marginals when the underlying graph is a tree. But it has gained its popularity as …
Study shows price bubbles can exist even with heterogeneous beliefs.
problem Equilibrium price formation in markets with different belief groups.
method Analyzes continuous time asset trading with heterogeneous investors and mean reverting asset.
result Price bubbles may not form even with heterogeneous beliefs, contrary to initial expectations.
Develops a framework for quantifying agentic AI model risk using LLM-inferred Bayesian state filters.
problem Quantifying the risk of agentic AI systems due to uncertain beliefs and actions.
method Representing the system as a partially observed Markov decision process with latent states, Bayesian belief updates, control-dependent losses, and tail-risk functionals.
result Develops a rigorous framework for separating uncertainty quantification from risk measurement.
This paper presents a general framework for studying diverse beliefs in dynamic economies. Within this general framework, the characterization of a central-planner general equilbrium turns out to be very easy to derive, and leads to a range of interesting applications. We show how for an economy with log investors hold…
Unified Bayesian model explains in-context learning and activation steering in LLMs.
problem Understanding and controlling the behavior of large language models (LLMs) through prompts and activations.
method Developed a Bayesian model to explain and predict the effects of in-context learning and activation steering.
result Unified model predicts distinct phases and sudden shifts in LLM behavior, explaining prior empirical phenomena.
Recurrent networks learn beliefs from history in partially observable environments.
problem Learning optimal policies in partially observable environments.
method Trained recurrent neural networks to approximate value functions, measuring mutual information between hidden states and beliefs.
result Recurrent networks' hidden states correlate with beliefs of relevant state variables, improving expected return.
Unsupervised representation learning has succeeded with excellent results in many applications. It is an especially powerful tool to learn a good representation of environments with partial or noisy observations. In partially observable domains it is important for the representation to encode a belief state, a sufficie…
Deep belief networks can approximate any multivariate density with binary hidden units.
problem Approximating multivariate probability densities with binary hidden units.
method Sharp quantitative bounds on approximation error in terms of hidden units.
result Deep belief networks can approximate any multivariate density with binary hidden units under mild integrability requirements.
We present the first PAC optimal algorithm for Bayes-Adaptive Markov Decision Processes (BAMDPs) in continuous state and action spaces, to the best of our knowledge. The BAMDP framework elegantly addresses model uncertainty by incorporating Bayesian belief updates into long-term expected return. However, computing an e…
A new model validation framework for agentic AI systems based on POMDPs.
problem Model validation of agentic AI systems.
method A POMDP-based framework for belief-state, forecast, and policy validation.
result The framework decomposes autonomous decision making into information, beliefs, forecasts, actions, and utility.
POLAR learns efficient data acquisition policies using pretrained belief representations.
problem Challenges in learning effective policies for adaptive data acquisition.
method POLAR decouples representation learning from policy learning by leveraging pretrained predictive foundation models as belief-state encoders.
result POLAR outperforms state-of-the-art methods across diverse tasks while requiring fewer training samples.
The cost of belief changes with precision and is a hyperbolic geometry.
problem The cost of belief changes with precision and is a hyperbolic geometry.
method The cost of belief changes with precision and is a hyperbolic geometry.
result The cost of belief changes with precision and is a hyperbolic geometry.
SOLBP extends efficient inference to uncertain Bayesian networks.
problem Inference in uncertain Bayesian networks with second-order probabilities.
method Extends Loopy Belief Propagation to second-order Bayesian networks.
result Generates inferences consistent with sum-product networks, more efficient and scalable.
We introduce a formal language IE that is a variant of the language PAL developed in [van Benthem 2011] by adding a belief operator and a common belief operator,specializing to stochastic analysis. A constant symbol in the language denotes a stochastic process so that we can represent several financial events as formul…