New method uses entropy to improve policy gradient exploration.
problem Limited exploration in policy gradient methods.
method Entropy regularization with discounted future state distribution.
result Proves convergence to locally optimal policy.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “state distribution”
New method reduces state distribution mismatch in off-policy RL.
problem State distribution mismatch in off-policy RL algorithms.
method Develops a novel constrained off-policy gradient objective to minimize state distribution shift.
result Minimizing state distribution shift improves performance in off-policy RL algorithms.
This work improves policy optimization by maximizing entropy of state distribution, leading to better exploration.
problem Lack of exploration in state space when maximizing policy entropy.
method Proposes maximizing the entropy of a lower bound approximation to the state weighting distribution, based on latent space representation.
result Entropy regularization based on marginal state distribution achieves superior state space coverage and better performance in various domains.
In this letter we borrow from the inference techniques developed for unbounded state-cardinality (nonparametric) variants of the HMM and use them to develop a tuning-parameter free, black-box inference procedure for Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have latent states consisting…
The method approximates stationary distributions of Markov models by truncating irrelevant states.
problem Computing the stationary distribution of complex Markov models is computationally challenging.
method A state-space lumping scheme that aggregates states in a grid structure, iteratively refining the state-space.
result The method provides a well-justified finite-state projection tailored to the stationary behavior of Markov models.
New method estimates state-action stationary distribution for better off-policy policy evaluation.
problem Accurately estimating state-action stationary distribution for off-policy policy evaluation.
method Estimated Mixture Policy (EMP) for state and state-action stationary distribution corrections.
result Empirical validation shows improved accuracy over state-of-the-art methods.
CFIL uses coupled flows to model state distributions for imitation learning.
problem Lack of explicit modeling of state distributions in reinforcement and imitation learning.
method Coupled normalizing flows for state and state-action distributions.
result CFIL achieves state-of-the-art performance on benchmark tasks.
Study optimal offline RL with uncertainty sets and distribution shifts.
problem Optimal offline reinforcement learning with limited data.
method Construct uncertainty sets and distribution shifts, solve robust Markov decision process.
result Least conservative estimator for unknown true distribution.
State-constrained offline RL expands RL's learning scope.
problem Batch-constrained offline RL limits policies to seen actions.
method Introduces state-constrained offline RL focusing on state distribution.
result Policy can take high-quality out-of-distribution actions.
This paper tackles belief-state selection in simulators with latent states.
problem Selecting among approximate belief-state samplers for simulators with latent variables.
method Reduces belief-state selection to conditional distribution selection, develops algorithms and analyses.
result Different formulations of belief-state selection have varying guarantees under different roll-out methods.
Quantum models learn unitary actions on entangled states from product states.
problem Generalization to out-of-distribution data in quantum machine learning.
method Proved out-of-distribution generalization for learning unitary actions.
result Learned unitary actions on entangled states from product states.
In this article we present an alternative model for the distribution of household incomes in the United States. We provide arguments from two differing perspectives which both yield the proposed income distribution curve, and then fit this curve to empirical data on household income distribution obtained from the Unite…
Exponential and power-law probability distributions of wealth and income in the United Kingdom and the United Statescond-mat.stat-mech
We present the data on wealth and income distributions in the United Kingdom, as well as on the income distributions in the individual states of the USA. In all of these data, we find that the great majority of population is described by an exponential distribution, whereas the high-end tail follows a power law. The di…
The paper analyzes variational autoencoders for state space models with risk bounds.
problem Analyzing the risk associated with variational autoencoders for state space models.
method Backward factorization of variational distributions to analyze excess risk, providing oracle inequalities and upper bounds.
result Explicit upper bounds on variational estimation error for state space models under strong mixing assumptions.
Proposes a new model for time series that considers smooth transitions between states.
problem Models assume instantaneous transitions between discrete states, ignoring gradual changes.
method Dynamical Wasserstein Barycentric (DWB) model that estimates system state and pure state distributions over time.
result Accurately learns pure state distributions and improves state estimation for transition periods.
New algorithm for ML models in gradually adapting data settings.
problem Training models when data distribution reacts to the model over time.
method Stateful Performative Gradient Descent (Stateful PerfGD)
result Stateful PerfGD minimizes performative loss in gradually adapting data settings.
The problem of state estimation for unobservable distribution systems is considered. A deep learning approach to Bayesian state estimation is proposed for real-time applications. The proposed technique consists of distribution learning of stochastic power injection, a Monte Carlo technique for the training of a deep ne…
A new method learns state and proposal dynamics in state-space models using neural networks.
problem Inference in non-linear state-space models.
method StateMixNN method using neural networks for proposal and transition distributions.
result Significantly improved recovery of hidden state, especially in highly non-linear scenarios.
Learn true model from metastable samples of discrete distributions.
problem Learning true model from samples that get stuck in metastable states.
method Conditional likelihood estimator based on strong metastability condition.
result True model can be recovered from metastable samples of discrete distributions.
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …
The statistical distribution of money and the rate of money transferencecond-mat.stat-mech
The distribution of money is analysed in connection with the Boltzmann distribution of energy in the degenerate states of molecules. Plots of the population density of income distribution for various countries are well reproduced by a Gamma function, confirming the validity of the statistical distribution at equilibriu…
DeepGG generates graph distributions for drug discovery and molecular design.
problem Learning graph distributions for various applications.
method Improved deep graph generator based on deep state machines with graph and node embeddings.
result The state machine design favors specific graph distributions.
Improves out-of-distribution detection in neural networks.
problem Detecting out-of-distribution examples in neural networks.
method Normalizing flows and residual flow architecture for expressive density modeling.
result Significantly improved true negative rate (77.5%) compared to state-of-the-art (56.7%).
MPE framework proves universal approximation for quantum data distribution.
problem Challenges in generating quantum data from underlying distributions.
method Many-body Projected Ensemble (MPE) framework for quantum state design.
result MPE can approximate any quantum distribution within 1-Wasserstein distance error.
Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage b…
IHT improves sparse distribution learning.
problem Learning sparse discrete distributions.
method Iterative hard thresholding as a solution, with a greedy approximate projection.
result IHT achieves state of the art results for sparse distribution learning.
State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference \emph{and learning} (i.e. state estimation and system identification) in nonlinear nonparametric state-space models. We place a Gauss…
New approach models how explanations shift with distribution changes.
problem Model performance drops with changing input data distributions.
method Models explanation shifts and compares them to state-of-the-art techniques.
result Modeling explanation shifts better detects out-of-distribution behavior.
In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We achieve this by using quantile regression to approximate the full quantile function for the state-action return distribution. By reparameter…
Improved state estimation in nonlinear models using amortized backward variational inference.
problem State estimation in general state-space models.
method Amortized backward variational inference with neural network parameters.
result Linear growth of variational approximation error in number of observations.
Paper develops a new state estimation method for nonlinear systems.
problem State estimation for nonlinear state-space models is intractable.
method Developed a variational inference approach based on Gaussian approximations.
result The method outperforms alternative Gaussian approaches in various examples.
RegFlow models future states with flexible probability distributions.
problem Predicting future states under complex, non-deterministic scenarios.
method Hypernetwork architecture and continuous normalizing flow model.
result RegFlow achieves state-of-the-art results on benchmark datasets.
Causal Imitation Learning handles noisy measurements and distribution shifts.
problem Learning from noisy state observations and distributional shifts.
method Causal inference framework and adversarial RKHS learning.
result Improved robustness to distribution shifts compared to standard methods.
Paper presents a fast method for estimating hidden states in Bayesian models.
problem Estimating hidden states in Bayesian state space models efficiently.
method Amortized simulation-based inference with pretraining.
result The method achieves sufficient accuracy and fast inference times.
Simulations of financial markets in a Potts-like modelcond-mat.other
A three-state model based on the Potts model is proposed to simulate financial markets. The three states are assigned to "buy", "sell" and "inactive" states. The model shows the main stylized facts observed in the financial market: fat-tailed distributions of returns and long time correlations in the absolute returns. …
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' …
IDAC improves reinforcement learning efficiency by modeling implicit distributions.
problem Improving sample efficiency in reinforcement learning algorithms.
method IDAC uses two DGNs for a distributional critic and a semi-implicit actor to model implicit policy distributions.
result IDAC outperforms state-of-the-art algorithms on OpenAI Gym environments.
Studying general quantum many-body systems is one of the major challenges in modern physics because it requires an amount of computational resources that scales exponentially with the size of the system.Simulating the evolution of a state, or even storing its description, rapidly becomes intractable for exact classical…
A critical and challenging problem in reinforcement learning is how to learn the state-action value function from the experience replay buffer and simultaneously keep sample efficiency and faster convergence to a high quality solution. In prior works, transitions are uniformly sampled at random from the replay buffer o…
Distributed strategic learning has been getting attention in recent years. As systems become distributed finding Nash equilibria in a distributed fashion is becoming more important for various applications. In this paper, we develop a distributed strategic learning framework for seeking Nash equilibria under stochastic…
State aggregation is a popular model reduction method rooted in optimal control. It reduces the complexity of engineering systems by mapping the system's states into a small number of meta-states. The choice of aggregation map often depends on the data analysts' knowledge and is largely ad hoc. In this paper, we propos…
New RL approach uses future state and action visitation measures for better exploration.
problem Improving exploration in reinforcement learning.
method Intrinsic reward based on future state and action visitation measures, using contraction operators.
result Policies achieve good state-action space coverage and high performance.
We consider a network scenario in which agents can evaluate each other according to a score graph that models some interactions. The goal is to design a distributed protocol, run by the agents, that allows them to learn their unknown state among a finite set of possible values. We propose a Bayesian framework in which …
Efficiently estimates online variational learning using importance sampling.
problem Online variational estimation in state-space models.
method Variational approach with Monte Carlo importance sampling.
result Proposed efficient algorithm for streaming data.
Exploration is critical to a reinforcement learning agent's performance in its given environment. Prior exploration methods are often based on using heuristic auxiliary predictions to guide policy behavior, lacking a mathematically-grounded objective with clear properties. In contrast, we recast exploration as a proble…
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
problem Improving exploration in reinforcement learning agents.
method Intrinsic rewards proportional to the entropy of future state-action features.
result The new objective leads to improved visitation of features within individual trajectories.
Schrödinger bridge solved with Weyl calculus for quadratic state cost.
problem Optimal control policy to steer joint state statistics.
method Weyl calculus in quantum mechanics for reaction-diffusion PDEs.
result Explicit Markov kernel for quadratic state cost found.
Machine learning promises methods that generalize well from finite labeled data. However, the brittleness of existing neural net approaches is revealed by notable failures, such as the existence of adversarial examples that are misclassified despite being nearly identical to a training example, or the inability of recu…