Classifies state functions for framed tangles in a disk.
problem Understanding quantum invariants of framed tangles.
method Introduces state function for framed tangles, local and topologically invariant.
result Classified all state functions for the Kauffman bracket and quantum S U ( 3 ) SU(3) S U ( 3 ) -invariant. Method estimates observation functions in state-space models without supervision.
problem Unsupervised learning of non-invertible observation functions in nonlinear state-space models.
method Nonparametric generalized moment method using constrained regression.
result Estimates function space of identifiability from state process.
New ODE-Block handles stateful layers with continuous-in-depth functions using basis functions.
problem Handling stateful layers in ODE-Nets.
method Formulate ODE-Block using continuous-in-depth functions with basis function expansions.
result Enables state-of-the-art performance and reduces memory footprint.
New method improves value function estimation in noisy environments.
problem High variance in value-based reinforcement learning methods.
method Introduce Recurrent Value Functions (RVFs) to estimate value function of current state using past states.
result RVFs show robustness and improved performance in noisy environments.
A new RL paradigm reduces state-action-value function approximation inefficiency.
problem Challenges in state-action-value function approximation for RL.
method State Action Separable Reinforcement Learning (sasRL) decouples action space from value function learning.
result sasRL achieves up to 75% better performance than state-of-the-art MDP-based RL algorithms.
We protect value function approximators in continuous state spaces from differential privacy attacks.
problem Protecting reward information from inverse reinforcement learning attacks in continuous state spaces.
method Adding functional noise to the value function iteratively to ensure differential privacy.
result Rigorous privacy guarantees and approximate optimality of the algorithm in discrete state spaces.
Efficiently plans large MDPs with weak function approximations.
problem Planning in large MDPs with limited function approximation capabilities.
method Uses linear value function approximation with weak requirements and a generative oracle.
result Produces almost-optimal actions for any state with polynomial computation time.
New RL method learns value function for many policies using few key states.
problem Evaluate and improve policies in continuous control problems.
method Combines actor-critic architecture and policy embedding to learn a single value function for many policies.
result Value function minimizes prediction error by learning a small set of 'probing states' and their impact on policies' returns.
In this paper, we consider the generalized lambda constant and the existence of ground states of the generalized Perelman's W-functional from a variational formulation. One result is concerned with the estimation of the generalized λ λ λ constant. The other results are about the existence of ground states of generalized …
Paper introduces a new value function for state transitions and optimal policy learning.
problem Learning optimal policies from state transitions and actions.
method Develops a forward dynamics model to maximize a novel value function Q ( s , s ′ ) Q(s, s') Q ( s , s ′ ) . result Demonstrates benefits in value function transfer, redundant action spaces, and off-policy learning.
A new RNN architecture GVFN improves RNN trainability by constraining states to future predictions.
problem Improving RNN trainability in complex environments.
method Developed a novel RNN architecture called General Value Function Network (GVFN) that constrains states to future predictions.
result GVFNs are more robust to truncation levels, requiring only one-step gradient updates.
New RL method handles large state-action spaces with complex models.
problem Complex models and large state-action spaces in reinforcement learning.
method π-KRVI, an optimistic modification of least-squares value iteration using kernel ridge regression.
result First order-optimal regret guarantees under general settings, improving over state of the art.
We consider a nonlinear state-space model with the state transition and observation functions expressed as basis function expansions. The coefficients in the basis function expansions are learned from data. Using a connection to Gaussian processes we also develop priors on the coefficients, for tuning the model flexibi…
Algorithm finds latent structure in value functions for improved reinforcement learning.
problem Finding latent structure in value functions for efficient reinforcement learning.
method Proposes a practical algorithm using two posterior distributions over state abstractions and abstract-state values.
result Substantial performance gains in multi-task settings where tasks share a common, low-dimensional representation.
Introduces a neural network-based method for efficient state and parameter estimation in complex systems.
problem Efficiently estimating state paths and parameters from noisy measurements in high-dimensional nonlinear systems.
method Bayesian Information Field Theory with neural network parameterization and optimization algorithms.
result Proposes a method to simplify and enrich state path parameterizations using neural networks, improving inference accuracy.
DR-FRL learns functional states from irregular histories for causal inference.
problem Causal inference with irregularly sampled longitudinal data.
method DR-FRL workflow combining functional and temporal encoders, nuisance heads, and EIF-targeted validation.
result DR-FRL can improve causal inference when pseudo-outcomes are heavy-tailed or measurement is informative.
The paper characterizes brain states and transitions using functional MRI data.
problem Characterizing the dynamic reconfiguration of neural systems in brain states.
method Bayesian model-based characterization of latent brain states and posterior predictive discrepancy using the latent block model.
result The model detects transitions between latent brain states and identifies distinctive community patterns in task-fMRI data.
This paper explores a new DRL algorithm that approximates both state-value and state-action functions.
problem Overestimation bias in state-action value function.
method Developed and analyzed the Deep Quality-Value (DQV) algorithm to approximate both V V V and Q Q Q functions. result DQV and DQV-Max algorithms perform better due to less overestimation bias in Q Q Q function. This article introduces both a new algorithm for reconstructing epsilon-machines from data, as well as the decisional states. These are defined as the internal states of a system that lead to the same decision, based on a user-provided utility or pay-off function. The utility function encodes some a priori knowledge ex…
Tensor networks and RNNs are equivalent, improving wave function encoding.
problem Efficiently encoding quantum states in neural networks.
method Generalized RNN architecture for tensor networks, supporting polynomial time wave function evaluation.
result Tensorial RNNs can encode quantum states with lower bond dimensions and higher accuracy.
Agents learn state ambiguity from non-linear sensor data using Gaussian approximations.
problem Learning state representation from non-linear sensor data.
method Second-order Taylor approximation of Gaussian distribution for non-linear measurement functions.
result Induces a preference for states based on inferability from observations.
Echo state networks with random weights can approximate any continuous system.
problem Approximating continuous dynamical systems using echo state networks.
method Randomly generated internal weights and a sampling procedure for activation functions.
result Echo state networks with random weights can approximate any continuous casual time-invariant operators with high probability.
State2vec improves RL by learning state embeddings that generalize across policies.
problem Inefficient generalization across policies in RL.
method Extends node2vec to learn state embeddings accounting for discounted future state transitions.
result Captures the geometry of the state space, leading to sample-efficient value function approximation.
New IRL algorithm for continuous state spaces with formal guarantees.
problem Finding a reward function for expert behavior in continuous state spaces.
method Modeling the system using orthonormal functions and providing correctness proofs.
result Proof of correctness and formal guarantees on sample and time complexity.
A general Boltzmann machine with continuous visible and discrete integer valued hidden states is introduced. Under mild assumptions about the connection matrices, the probability density function of the visible units can be solved for analytically, yielding a novel parametric density function involving a ratio of Riema…
Simple algorithm controls unknown systems with optimal regret.
problem Online reinforcement learning for unknown systems with arbitrary state and action spaces.
method Upper-confidence reinforcement learning algorithm using optimistic Q functions.
result Regret bound of $O(HL(KH)^{ rac{d-1}{d}})$ for finite horizon control systems.
Neural-Network Quantum States have been recently introduced as an Ansatz for describing the wave function of quantum many-body systems. We show that there are strong connections between Neural-Network Quantum States in the form of Restricted Boltzmann Machines and some classes of Tensor-Network states in arbitrary dime…
Efficient RL in large POMDPs with latent determinism and embeddings.
problem Efficient reinforcement learning in large-scale POMDPs with latent states and observations.
method Conditional Hilbert space embeddings, linear optimal Q Q Q -function, deterministic latent transitions, gap assumption. result Computationally and statistically efficient algorithm for exact optimal policy.
The study explores how Matrix Product States can represent boolean and continuous functions.
problem Representing arbitrary boolean and continuous functions using Matrix Product States.
method Developed a construction method for MPS to represent boolean gates and proved density in continuous function space.
result MPS can accurately represent arbitrary boolean functions and continuous functions densely.
PBVFs generalize across policies using learned value functions.
problem RL algorithms forget information about old policies when updating value functions to track the learned policy.
method Introduce Parameter-Based Value Functions (PBVFs) that include policy parameters in their inputs, enabling them to generalize across different policies.
result PBVFs enable zero-shot learning of new policies that outperform any policy seen during training.
BrainSurfCNN predicts task contrasts from resting-state fingerprints, improving accuracy over baseline.
problem Predicting task-evoked activity from resting-state functional connectivity.
method Surface-based convolutional neural network (BrainSurfCNN) with reconstructive-contrastive loss.
result Significantly improved accuracy in predicting task contrasts over baseline.
We consider nonparametric estimation of the state price density encapsulated in option prices. Unlike usual density estimation problems, we only observe option prices and their corresponding strike prices rather than samples from the state price density. We propose to model the state price density directly with a nonpa…
We present a comprehensive study of utility function of the minority game in its efficient regime. We develop an effective description of state of the game. For the payoff function $g(x)=\sgn (x)$ we explicitly represent the game as the Markov process and prove the finitness of number of states. We also demonstrate bou…
Recurrent networks learn beliefs from history in partially observable environments.
problem Learning optimal policies in partially observable environments.
method Trained recurrent neural networks to approximate value functions, measuring mutual information between hidden states and beliefs.
result Recurrent networks' hidden states correlate with beliefs of relevant state variables, improving expected return.
New RL policy for unbounded state space with stability guarantee.
problem Traditional RL methods fail for unbounded state space.
method Proposes stability as performance metric, uses Sparse-Sampling-based Monte Carlo Oracle.
result Proposed policy ensures state dynamics remain bounded with high probability.
We consider the estimation of the policy gradient in partially observable Markov decision processes (POMDP) with a special class of structured policies that are finite-state controllers. We show that the gradient estimation can be done in the Actor-Critic framework, by making the critic compute a "value" function that …
Paper uses deep learning to compute committor functions for rare events in complex systems.
problem Computing committor functions for low-temperature, high-dimensional systems is challenging.
method Combines deep learning, data sampling, and feature engineering.
result Achieves good performance on complex benchmark problems with rough energy landscapes.
A new method for state estimation in state-space models using incomplete data.
problem State estimation in nonlinear state-space models with incomplete observations.
method Statistical analysis of incomplete observations, score function, observed information matrices, EM-gradient-particle filtering.
result Maximum likelihood estimation of state-vector with explicit form of observed information matrix.
Global EQG sums boundary states over manifold diffeomorphism classes.
problem Summing boundary states over manifold diffeomorphism classes.
method Formulated as classical statistical physics, weights determined by general principles.
result Hartle-Hawking state as a probability measure.
A new IRL model recovers reward and state structure from expert demonstrations.
problem Limitation of classical maximum entropy model in capturing state structure.
method Generalized maximum causal entropy for IRL models.
result Empirically outperforms classical models in recovering reward and state structure.
Method explains MDP optimal policies and reward collection.
problem Understanding optimal policies and reward collection in MDPs.
method Deterministic MDPs with a method to map state space and explain rewards.
result Algorithm determines which rewards will be collected and their frequency.
This work tackles model-based RL by optimizing state-action queries to learn policies with minimal data.
problem Expensive state transitions in practical RL problems limit the use of standard RL algorithms.
method Bayesian optimal experimental design to guide selection of state-action queries.
result Data-efficient RL approach that learns optimal policies with up to 1,000x less data.
New RL algorithm tackles large state spaces using optimistic function approximation.
problem Challenges in reinforcement learning with large state spaces and modern function approximators.
method Optimistic modification of least-squares value iteration algorithm for kernel functions or overparameterized neural networks.
result Polynomial runtime and sample complexity, independent of the number of states, with i l d e O ( δ F H 2 T ) ilde{\mathcal{O}}(δ_{\mathcal{F}} H^2 \sqrt{T}) i l d e O ( δ F H 2 T ) regret. The paper provides high-confidence error estimates for learned value functions in large state-spaces.
problem Estimating the accuracy of learned value functions in large, continuous state-spaces.
method Developed a high-confidence bound on empirical value error to true value error, and an offline sampling algorithm to repeatedly compute value error estimates.
result Demonstrated that the offline sampling algorithm can provide high-confidence estimates of value error for learned value functions.
Paper learns meaningful state and action representations from MDP trajectories.
problem Learning good state and action representations from MDP trajectories.
method Tensor decomposition, kernelization, importance sampling, low-Tucker-rank approximation.
result The learned state/action abstractions provide accurate approximations to latent block structures.
Extends Hindsight Experience Replay to learn from multiple reward functions.
problem Learning policies for multiple reward functions without trial runs.
method Develops a method to use a reward function to calculate rewards for states not encountered, and learns policies for multiple reward functions.
result A single policy can generalize across all linear combinations of multi-objective rewards.
The generalized coherent states attached to the Jacobi group realize the squeezed states. Imposing hermitian conjugacy to the generators of the Jacobi algebra, we find out the form of the weight function appearing in the scalar product. We show effectively the orthonormality of the base functions with respect to the sc…
New method improves deep policy gradient algorithms by learning relative state values.
problem High sample complexity and instability in policy gradient methods.
method Uses a new state-value function approximation based on residual variance.
result Empirical improvement across diverse continuous control tasks and algorithms.