A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper explains why estimating a history-dependent policy can reduce MSE in reinforcement learning.
problem Understanding why history-dependent policies can improve MSE in off-policy evaluation.
method The paper derives a bias-variance decomposition of MSE for various OPE estimators, showing how history-dependent policies can decrease variance and increase bias.
result History-dependent policies can decrease the variance of importance sampling estimators, leading to lower MSE.
When governed by underlying low-dimensional dynamics, the interdependence of simultaneously recorded population of neurons can be explained by a small number of shared factors, or a low-dimensional trajectory. Recovering these latent trajectories, particularly from single-trial population recordings, may help us unders…
Paper proposes a policy gradient method for confounded POMDPs.
problem Estimating policy gradients for confounded POMDPs with continuous state and observation spaces.
method Developed a novel identification result to estimate policy gradients using offline data, solved conditional moment restrictions, and applied min-max learning with function approximation.
result Showed global convergence of the proposed algorithm in finding the optimal policy.
Study designs incentives for adapting multi-agent systems without knowing their learning dynamics.
problem Designing incentives for an adapting population in multi-agent systems without prior knowledge of their learning dynamics.
method Introduces a model-based non-episodic Reinforcement Learning (RL) formulation for steering Markovian agents towards desired policies, focusing on history-dependent strategies to handle model uncertainty.
result Identifies conditions for the existence of steering strategies to guide agents to desired policies and provides empirical algorithms to approximately solve the objective.
Market impact is a key concept in the study of financial markets and several models have been proposed in the literature so far. The Transient Impact Model (TIM) posits that the price at high frequency time scales is a linear combination of the signs of the past executed market orders, weighted by a so-called propagato…
The roundworm C. elegans exhibits robust escape behavior in response to rapidly rising temperature. The behavior lasts for a few seconds, shows history dependence, involves both sensory and motor systems, and is too complicated to model mechanistically using currently available knowledge. Instead we model the process p…
In many settings (e.g., robotics) demonstrations provide a natural way to specify tasks; however, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the tasks, such as rewards or policies, can be safely composed and/or do not explicitly capture history dependen…
Flexible Hawkes model with Gaussian process self-effects for time-dependent data.
problem Modeling time-dependent point processes with history dependence and self-effects.
method Extended Hawkes process with Gaussian process self-effects for both excitatory and inhibitory types, using Bayesian inference and mean-field variational approximation.
result Efficient approximate Bayesian inference achieved via data augmentation and mean-field variational approach.
Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets…
Neural responses are highly variable, and some portion of this variability arises from fluctuations in modulatory factors that alter their gain, such as adaptation, attention, arousal, expected or actual reward, emotion, and local metabolic resource availability. Regardless of their origin, fluctuations in these signal…
We propose a general framework to describe the impact of different events in the order book, that generalizes previous work on the impact of market orders. Two different modeling routes can be considered, which are equivalent when only market orders are taken into account. One model posits that each event type has a te…
Strategy evaluation schemes are a crucial factor in any agent-based market model, as they determine the agents' strategy preferences and consequently their behavioral pattern. This study investigates how the strategy evaluation schemes adopted by agents affect their performance in conjunction with the market circumstan…
While the long-ranged correlation of market orders and their impact on prices has been relatively well studied in the literature, the corresponding studies of limit orders and cancellations are scarce. We provide here an empirical study of the cross-correlation between all these different events, and their respective i…
We present a noise-injected version of the Expectation-Maximization (EM) algorithm: the Noisy Expectation Maximization (NEM) algorithm. The NEM algorithm uses noise to speed up the convergence of the EM algorithm. The NEM theorem shows that injected noise speeds up the average convergence of the EM algorithm to a local…
Paper tackles online control of linear systems with unbounded noise.
problem Online control of linear systems under unbounded noise with unknown convex cost functions.
method Developed an algorithm achieving ildeO(T) high-probability regret under unbounded noise, and established O(mpoly(logT)) regret bound for strongly convex costs and sub-Gaussian noise.
result Achieved ildeO(T) high-probability regret under unbounded noise, and O(mpoly(logT)) regret bound for specific noise and cost conditions.