A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Study compares model-free valuation to actual financial outcomes, finds it slightly conservative.
New model-free DR-RL algorithm with finite sample complexity.
We prove the existence and uniqueness of solutions of SDEs with Lipschitz coefficients, driven by continuous, model-free martingales. The main tool in our reasoning is Picard's iterative procedure and a model-free version of the Burkholder-Davis-Gundy inequality for integrals driven by model-free, continuous martingale…
Unified reinforcement learning methods using hybrid inference.
A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.
New model-free algorithm achieves similar LQR regret guarantees.
FORK improves model-free reinforcement learning performance.
Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…
Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method base…
New hybrid RL algorithm outperforms model-free and model-based methods.
Develops model-free methods for event history analysis and efficient covariate adjustment.
New algorithms for model selection in off-policy evaluation of reinforcement learning.
New algorithm borrows future randomness to stabilize model-free control.
Improved model-free RL algorithm with reduced sample complexity.
We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonu…
Improves sample efficiency in RL by matching model-based gradients.
RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.
Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15]. Recently, equipped with the idea of optimism in the face of uncertainty, Q-learning algorithms [Jin, Al…
Optimizes control of noisy discrete systems without system matrix knowledge.
A new model-free subsampling method using uniform designs is proposed.
The paper presents a model-free method for stabilizing unknown control systems.
New model-free algorithms learn representations for low-rank MDPs efficiently.
Simple model-based reinforcement learning outperforms model-free methods in complex tasks.
A new model-free variable importance method (IGCS) is introduced for high-dimensional data.
This paper compares model-based and model-free control methods using neural networks.
Simple image augmentation boosts deep RL from pixels.
A new feature selection method using random forest and Kolmogorov filter.
Integrating model-free and model-based approaches in reinforcement learning has the potential to achieve the high performance of model-free algorithms with low sample complexity. However, this is difficult because an imperfect dynamics model can degrade the performance of the learning algorithm, and in sufficiently com…
Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of c…
We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy. We design new algorithms for RL with a generic model class and analyze their statistical properties. Our algorithms have sample …
The OGY method is one of control methods for a chaotic system. In the method, we have to calculate a stabilizing periodic orbit embedded in its chaotic attractor. Thus, we cannot use this method in the case where a precise mathematical model of the chaotic system cannot be identified. In this case, the delayed feedback…
Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimizat…
Unified Latent Dynamics unifies model-free and model-based reinforcement learning.
Improved model-free reinforcement learning with decision-estimation coefficient.
A new method selects a representative subsample for efficient kernel density estimation.
Develops a new model-free approach to portfolio theory using rough paths.
In many optimization problems in wireless communications, the expressions of objective function or constraints are hard or even impossible to derive, which makes the solutions difficult to find. In this paper, we propose a model-free learning framework to solve constrained optimization problems without the supervision …
Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…
A deep learning method for regression without model assumptions.
Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, mode…
In this paper we introduce a new approach to model-free path-dependent option pricing. We first introduce a general duality result for linear optimisation problems over signed measures introduced in [3] and show how the the problem of model-free option pricing can be formulated in the new framework. We then introduce a…
The paper develops a new model-free formula for option initial margins.
Framework uses probabilistic programming for physics simulation in games.
We provide the first solution for model-free reinforcement learning of ω-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of ω-regular objectives to an almost- sure reachability problem and extend this technique to learning how to control an …
The effectiveness of model-based versus model-free methods is a long-standing question in reinforcement learning (RL). Motivated by recent empirical success of RL on continuous control tasks, we study the sample complexity of popular model-based and model-free algorithms on the Linear Quadratic Regulator (LQR). We show…
Robotic table tennis learns efficient policies to return balls at 100Hz.
Model-free reinforcement learning (RL) algorithms, such as Q-learning, directly parameterize and update value functions or policies without explicitly modeling the environment. They are typically simpler, more flexible to use, and thus more prevalent in modern deep RL than model-based approaches. However, empirical wor…
DREAM learns optimal strategies in imperfect games without needing a simulator.