arXiv research

A new method for matrix completion with model-free weights.

Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimizat…

A new model forecasts optimal portfolio weights from high-frequency data.

Geometric Mean Market Makers super-hedge impermanent loss without models.

New algorithms cluster nodes in SBM graphs faster and more accurately.

Study compares model-free valuation to actual financial outcomes, finds it slightly conservative.

By specifying model free preferences towards simple nested classes of lottery pairs, we develop the dual story to stand on equal footing with that of (primal) risk apportionment. The dual story provides an intuitive interpretation, and full characterization, of dual counterparts of such concepts as prudence and tempera…

A novel Bayesian computation method using importance weighting improves numerical stability and performance.

We consider the problem of detecting abrupt changes in the distribution of a multi-dimensional time series, with limited computing power and memory. In this paper, we propose a new, simple method for model-free online change-point detection that relies only on fast and light recursive statistics, inspired by the classi…

New model-free DR-RL algorithm with finite sample complexity.

The paper tackles robust reinforcement learning with performance guarantees.

VO$Q$L optimizes RL with sparse rewards using weighted bounds.

We prove the existence and uniqueness of solutions of SDEs with Lipschitz coefficients, driven by continuous, model-free martingales. The main tool in our reasoning is Picard's iterative procedure and a model-free version of the Burkholder-Davis-Gundy inequality for integrals driven by model-free, continuous martingale…

New method improves deep RL by combining emphatic weightings with replay data.

Unified reinforcement learning methods using hybrid inference.

A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.

New model-free algorithm achieves similar LQR regret guarantees.

FORK improves model-free reinforcement learning performance.

Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…

Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method base…

New hybrid RL algorithm outperforms model-free and model-based methods.

Simple object representations improve model-free RL performance.

Develops model-free methods for event history analysis and efficient covariate adjustment.

New algorithms for model selection in off-policy evaluation of reinforcement learning.

New algorithm borrows future randomness to stabilize model-free control.

A common difficulty in applications of machine learning is the lack of any general principle for guiding the choices of key parameters of the underlying neural network. Focusing on a class of recurrent neural networks - reservoir computing systems that have recently been exploited for model-free prediction of nonlinear…

Improved model-free RL algorithm with reduced sample complexity.

We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonu…

Improves sample efficiency in RL by matching model-based gradients.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

We develop robust pricing and hedging of a weighted variance swap when market prices for a finite number of co--maturing put options are given. We assume the given prices do not admit arbitrage and deduce no-arbitrage bounds on the weighted variance swap along with super- and sub- replicating strategies which enforce t…

Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15]. Recently, equipped with the idea of optimism in the face of uncertainty, Q-learning algorithms [Jin, Al…

Optimistic algorithm reduces regret in non-stationary linear MDPs.

The combined algorithm selection and hyperparameter tuning (CASH) problem is characterized by large hierarchical hyperparameter spaces. Model-free hyperparameter tuning methods can explore such large spaces efficiently since they are highly parallelizable across multiple machines. When no prior knowledge or meta-data e…

Optimizes control of noisy discrete systems without system matrix knowledge.

A new model-free subsampling method using uniform designs is proposed.

The paper presents a model-free method for stabilizing unknown control systems.

New model-free algorithms learn representations for low-rank MDPs efficiently.

Simple model-based reinforcement learning outperforms model-free methods in complex tasks.

A new model-free variable importance method (IGCS) is introduced for high-dimensional data.

This paper compares model-based and model-free control methods using neural networks.

This paper addresses the model-free nonlinear optimal problem with generalized cost functional, and a data-based reinforcement learning technique is developed. It is known that the nonlinear optimal control problem relies on the solution of the Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear partial differ…

Simple image augmentation boosts deep RL from pixels.

A new feature selection method using random forest and Kolmogorov filter.

Integrating model-free and model-based approaches in reinforcement learning has the potential to achieve the high performance of model-free algorithms with low sample complexity. However, this is difficult because an imperfect dynamics model can degrade the performance of the learning algorithm, and in sufficiently com…

Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of c…

We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy. We design new algorithms for RL with a generic model class and analyze their statistical properties. Our algorithms have sample …

Deep RL optimizes US stock allocations with better performance.