Study optimizes reinforcement learning options under time constraints.
problem Learning useful options for diverse tasks with limited time.
method Directly searched for optimal option sets considering time budget.
result Discovered options outperform existing heuristics.
PODNet discovers plannable options from unstructured demonstrations.
problem Learning from unstructured, multi-objective demonstrations.
method Custom categorical variational autoencoder, recurrent option inference network, option-conditioned policy network, and option dynamics model.
result PODNet enables learning from demonstration for multiple tasks and planning.
New algorithms improve learning of long-term actions in reinforcement learning.
problem Violation of parameter independence assumption in deep function approximation.
method Reconsidered option-critic and hierarchical option-critic training for deep settings.
result Significantly improved stability and faster convergence in Atari games.
In reinforcement learning (RL), temporal abstraction still remains as an important and unsolved problem. The options framework provided clues to temporal abstraction in the RL, and the option-critic architecture elegantly solved the two problems of finding options and learning RL agents in an end-to-end manner. However…
ANNs solve financial option valuation problems without numerical methods.
problem Valuation of European and American financial options.
method Unsupervised learning with artificial neural networks (ANNs) for solving PDEs.
result ANNs accurately compute option values for various stock scenarios.
Options are generally learned by using an inaccurate environment model (or simulator), which contains uncertain model parameters. While there are several methods to learn options that are robust against the uncertainty of model parameters, these methods only consider either the worst case or the average (ordinary) case…
Building systems that autonomously create temporal abstractions from data is a key challenge in scaling learning and planning in reinforcement learning. One popular approach for addressing this challenge is the options framework (Sutton et al., 1999). However, only recently in (Bacon et al., 2017) was a policy gradient…
Paper uses RL to optimize bid-ask spreads for diverse options.
problem Optimizing bid-ask spreads for options with various maturities and strikes.
method Combines stochastic policy with reinforcement learning.
result Proposes an effective approach for market making of options.
Deep learning solves barrier options with stochastic volatility.
problem Solving barrier options with stochastic volatility.
method Unsupervised deep learning neural networks trained to satisfy PDE and boundary conditions.
result Neural networks accurately price barrier options in a single framework.
HO2 learns options from data efficiently, improving robot manipulation tasks.
problem Learning options from raw pixel inputs in 3D robot manipulation tasks.
method HO2 infers likely option choices and trains all policy components off-policy.
result HO2 outperforms existing methods on 3D robot manipulation tasks.
Machine learning models outperform traditional option pricing models.
problem Improving option pricing accuracy using complex models.
method Evaluation of machine learning (NN, RF, CatBoost) and traditional models (Black-Scholes, Heston) on synthetic and real data.
result Machine learning models outperform traditional models in predicting option prices.
Neural network learns to solve Black-Scholes for stock options.
problem Stock option pricing using the Black-Scholes Equation.
method Neural Networks applied to solve the Black-Scholes Equation.
result Neural network can accurately forecast stock option prices.
We present Multitask Soft Option Learning(MSOL), a hierarchical multitask framework based on Planning as Inference. MSOL extends the concept of options, using separate variational posteriors for each task, regularized by a shared prior. This ''soft'' version of options avoids several instabilities during training in a …
This paper uses deep learning to price American options under stochastic volatility.
problem Pricing American options with a time-varying exercise boundary under the Heston model.
method Coupled PINNs with curriculum learning and adaptive resampling.
result Demonstrates the effectiveness of the proposed deep learning framework for American option pricing.
Method learns reusable actions from human demonstrations.
problem Learning reusable actions from human demonstrations.
method Skill segmentation, inverse reinforcement learning, one-class support vector machine.
result Inferred options improve learning and planning.
This paper compares machine learning models for pricing European options.
problem Pricing European options using traditional methods like Black Scholes Model.
method Google AutoML Regressor, TensorFlow Neural Networks, and XGBoost Gradient Boosting Decision Trees.
result All models outperformed the Black Scholes Model in terms of mean absolute error.
Tensor trains speed up option pricing for multi-asset options.
problem Speeding up option pricing for multi-asset options.
method Tensor train learning algorithms to compress functions with parameter dependence.
result The proposed method outperforms Monte Carlo-based pricing in computational complexity.
Deep learning models predict option prices from 3D tensor data.
problem Predicting option prices for risk management and trading.
method 3D tensor representation of financial data, deep learning models (2D tensors in 3 channels).
result Proposed models outperform traditional methods like B-S model and vector-based LSTM.
We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yieldi…
Deep learning models price options using volatility surfaces.
problem Pricing exotic options with high accuracy and efficiency.
method Variational autoencoder for volatility surface compression, multilayer perceptron for option pricing.
result Trained model achieves high accuracy across American and Asian options.
While a large body of empirical results show that temporally-extended actions and options may significantly affect the learning performance of an agent, the theoretical understanding of how and when options can be beneficial in online reinforcement learning is relatively limited. In this paper, we derive an upper and l…
Machine learning struggles to predict binary options movements due to randomness.
problem Predicting binary options movements using machine learning.
method Tested multiple machine learning models (RF, LR, GB, kNN) and neural networks (MLP, LSTM) on EUR/USD currency pairs.
result None of the models surpassed the ZeroR baseline accuracy, indicating randomness in binary options.
Paper uses deep learning to price and hedge options in incomplete markets.
problem Incomplete markets lack unique no-arbitrage solutions for pricing and hedging European options.
method Constrained deep learning approach with a single neural network representing option prices and hedging strategies.
result Constrained networks produce superior P&L distributions compared to unconstrained networks.
QLBS and RLOP methods improve option pricing and hedging performance.
problem Improving option pricing and hedging performance under market frictions.
method Incorporates risk aversion and trading costs into QLBS, proposes RLOP approach.
result RLOP outperforms in dynamic hedging by reducing shortfall probability.
A new deep learning method for option pricing in rough volatility models.
problem Efficient pricing of European options in high-dimensional rough volatility models.
method Time-stepping deep gradient flow method reformulating the option pricing PDE as an energy minimization problem.
result The method respects asymptotic behavior and known bounds for option prices.
Deep learning method for pricing and hedging American-style options.
problem Pricing and hedging American-style options with high accuracy.
method Computes optimal stopping policy, derives bounds, calculates point estimate and confidence intervals, constructs hedging strategy.
result Highly accurate prices and dynamic hedging strategies with small replication errors.
Deep learning improves options trading without market assumptions.
problem Traditional options trading requires market dynamics and pricing models.
method End-to-end deep learning approach that learns from market data.
result Deep learning models outperform existing trading strategies.
Paper solves multi-dimensional passport option pricing problem using machine learning.
problem Pricing multi-dimensional passport options in correlated markets remains unsolved.
method Discrete-time solution for multi-dimensional BS markets with uncorrelated assets; machine learning approaches.
result Machine learning-powered approaches successfully price passport options in both 1D and multi-dimensional uncorrelated BS markets.
The paper compares machine learning methods with traditional techniques for pricing and sensitivities of financial products with path-dependent structures.
problem Evaluating financial products with early-termination clauses, especially those with path-dependent structures.
method The paper compares regression methods including randomized recurrent and feed-forward neural networks, and a novel approach using signatures of the underlying price process, with traditional polynomial basis functions for pricing and sensitivities.
result Machine learning algorithms often match the accuracy and efficiency of traditional methods for Asian and look-back options, while randomized neural networks are best for callable certificates.
The paper learns robot skills from demonstrations without supervision.
problem Discovering robotic options from unlabelled demonstrations.
method Temporal variational inference for latent variable learning.
result The framework can learn options across multiple datasets.
Reinforcement learning improves option pricing and hedging accuracy.
problem Improving financial instrument pricing and hedging accuracy.
method Q-Learning Black Scholes approach applied to option pricing and hedging.
result The reinforcement learning model accurately estimates option prices and hedging strategies under various volatility and moneyness levels.
Physics-Informed Neural Network improves option pricing accuracy.
problem Improving option pricing accuracy using machine learning.
method Physics-Informed Neural Network (PINN) applied to Black-Scholes equation.
result PINN model accurately captures option pricing behavior on both simulated and real market data.
FINN learns option pricing and hedging using financial theory.
problem Learning accurate option prices and sensitivities from financial theory.
method Self-supervised replication objective based on dynamic hedging.
result FINN accurately recovers classical Black--Scholes prices and performs robustly in stochastic volatility environments.
A new method for pricing European options in changing market conditions.
problem Lack of closed-form solutions for pricing European options in regime-switching models.
method Physics-informed residual learning (PIRL) for efficient option pricing.
result PIRL eliminates the need for retraining and offers near-instantaneous pricing.
Novel online algorithm for hierarchical imitation learning.
problem Scalability issue in reinforcement learning and options discovery.
method Online Baum-Welch algorithm for hierarchical imitation learning.
result The online algorithm outperforms the batch version in both discrete and continuous environments.
This paper solves inverse reinforcement learning with options framework.
problem Reinforcement learning in complex environments with expert demonstrations.
method Gradient method for parametrized options, deducing Q-feature and reward feature spaces, optimal reward function selection.
result Recovered rewards provide effective solution to IRL problem and accelerate transfer learning.
Framework uses expert intervention to solve long-horizon reinforcement learning tasks.
problem Long horizon robot learning tasks with sparse rewards.
method Option templates and expert intervention to enable high-level task understanding.
result Framework outperforms state-of-the-art approaches by two orders of magnitude.
Deep learning models predict S&P500 option hedge ratios.
problem Optimizing hedging strategies for S&P500 index options.
method Feedforward neural network with time to maturity, delta, and sentiment variables.
result Deep learning model outperforms traditional hedging methods.
Robust reinforcement learning aims to produce policies that have strong guarantees even in the face of environments/transition models whose parameters have strong uncertainty. Existing work uses value-based methods and the usual primitive action setting. In this paper, we propose robust methods for learning temporally …
An unsupervised deep learning method solves PIDEs for option pricing.
problem Solving partial integro-differential equations for financial option pricing.
method Employing unsupervised deep learning to directly solve PIDEs without requiring labeled data.
result An unsupervised neural network accurately solves PIDEs and calculates derivatives and integrals.
The options framework in reinforcement learning models the notion of a skill or a temporally extended sequence of actions. The discovery of a reusable set of skills has typically entailed building options, that navigate to bottleneck states. This work adopts a complementary approach, where we attempt to discover option…
In this work, we consider the problem of autonomously discovering behavioral abstractions, or options, for reinforcement learning agents. We propose an algorithm that focuses on the termination condition, as opposed to -- as is common -- the policy. The termination condition is usually trained to optimize a control obj…
In this paper we combine one method for hierarchical reinforcement learning - the options framework - with deep Q-networks (DQNs) through the use of different "option heads" on the policy network, and a supervisory network for choosing between the different options. We utilise our setup to investigate the effects of ar…
A hybrid framework uses machine learning to price options faster and more accurately.
problem Rapid recalibration of option pricing models in dynamic markets.
method Integrates smooth offset algorithm with supervised machine learning models.
result Surrogate pricing operators achieve up to 1000x speedup over direct SOA evaluation.
It is well known that options can make planning more efficient, among their many benefits. Thus far, algorithms for autonomously discovering a set of useful options were heuristic. Naturally, a principled way of finding a set of useful options may be more promising and insightful. In this paper we suggest a mathematica…
As is known, an option price is a solution to a certain partial differential equation (PDE) with terminal conditions (payoff functions). There is a close association between the solution of PDE and the solution of a backward stochastic differential equation (BSDE). We can either solve the PDE to obtain option prices or…
MO2 learns useful behaviours from past experience for new tasks.
problem Discovering useful behaviours from past experience and transferring them to new tasks.
method Model-Based Offline Options (MO2) framework supporting sample-efficient bottleneck option discovery over continuous state-action spaces.
result MO2 outperforms recent option learning methods on complex long-horizon continuous control tasks.
New approach minimizes tail risk in option hedging.
problem Minimizing tail risk in option hedging strategies.
method Risk-sensitive reinforcement learning without parametric models.
result Significantly lower tail risk and higher mean P&L than delta hedging.