Richard Bellman's Principle of Optimality, formulated in 1957, is the heart of dynamic programming, the mathematical discipline which studies the optimal solution of multi-period decision problems. In this paper, we look at the main trading principles of Jesse Livermore, the legendary stock operator whose method was pu…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Bellman optimality”
Discrete time multi-period mean-variance model: Bellman type strategy and Empirical analysisq-fin.MF
Paper introduces dynamic strategies for multi-period investment models.
problem Optimizing investment strategies over multiple periods with risk and return considerations.
method Developed a Bellman principle for discrete time multi-period mean-variance models, leading to dynamic optimal strategies and efficient frontiers.
result Dynamic optimal strategies can achieve higher returns with lower risk compared to the 1/n strategy.
Linear Bellman Completeness Suffices for Efficient Online Reinforcement Learning with Few Actionscs.LG
Polynomial-time RL algorithm for constant actions under linear Bellman completeness.
problem Efficient online reinforcement learning with few actions.
method Polynomial-time algorithm based on linear function approximation.
result First computationally efficient algorithm for RL with constant actions under linear Bellman completeness.
We propose a new objective function for finite-horizon episodic Markov decision processes that better captures Bellman's principle of optimality, and provide an expression for the gradient of the objective.
We study utility maximization for power utility random fields with and without intermediate consumption in a general semimartingale model with closed portfolio constraints. We show that any optimal strategy leads to a solution of the corresponding Bellman equation. The optimal strategies are described pointwise in term…
Study solves optimal portfolio selection using HJB equation.
problem Optimal portfolio selection problem.
method Maximal monotone operator method, Banach fixed-point theorem, Fourier transform, monotone operators technique.
result Existence and uniqueness of solution to HJB equation.
Optimizes portfolios with costs, showing existence of optimal strategies.
problem Risk-sensitive portfolio optimization with transaction costs.
method Log-return i.i.d. framework, Bellman equation analysis.
result Existence of optimal strategies for risk-averse and risk-seeking cases.
The impact of softmax on the value function itself in reinforcement learning (RL) is often viewed as problematic because it leads to sub-optimal value (or Q) functions and interferes with the contraction properties of the Bellman operator. Surprisingly, despite these concerns, and independent of its effect on explorati…
New method solves continuous time mean-variance model for consistent investment strategy.
problem Time-consistent optimal strategy for continuous time mean-variance model.
method Developed a new Bellman principle method.
result Obtained a time-consistent dynamic optimal strategy.
New RL algorithm minimizes distributional learning error.
problem Improving distributional reinforcement learning for better error minimization.
method Proposes a new model-based algorithm with theoretical minimax optimality.
result Proves minimax optimality for approximating return distributions.
This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …
Deep Bellman Hedgingq-fin.CP
Deep Bellman Hedging uses reinforcement learning to optimize financial portfolio hedging.
problem Optimizing financial portfolio hedging with derivatives and trading frictions.
method Actor-critic reinforcement learning algorithm with continuous state and action spaces.
result Trained model provides optimal hedge for any initial portfolio and market state.
Deep neural network approximation for high-dimensional parabolic Hamilton-Jacobi-Bellman equationsmath.NA
Deep neural nets approximate high-dimensional HJB equations efficiently.
problem Approximating solutions to high-dimensional HJB equations.
method Deep neural networks for approximating solutions.
result Deep neural networks can approximate solutions without the curse of dimensionality.
Machine Learning and Hamilton-Jacobi-Bellman Equation for Optimal Decumulation: a Comparison Studymath.OC
A neural network approach solves optimal decumulation problems for pension plans.
problem Optimal asset allocation and withdrawal strategies for DC pension holders.
method Data-driven neural network optimization with customized activation functions.
result The neural network approach learns near-optimal solutions comparable to HJB PDE methods.
The paper tackles non-cumulative objectives in reinforcement learning and proposes modifications to existing algorithms.
problem Optimizing objectives that are not naturally expressed as summations of rewards in various fields.
method The paper modifies the Bellman optimality equation to handle non-cumulative objectives by replacing summation with a generalized operation.
result The modified Bellman updates can converge to the globally optimal solution under certain conditions.
Paper solves investment strategy optimization with deep learning.
problem Maximizing investor utility with optimal asset allocation.
method Solves PDEs with Deep Galerkin method.
result Deep learning algorithm outperforms finite difference method.
This paper aims at theoretically and empirically comparing two standard optimization criteria for Reinforcement Learning: i) maximization of the mean value and ii) minimization of the Bellman residual. For that purpose, we place ourselves in the framework of policy search algorithms, that are usually designed to maximi…
Market makers optimize trading with a new implicit scheme for complex inequalities.
problem Optimizing trading in a limit order book with stochastic and impulse control.
method Implicit numerical scheme coupled with policy iteration algorithm.
result Convergence to the unique viscosity solution of the HJBQVI.
Option Pricing Model for Incomplete Marketcond-mat.stat-mech
The problem of determining the European-style option price in the incomplete market has been examined within the framework of stochastic optimization. An analytic method based on the discrete dynamic programming equation (Bellman equation) has been developed that gives the general formalism for determining the option p…
A new estimator combines bootstrapping and rollout methods in RL.
problem Combining strengths of bootstrapping and rollout methods in RL.
method Subgraph Bellman operators and fixed point solving.
result Upper bound on error approaches optimal TD variance with additional term.
The aim of this paper is to construct and analyze solutions to a class of Hamilton-Jacobi-Bellman equations with range bounds on the optimal response variable. Using the Riccati transformation we derive and analyze a fully nonlinear parabolic partial differential equation for the optimal response function. We construct…
DSPI connects natural policy gradient to policy iteration, proving global convergence.
problem Optimizing policies in reinforcement learning.
method DSPI framework, combining smoothed policy iteration and natural policy gradient.
result DSPI achieves geometric convergence and optimal complexity for policy optimization.
Optimal contracts are found for agents with quadratic effort costs.
problem Finding optimal contracts in principal-agent problems with quadratic effort costs.
method Modeling the problem using Hamilton-Jacobi-Bellman (HJB) equations and proving the existence of classical solutions.
result Existence of optimal contracts for agents with quadratic effort costs is proven.
In this paper we propose and analyze a method based on the Riccati transformation for solving the evolutionary Hamilton-Jacobi-Bellman equation arising from the stochastic dynamic optimal allocation problem. We show how the fully nonlinear Hamilton-Jacobi-Bellman equation can be transformed into a quasi-linear paraboli…
New method improves stability of soft FQI for offline RL.
problem Stability issues in soft FQI under function approximation.
method Stationary reweighting to align operator norms.
result Local linear convergence proved under certain conditions.
New BE dimension measure reveals rich RL problems with sample-efficient algorithms.
problem Finding sample-efficient algorithms for complex RL problems.
method Introducing Bellman Eluder (BE) dimension and designing GOLF and OLIVE algorithms.
result GOLF and OLIVE algorithms learn near-optimal policies for low BE dimension problems with polynomial samples.
Improved machine learning for reservoir optimization problems.
problem Optimizing control in high-dimensional storage problems.
method Modified dynamic programming algorithm with neural networks for Bellman values and conditional cuts.
result Neural networks outperform classical feedforward networks in estimating Bellman values.
We study the structure of a simple dynamic optimization problem consisting of one state and one control variable, from a physicist's point of view. By using an analogy to a physical model, we study this system in the classical and quantum frameworks. Classically, the dynamic optimization problem is equivalent to a clas…
A model optimizes carbon emission reduction and allowance purchasing for companies.
problem Optimizing carbon emissions and allowance purchasing for companies.
method Established an optimal control model involving two stochastic processes with two control variables, converted into an HJB equation, proved existence and uniqueness of solution.
result Proved the existence and uniqueness of the solution to the HJB equation.
The Noether theorem is extended to stochastic control problems using contact symmetries.
problem Stochastic optimal control problems.
method Exploiting jet bundles and contact geometry, the authors prove the existence of conserved quantities.
result Optimal control problems admit infinitely many conserved quantities in the form of local martingales.
Optimizes control of infectious disease spread using stochastic methods.
problem Optimizing control of highly infectious diseases like COVID-19.
method Reformulated Hamilton-Jacobi-Bellman equation as stochastic minimum principle, leading to forward-backward stochastic differential equations.
result Numerous numerical solutions presented under various scenarios.
Empowerment is an information-theoretic method that can be used to intrinsically motivate learning agents. It attempts to maximize an agent's control over the environment by encouraging visiting states with a large number of reachable next states. Empowered learning has been shown to lead to complex behaviors, without …
This work is motivated by numerical solutions to Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) associated with combined stochastic and impulse control problems. In particular, we consider (i) direct control, (ii) penalized, and (iii) semi-Lagrangian discretization schemes applied to the HJBQVI proble…
Corrects gaps in a method for optimizing high-frequency trading strategies.
problem Optimizing bid and ask limit order strategies in high-frequency trading.
method Uses an approximation method based on Avellaneda and Stoikov's 2008 article, correcting gaps found in it.
result The main answer in Avellaneda and Stoikov's article remains unchanged despite corrections.
Trading strategy mimics optimal control with simple heuristic.
problem Maximizing profit in markets with limited optimal control.
method Using the gradient of the Bellman value function as effective microstructure alphas.
result Trading strategy approximates optimal behavior in long-term trades.
The Bellman error is a poor proxy for value function accuracy, even with all state-action pairs.
problem The Bellman error is a poor proxy for the accuracy of the value function.
method Study of the Bellman equation as a surrogate objective for value prediction accuracy.
result The magnitude of the Bellman error is only weakly related to the distance to the true value function, even with all state-action pairs.
Optimal Dynamic Futures Portfolios Under a Multiscale Central Tendency Ornstein-Uhlenbeck Modelq-fin.MF
Study optimal futures trading strategies for assets with multiscale central tendency price model.
problem Optimal dynamic trading of futures with multiscale central tendency price model.
method Derive no-arbitrage futures prices, solve HJB equations for optimal strategies.
result Optimal trading strategies depend on asset parameters and futures risk premia.
Deep learning for HJB PDEs using synthetic data and residual minimization.
problem Solving Hamilton-Jacobi-Bellman PDEs for optimal control problems.
method Gradient-augmented synthetic dataset for supervised learning, residual minimization.
result Improves accuracy and efficiency of deep learning for HJB PDEs.
New method stabilizes FQE by reweighting Bellman targets.
problem Stability guarantees for FQE often rely on Bellman completeness, which can fail with function approximation.
method Proposes stationary-weighted FQE, reweighting Bellman targets by stationary target-to-behavior density ratio.
result Proves finite-sample linear convergence to stationary projected Bellman fixed point without Bellman completeness.
The paper solves a complex financial optimization problem using a novel mathematical technique.
problem Optimizing portfolio selection in financial markets.
method Maximal monotone operator method and Riccati transformation.
result Existence and uniqueness of a solution to the transformed parabolic equation in a Sobolev space.
Study optimal investment strategies for an insurer in two currency markets.
problem Maximizing expected exponential utility of terminal wealth for an insurer in two currency markets.
method Dynamic programming method applied to solve Hamilton-Jacobi-Bellman equations.
result Optimal investment strategies and value functions are derived.
Deep nets solve MDPs without high dimensions.
problem Solving Bellman equations for MDPs in high dimensions.
method Deep neural networks with ReLU activation approximating payoff and transition functions.
result Deep nets can approximate -functions in polynomially bounded parameters.
Study optimal consumption and investment strategies with leverage constraints using Epstein-Zin utility.
problem Optimal portfolio choice under leverage constraints and Epstein-Zin utility.
method Established viscosity solution to HJB equation, demonstrated smoothness, characterized optimal strategies, derived explicit solutions.
result Explicit solutions for optimal consumption and investment strategies under leverage constraints.
The paper calibrates SPX and VIX options using optimal transport.
problem Joint calibration of SPX and VIX options or futures.
method Semimartingale optimal transport problem with PDE formulation and dual formulation.
result The model accurately calibrates SPX, VIX options, and futures simultaneously.
A new method calibrates value predictions in offline RL to improve reliability.
problem Difficulty in long-horizon value prediction in offline reinforcement learning.
method Bellman calibration, a weak reliability criterion, and Iterated Bellman Calibration.
result Finite-sample guarantees show that Bellman calibration error is controlled at nonparametric rates.
Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcs.LG
Improved risk-sensitive RL with exponential Bellman equation and better regret bounds.
problem Exponential gap between upper and lower bounds in risk-sensitive RL.
method Identified and addressed deficiencies in existing algorithms and analysis; developed novel analysis and exploration mechanism.
result Improved regret upper bounds over existing ones.
We study an optimal execution problem in a continuous-time market model that considers market impact. We formulate the problem as a stochastic control problem and investigate properties of the corresponding value function. We find that right-continuity at the time origin is associated with the strength of market impact…
New Bellman error estimator improves offline model selection performance.
problem Selecting the best policy from logged data using mean squared Bellman error.
method Developed a more accurate estimator of MSBE and analyzed conditions for successful OMS.
result New estimator achieves impressive offline model selection performance on diverse tasks.