In this paper, we analyze dynamic programming as a novel approach to solve the problem of maximizing the profits of a bank. The mathematical model of the problem and the description of a bank's work is described in this paper. The problem is then approached using the method of dynamic programming. Dynamic programming m…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “dynamic programming”
Approximate dynamic programming algorithms, such as approximate value iteration, have been successfully applied to many complex reinforcement learning tasks, and a better approximate dynamic programming algorithm is expected to further extend the applicability of reinforcement learning to various tasks. In this paper w…
This paper improves dynamic hedging accuracy using genetic programming to forecast implied volatilities.
problem Improving the accuracy of dynamic hedging using implied volatilities.
method The paper uses genetic programming to forecast implied volatilities and tests the performance of these forecasts in dynamic hedging strategies.
result Genetic programming-generated implied volatilities improve hedging accuracy compared to static training methods.
dynestyx: A library for probabilistic programming of dynamical systems
problem integrating state-space models into probabilistic programming languages
method a unified interface for specifying priors and performing inference
result principled uncertainty quantification for state and parameters
New method uses dynamic programming for meta continual learning.
problem Challenges of generalization and catastrophic forgetting in sequential learning.
method Developed a theoretical framework using dynamic programming for meta continual learning.
result Theoretical and practical method achieves better accuracy than existing methods.
Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of dimensionality by minimizing a pessimistic bound on the policy loss. This approach tur…
Framework optimizes battery storage for markets by separating long-term degradation from short-term market dynamics.
problem Intractable computation due to timescale mismatch between battery degradation and market dynamics.
method Approximate dynamic programming with value function approximation and pseudo-time encoding.
result Policy outperforms benchmarks in real-time market scenarios.
Unsupervised clustering of series using dynamic programming.
problem Clustering coherent segments in multi-variate series.
method Dynamic programming algorithm with constraints.
result Clusters of coherent segments identified using Waxman-Smits equation.
DP-Net uses dynamic programming for efficient deep neural network compression.
problem Efficiently compressing deep neural networks while maintaining accuracy.
method Dynamic Programming for optimal weight quantization and clustering-friendly training.
result Achieves up to 77X compression ratio on Wide ResNet with minimal accuracy loss.
We present the preliminary high-level design and features of DynamicPPL.jl, a modular library providing a lightning-fast infrastructure for probabilistic programming. Besides a computational performance that is often close to or better than Stan, DynamicPPL provides an intuitive DSL that allows the rapid development of…
Deep learning solves dynamic programming with recursive utility.
problem Challenges in solving high-dimensional discrete-time dynamic programming problems with recursive utility.
method Certainty Equivalent Learning (CEL) algorithm that learns certainty-equivalent value directly with neural networks.
result Accurate value and policy approximations in high-dimensional problems, comparable to VFI in some cases.
In this paper, we consider the problem of optimization of a portfolio consisting of securities. An investor with an initial capital, is interested in constructing a portfolio of securities. If the prices of securities change, the investor shall decide on reallocation of the portfolio. At each moment of time, the prices…
Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decompositioncs.RO
New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.
problem Efficiently discovering rare failure events in autonomous vehicle simulations.
method Approximate dynamic programming and scene decomposition to estimate failure distribution.
result Increased number of failures discovered compared to baseline approaches.
Optimal Control Theory optimizes neural networks, improving robustness and efficiency.
problem Optimizing deep neural networks (DNNs) for better performance and efficiency.
method Integrating Optimal Control Theory with Backpropagation to develop a new optimizer.
result Optimal Control Theoretic Neural Optimizer (OCNOpt) improves upon existing methods in robustness and efficiency.
New method solves nonseparable stochastic control problems.
problem Nonseparable and non-monotonic stochastic control problems.
method Scenario-decomposition solution framework using progressive hedging algorithm.
result Extends reach of stochastic optimal control.
A neural network approach solves dynamic portfolio optimization without dynamic programming.
problem Dynamic portfolio optimization with multiple constraints and high rebalancing frequency.
method Parsimonious neural network without dynamic programming, avoiding high-dimensional expectations.
result Proves convergence to theoretical optimal solution under general conditions.
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
CEFOL uses deep learning for dynamic programming with recursive utility.
problem Challenges in solving dynamic programming problems with recursive utility.
method Introduces a separate neural network for certainty equivalent, uses first-order optimality conditions to learn value and policy functions.
result CEFOL achieves high accuracy in learning value and policy functions, matching VFI benchmarks.
This research develops an evolutionary approach to discover non-Gaussian stochastic dynamical systems.
problem Discovering explicit governing equations of stochastic dynamical systems with Lévy noise from data.
method ESSR approach using genetic programming, sparse regression, and nonlocal Kramers-Moyal formulas.
result The approach effectively extracts non-Gaussian stochastic dynamical systems from sample path data.
Dynamic rule-based investment strategies outperform static ones in pension schemes.
problem Managing retirement income with dynamic investment strategies.
method Rule-based investment strategies compared to dynamic programming.
result Rule-based strategies achieve higher probability of meeting retirement income targets.
Algorithm selects best model based on state, reducing costs.
problem Choosing the best model among many in different states of the world.
method Reinforcement learning algorithm to estimate optimal policy.
result Algorithm consistently selects optimal model based on covariates.
Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic Programmingstat.ML
Paper introduces a method to assess the statistical reliability of changepoints using selective inference and dynamic programming.
problem Assessing the statistical reliability of detected changepoints.
method Selective inference framework combined with dynamic programming for exact p-value computation.
result Proposes a method with high statistical power and decent computational efficiency.
Solves portfolio optimization with costs using numerical methods.
problem Dynamic portfolio optimization with transaction costs and constraints.
method Numerical dynamic programming techniques.
result Problems can now be solved tractably.
We consider an optimal stopping problem where a constraint is placed on the distribution of the stopping time. Reformulating the problem in terms of so-called measure-valued martingales allows us to transform the marginal constraint into an initial condition and view the problem as a stochastic control problem; we esta…
DPDP combines neural heuristics with DP for vehicle routing problems.
problem Vehicle routing problems with large scale.
method Deep Policy Dynamic Programming (DPDP) that uses a neural network policy to prioritize and restrict the DP state space.
result DPDP improves upon classical DP algorithms and outperforms neural approaches for TSP, VRP, and TSPTW.
Paper approximates solutions for complex decision processes with limited precision.
problem Approximating the set of all solutions for Multi-objective Markov Decision Processes.
method Limited precision approach based on White's multi-objective value-iteration dynamic programming algorithm.
result The number of calculated solutions is tractable and approximates the true Pareto front.
Reinforcement learning has gained wide popularity as a technique for simulation-driven approximate dynamic programming. A less known aspect is that the very reasons that make it effective in dynamic programming can also be leveraged for using it for distributed schemes for certain matrix computations involving non-nega…
This paper studies dynamic stochastic optimization problems parametrized by a random variable. Such problems arise in many applications in operations research and mathematical finance. We give sufficient conditions for the existence of solutions and the absence of a duality gap. Our proof uses extended dynamic programm…
The paper analyzes error propagation in dynamic programming for stochastic control and option pricing.
problem Error propagation in dynamic programming for stochastic control and option pricing.
method Formulated a general dynamic programming framework, used RKHSs for nonparametric regression, and Monte Carlo subsampling for estimating continuation value.
result Proposed a rigorous error decomposition and control mechanism for error propagation in dynamic programming.
Real-world problems of operations research are typically high-dimensional and combinatorial. Linear programs are generally used to formulate and efficiently solve these large decision problems. However, in multi-period decision problems, we must often compute expected downstream values corresponding to current decision…
Some of the most important tasks take place in environments which lack cheap and perfect simulators, thus hampering the application of model-free reinforcement learning (RL). While model-based RL aims to learn a dynamics model, in a more general case the learner does not know a priori what the action space is. Here we …
Paper analyzes convergence of dynamic policy gradient for MDPs, improving performance in finite-time problems.
problem Optimal policies in finite-time MDPs are not stationary and require epoch-specific training.
method Introduces dynamic policy gradient combining dynamic programming and policy gradient, analyzes convergence for softmax parametrisation.
result Dynamic policy gradient training exploits finite-time structure, leading to better convergence bounds.
This paper improves volatility forecasting using dynamic subset selection in genetic programming.
problem Improving accuracy of implied volatility forecasting.
method Dynamic training-subset selection methods applied to genetic programming.
result Dynamic subset selection improves predictive accuracy of genetic programming models.
Improves probabilistic programming by analyzing program structure.
problem Inefficiency and limitations of single inference algorithms in probabilistic programming.
method Three novel techniques: static and dynamic analyses to adapt programs for more efficient inference.
result Improves probabilistic programming by making inference more efficient.
We describe an abstract control-theoretic framework in which the validity of the dynamic programming principle can be established in continuous time by a verification of a small number of structural properties. As an application we treat several cases of interest, most notably the lower-hedging and utility-maximization…
In this paper a neural network heuristic dynamic programing (HDP) is used for optimal control of the virtual inertia based control of grid connected three phase inverters. It is shown that the conventional virtual inertia controllers are not suited for non inductive grids. A neural network based controller is proposed …
This work presents the concept of kernel mean embedding and kernel probabilistic programming in the context of stochastic systems. We propose formulations to represent, compare, and propagate uncertainties for fairly general stochastic dynamics in a distribution-free manner. The new tools enjoy sound theory rooted in f…
The classical optimal investment and consumption problem with infinite horizon is studied in the presence of transaction costs. Both proportional and fixed costs as well as general utility functions are considered. Weak dynamic programming is proved in the general setting and a comparison result for possibly discontinu…
New method for optimistic planning in MDPs using regularization.
problem Optimistic planning in infinite-horizon discounted MDPs.
method Regularized dynamic programming for approximate value iteration.
result Achieves near-optimal statistical guarantees in learning policies.
We provide a dynamic programming principle for stochastic optimal control problems with expectation constraints. A weak formulation, using test functions and a probabilistic relaxation of the constraint, avoids restrictions related to a measurable selection but still implies the Hamilton-Jacobi-Bellman equation in the …
Linear time dynamic programming for the exact path of optimal models selected from a finite setcs.LG
New algorithm speeds up path computation for optimal models.
problem Finding the exact path of optimal models from a finite set.
method Dynamic programming approach for linear time computation.
result Dynamic programming achieves linear time for breakpoints computation.
A Finsler Geometrical Programming for the Nonlinear Complementarity Problem of Traffic Equilibriummath.OC
Geometric programming approach for traffic equilibrium problems.
problem Optimizing traffic equilibrium in transportation systems.
method Finslerian dynamical model for nonlinear complementarity problems.
result Effective solution for various equilibrium problems in transportation.
We describe an approximate dynamic programming (ADP) approach to compute approximations of the optimal strategies and of the minimal losses that can be guaranteed in discounted repeated games with vector-valued losses. Such games prominently arise in the analysis of regret in repeated decision-making in adversarial env…
Price responsiveness is a major feature of end use customers (EUCs) that participate in demand response (DR) programs, and has been conventionally modeled with static demand functions, which take the electricity price as the input and the aggregate energy consumption as the output. This, however, neglects the inherent …
This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the valu…
New metric derived for robust optimization in stochastic control problems.
problem Non-parametric uncertainty in multiperiod stochastic control problems.
method Derived a new metric, adapted --Wasserstein distance, and used dynamic programming principle.
result Dynamic programming principle for DRO problems with semi-separable cost functions.
Develops a dynamic mean field theory for reinforcement learning.
problem Finite state and action Bayesian reinforcement learning in large state spaces.
method Analogies with statistical physics, interpreting probabilities as couplings and values as spins, solving mean field equations.
result State-action values are statistically independent in the asymptotic state space limit, with exact or approximate equations for computation.
ExDBN: Learning Dynamic Bayesian Networks using Extended Mixed-Integer Programming Formulationscs.LG
ExDBN learns dynamic Bayesian networks using mixed-integer programming.
problem Learning dynamic causal relationships from time series data.
method Score-based learning algorithm using mixed-integer quadratic programming with branch-and-cut method.
result The proposed method produces more accurate results than state-of-the-art approaches.