Paper introduces RCaI, a risk-sensitive control method using Rényi divergence.
problem Risk-sensitive control in reinforcement learning.
method RCaI extends CaI using Rényi divergence variational inference.
result Risk-sensitive optimal policy can be obtained by solving a soft Bellman equation.
Paper introduces a new method for risk-sensitive investment management using RL.
problem Risk-sensitive portfolio management with unknown model parameters.
method Combines RL and risk-sensitive stochastic control with Gaussian perturbations for exploration.
result Endogenous relative-entropy regularization and optimal investment strategy derived.
We establish a stochastic maximum principle (SMP) for control problems of partially observed diffusions of mean-field type with risk-sensitive performance functionals.
Separates estimation and control in risk-sensitive investment problems with partial observation.
problem Risk-sensitive investment problems with incomplete observation.
method Investigates separability of a general class of risk-sensitive investment management problems using a finite-dimensional filter.
result The separated problem is strictly equivalent to the original control problem.
In this paper we study mean-field type control problems with risk-sensitive performance functionals. We establish a stochastic maximum principle (SMP) for optimal control of stochastic differential equations (SDEs) of mean-field type, in which the drift and the diffusion coefficients as well as the performance function…
Paper tackles risk-sensitive decision-making under uncertainty.
problem Risk-sensitive decision-making problem under uncertainty.
method Formulated as a stochastic control problem, delineated necessary optimality conditions.
result Illustrative examples from optimal betting and inventory management support the theory.
In this paper we consider long-run risk sensitive average cost impulse control applied to a continuous-time Feller-Markov process. Using the probabilistic approach, we show how to get a solution to a suitable continuous-time Bellman equation and link it with the impulse control problem. The optimal strategy for the und…
We study a risk sensitive control version of the lifetime ruin probability problem. We consider a sequence of investments problems in Black-Scholes market that includes a risky asset and a riskless asset. We present a differential game that governs the limit behavior. We solve it explicitly and use it in order to find …
This paper deals with discrete-time Markov control processes on a general state space. A long-run risk-sensitive average cost criterion is used as a performance measure. The one-step cost function is nonnegative and possibly unbounded. Using the vanishing discount factor approach, the optimality inequality and an optim…
Reinforcement learning for continuous-time risk-sensitive asset allocation
problem Continuous-time risk-sensitive asset allocation
method Free energy-entropy duality reformulation and q-learning actor-critic method result Optimal policy learning with high accuracy
Develops variational framework for LQG risk-sensitive MFGs with major-minor interactions.
problem Risk-sensitive optimal control in LQG systems with major-minor interactions.
method Variational approach, nonlinear necessary and sufficient condition of optimality, equivalent risk-neutral measure, Markovian closed-loop best-response strategies.
result Derives optimal control strategies for LQG risk-sensitive MFGs with major-minor interactions, establishing Nash and ε-Nash equilibria. New algorithm for risk-sensitive reinforcement learning with natural policy gradients.
problem Risk-sensitive reinforcement learning with downside risk constraints.
method Introduce a new Bellman equation to estimate the lower partial moment of returns, use natural policy gradients, and extend Reward Constrained Policy Optimization.
result Sample-efficient estimation of partial moments and effective risk-sensitive control.
We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for…
This paper introduces a method to incorporate risk sensitivity in RL using quadratic variation penalties.
problem Risk-sensitive reinforcement learning under entropy regularization.
method Equivalent martingale property and quadratic variation penalty for value process.
result The proposed method improves finite-sample performance in linear-quadratic control problems.
In this paper long-run risk sensitive optimisation problem is studied with dyadic impulse control applied to continuous-time Feller-Markov process. In contrast to the existing literature, focus is put on unbounded and non-uniformly ergodic case by adapting the weight norm approach. In particular, it is shown how to com…
Unified market making controls risk, arbitrage, and volatility surfaces.
problem Market making risk, arbitrage, and volatility surface consistency.
method Constrained RL and stochastic control for risk-sensitive execution and hedging.
result Agent achieves positive P&L with zero calendar and butterfly violations.
Solves risk-sensitive investment via duality, entropic regularization, and RL.
problem Risk-sensitive portfolio management in a factor-based setting.
method Free energy-entropy duality, Kuroda-Nagai change-of-measure, RL algorithm.
result Direct analytical solution, explicit controls, two interpretations of optimal allocation.
Study risk-sensitive market making with entropy regularization for better quote control.
problem Risk-sensitive market making with exponential utility and penalties.
method Entropy-regularized certainty-equivalent Bellman policies for discrete-time market dynamics.
result Proves convergence and performance bounds for entropy-regularized policies.
We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate …
This paper considers a portfolio optimization problem in which asset prices are represented by SDEs driven by Brownian motion and a Poisson random measure, with drifts that are functions of an auxiliary diffusion factor process. The criterion, following earlier work by Bielecki, Pliska, Nagai and others, is risk-sensit…
In this paper, we extend the jump-diffusion model proposed by Davis and Lleo to include jumps in asset prices as well as valuation factors. The criterion, following earlier work by Bielecki, Pliska, Nagai and others, is risk-sensitive optimization (equivalent to maximizing the expected growth rate subject to a constrai…
We study an open problem of risk-sensitive portfolio allocation in a regime-switching credit market with default contagion. The state space of the Markovian regime-switching process is assumed to be a countably infinite set. To characterize the value function, we investigate the corresponding recursive infinite-dimensi…
This paper investigates the finite horizon risk-sensitive portfolio optimization in a regime-switching credit market with physical and information-induced default contagion. It is assumed that the underlying regime-switching process has countable states and is unobservable. The stochastic control problem is formulated …
Deep neural RDEs improve portfolio optimization accuracy and risk sensitivity.
problem High-dimensional, path-dependent valuation and control problems.
method Coupling truncated log-signatures with a neural RDE backbone.
result Improved accuracy, tail fidelity, and training stability across various financial models.
Study risk-sensitive RL in offline settings, improving efficiency and accuracy.
problem Efficiently derive near-optimal policies for risk-sensitive RL using offline data.
method Introduced two provably sample-efficient algorithms for risk-sensitive offline RL in linear MDPs.
result First provably efficient risk-sensitive offline RL algorithms.
Paper introduces risk-sensitive bandits with optimal arm mixtures.
problem Designing algorithms for risk-sensitive multi-armed bandits.
method Formalizes risk-sensitive bandits framework, identifies optimal arm mixtures, designs regret-efficient algorithms.
result Regret-efficient algorithms track optimal arm mixtures or solitary arms.
This paper considers a portfolio optimization problem in which asset prices are represented by SDEs driven by Brownian motion and a Poisson random measure, with drifts that are functions of an auxiliary diffusion 'factor' process. The criterion, following earlier work by Bielecki, Pliska, Nagai and others, is risk-sens…
We consider a long-term optimal investment problem where an investor tries to minimize the probability of falling below a target growth rate. From a mathematical viewpoint, this is a large deviation control problem. This problem will be shown to relate to a risk-sensitive stochastic control problem for a sufficiently l…
Improved risk-sensitive RL with exponential Bellman equation and better regret bounds.
problem Exponential gap between upper and lower bounds in risk-sensitive RL.
method Identified and addressed deficiencies in existing algorithms and analysis; developed novel analysis and exploration mechanism.
result Improved regret upper bounds over existing ones.
Improved sample complexity for identifying best policies in risk-sensitive reinforcement learning.
problem Identifying approximately optimal policies in risk-sensitive reinforcement learning with exponential horizon dependence.
method Forward-model based algorithm with KL-based exploration bonuses adapted for entropic criterion, leveraging smoothness properties of exponential utility and a new stopping rule.
result Achieved sample complexity matching the lower bound, closing the gap between upper and lower bounds.
Study risk-sensitive reinforcement learning with Lipschitz dynamic risk measures, establishing regret bounds.
problem Risk-sensitive reinforcement learning in Markov decision processes.
method Two model-based algorithms for Lipschitz dynamic risk measures, focusing on regret bounds.
result Upper bounds demonstrate optimal dependencies on actions and episodes, reflecting risk sensitivity vs. sample complexity trade-off.
Develops an actor-critic algorithm for risk-sensitive Markov decision processes.
problem Risk-sensitive cost criterion in Markov decision processes.
method Actor-critic algorithm with function approximation.
result Asymptotic convergence of the actor-critic algorithm.
We explore a new method for discrete-time control problems using randomization and entropy.
problem Discrete-time linear-exponential quadratic Gaussian (LEQG) control problem.
method Introduce exploration through randomization and apply duality between free energy and relative entropy.
result Reduced LEQG problem to equivalent risk-neutral LQG control problem with entropy regularization.
Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.
problem Optimizing decision-making under risk in Markov processes.
method Analyzes risk-sensitive criteria using Optimized Certainty Equivalent, including entropic risk and Conditional Value-at-Risk.
result Conditions for the existence of optimal policies and solution procedures are provided.
Study risk-sensitive reinforcement learning with entropic risk measures and generative models.
problem Risk-sensitive reinforcement learning in discounted MDPs with recursive entropic risk measures.
method Introduced Model-Based ERM Q-Value Iteration (MB-RS-QVI) and derived PAC bounds on sample complexity for value and policy learning. result PAC bounds show exponential dependence on ∣β∣/(1−γ), with tight bounds in S and A. In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in rewards in addition to maximizing a standard criterion. Variance related risk measures are among the most common risk-sensitive criteria in finance and operations research. However, optimizing many such c…
This paper analyzes risk-sensitive reinforcement learning with Conditional Value-at-Risk (CVaR) for robust Markov Decision Processes.
problem Risk-sensitive reinforcement learning for robust Markov Decision Processes (RMDPs) with state-action-dependent ambiguity sets.
method The paper establishes a connection between robustness and risk sensitivity, defining a new risk measure NCVaR and proposing value iteration algorithms.
result The proposed approach using NCVaR optimization and value iteration algorithms can solve problems with state-action-dependent ambiguity sets.
The paper uses stochastic control to analyze interest rate markets with roll-over risk.
problem Analyzing interest rate markets with roll-over risk without classical arbitrage assumptions.
method Stochastic optimal control problems with power-type objective functionals.
result Endogenously determined funding-liquidity spread.
New algorithms optimize risk in reinforcement learning with exponential utility.
problem Optimizing rewards under risk in reinforcement learning with unknown transition kernels.
method Two model-free algorithms: Risk-Sensitive Value Iteration (RSVI) and Risk-Sensitive Q-learning (RSQ).
result Proved near-optimal regret bounds for RSVI and RSQ.
Kuroda and Nagai \cite{KN} state that the factor process in the Risk Sensitive control Asset Management (RSCAM) is stable under the Föllmer-Schweizer minimal martingale measure . Fleming and Sheu \cite{FS} and more recently Föllmer and Schweizer \cite{FoS} have observed that the role of the minimal martingale measure i…
This survey reviews portfolio selection problem for long-term horizon. We consider two objectives: (i) maximize the probability for outperforming a target growth rate of wealth process (ii) minimize the probability of falling below a target growth rate. We study the asymptotic behavior of these criteria formulated as l…
Study gap-dependent regret bounds for risk-sensitive RL.
problem Risk-sensitive reinforcement learning with entropic risk measure.
method Propose cascaded gaps to adapt to problem structures, derive regret bounds.
result Exponential improvement over existing bounds in appropriate settings.
Novel framework for risk-sensitive reinforcement learning using martingale decomposition.
problem Risk sensitivity in sequential decision-making with uncertain rewards.
method Martingale decomposition and chaotic variation for reward uncertainty, integrated into model-free reinforcement learning algorithms.
result Demonstrated relevance of risk-sensitive reinforcement learning in grid world and portfolio optimization problems.
For a stochastic factor model we maximize the long-term growth rate of robust expected power utility with parameter λ∈(0,1). Using duality methods the problem is reformulated as an infinite time horizon, risk-sensitive control problem. Our results characterize the optimal growth rate, an optimal long-term trading s…
Novel framework for risk-sensitive reinforcement learning with robustness against uncertainty.
problem Risk-sensitive reinforcement learning with uncertainty in transition dynamics.
method Developed a risk-sensitive robust Markov decision process (RSRMDP), derived its Bellman equation, and proposed a Bayesian Dynamic Programming (Bayesian DP) algorithm.
result Demonstrated convergence to near-optimal policies and analyzed sample and computational complexities.
The paper studies risk-sensitive learning schemes and provides learning bounds for empirical OCE minimizers.
problem Risk-sensitive learning aims to minimize risk-averse measures of loss.
method Proposes learning bounds for empirical OCE minimizers based on Rademacher average and variance.
result Provides two learning bounds on the performance of empirical OCE minimizers.
Survey of theoretical foundations for policy optimization in control.
problem Understanding the theoretical properties of gradient-based methods in control and reinforcement learning.
method Interdisciplinary review of optimization landscape, convergence, and sample complexity for various control problems.
result Recent theoretical results on stability and robustness in learning-based control.
Framework improves ETF volatility forecasting by adapting to market conditions.
problem Challenges in volatility forecasting due to shifting market conditions and varying model performance.
method Risk-sensitive specialist routing using online risk-sensitive evaluation and state-dependent gating.
result Reduces forecast loss by 24% and underprediction loss by 22% compared to rolling-best baseline.