On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “constrained reinforcement learning”

Constrained episodic reinforcement learning in concave-convex and knapsack settingscs.LG

Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.

problem Constrained episodic reinforcement learning with concave rewards and convex constraints.

method Modular analysis with strong theoretical guarantees for concave-convex and knapsack settings.

result Significantly outperforms existing approaches in constrained episodic environments.

reinforcement learning constrained optimization episodic concave-convex knapsack

2020-06-09abs ↗pdf ↗

Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learningcs.LG

Develops new reinforcement learning methods for complex constrained decision-making problems.

problem Complex constrained decision-making problems with a continuum of constraints.

method Proposes semi-infinitely constrained Markov decision processes (SICMDPs) and two reinforcement learning algorithms: SI-CRL and SI-CPO.

result Demonstrates the effectiveness of SI-CRL and SI-CPO in solving complex sequential decision-making tasks.

Reinforcement Learning Markov Decision Processes Constrained Optimization Semi-Infinitely Programming Deep Reinforcement Learning

2023-04-29abs ↗pdf ↗

Foundations of Safe Online Reinforcement Learning in the Linear Quadratic Regulator:

\sqrt{T}

-Regretstat.ML

Safe RL in linear systems achieves $\sqrt{T}$ -regret.

problem Efficiently learning in safety-constrained online reinforcement learning.

method Study of linear quadratic regulator with safety constraints.

result First safe algorithm with

ilde{O}_T(\sqrt{T})

-regret.

safe reinforcement learning linear dynamical systems regret bounds online learning control theory

2025-04-25abs ↗pdf ↗

Novel Policy Seeking with Constrained Optimizationcs.LG

Novel methods generate diverse policies in reinforcement learning.

problem Generating diverse policies in reinforcement learning.

method Constrained optimization perspective, introducing new metrics, and novel policy generation methods.

result Improved novelty and performance of generated policies.

reinforcement learning novel policies constrained optimization diversity cumulative reward

2020-05-21abs ↗pdf ↗

Constrained Combinatorial Optimization with Reinforcement Learningcs.LG

Paper uses RL to solve constrained combinatorial optimization problems.

problem Constrained combinatorial optimization problems.

method Extending NCO theory to CMDPs, using RL with penalty signals.

result Proposes a superior method for solving constrained problems.

Reinforcement Learning Combinatorial Optimization Constraint Satisfaction Markov Decision Processes Job Shop Problem

2020-06-22abs ↗pdf ↗

Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learningcs.LG

Eikonal-Constrained QRL improves goal-reaching in reinforcement learning.

problem Reward design and out-of-distribution generalization in reinforcement learning.

method Eikonal-Constrained Quasimetric Reinforcement Learning (Eik-QRL) using the Eikonal PDE.

result Eik-QRL achieves state-of-the-art performance in offline goal-conditioned navigation and manipulation tasks.

reinforcement learning goal-reaching quasimetric Eikonal PDE hierarchical

2025-12-12abs ↗pdf ↗

Offline Constrained Reinforcement Learning under Partial Data Coveragestat.ML

Oracle-efficient algorithm for offline RL with partial data coverage.

problem Offline reinforcement learning with partial data coverage and constraints.

method PDOCRL, a primal-dual algorithm with decomposed linear-programming formulation.

result Near-optimal, near-feasible policy with $\widetilde{\mathcal O}(ε^{-2})$ sample guarantee.

offline reinforcement learning constrained reinforcement learning oracle efficiency partial data coverage primal-dual algorithm

2025-05-23abs ↗pdf ↗

Efficient Exploration in Constrained Environments with Goal-Oriented Reference Pathcs.LG

Paper tackles efficient navigation in constrained environments using supervised and reinforcement learning.

problem Building learning agents that efficiently navigate in obstacle-cluttered environments.

method Synergistic use of supervised learning for path prediction and reinforcement learning for path following.

result Proposed method achieves good generalization and faster learning compared to existing work.

navigation constrained environments reinforcement learning supervised learning path planning

2020-03-03abs ↗pdf ↗

Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecificationcs.LG

Algorithm mitigates performance loss in constrained reinforcement learning with model misspecification.

problem Performance loss in reinforcement learning policies due to model misspecification in constrained control systems.

method Proposes an algorithm to handle constrained model misspecification in continuous control systems.

result Algorithm successfully mitigates performance loss in real-world reinforcement learning tasks.

Reinforcement Learning Model Misspecification Continuous Control Constraint Satisfaction Robustness

2020-10-20abs ↗pdf ↗

Deep Constrained Q-learningcs.LG

Proposes Constrained Q-learning for reinforcement learning with constraints.

problem Optimizing multiple objectives while adhering to constraints in reinforcement learning.

method Directly restricts the action space in Q-update to learn optimal Q-function for constrained MDP.

result Improves safety and optimality in high-level decision making for autonomous driving.

reinforcement learning constraint satisfaction Q-learning autonomous driving safe policies

2020-03-20abs ↗pdf ↗

State-Constrained Offline Reinforcement Learningstat.ML

State-constrained offline RL expands RL's learning scope.

problem Batch-constrained offline RL limits policies to seen actions.

method Introduces state-constrained offline RL focusing on state distribution.

result Policy can take high-quality out-of-distribution actions.

reinforcement learning offline RL state-constrained distributional shift deep learning

2024-05-23abs ↗pdf ↗

Off-Policy Deep Reinforcement Learning without Explorationcs.LG

Many practical applications of reinforcement learning constrain agents to learn from a fixed batch of data which has already been gathered, without offering further possibility for data collection. In this paper, we demonstrate that due to errors introduced by extrapolation, standard off-policy deep reinforcement learn…

2018-12-07abs ↗pdf ↗

Entropic Risk Constrained Soft-Robust Policy Optimizationcs.LG

Paper proposes risk-averse reinforcement learning algorithms.

problem Managing model uncertainty in reinforcement learning.

method Entropic risk constrained policy gradient and actor-critic algorithms.

result Demonstrates usefulness of risk-averse algorithms on various domains.

reinforcement learning model uncertainty risk management policy optimization

2020-06-20abs ↗pdf ↗

Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learningcs.LG

Algorithm optimizes constrained reinforcement learning with dual variables.

problem Minimizing convex functional subject to convex constraint in large state spaces.

method VPDPO algorithm using Lagrangian and Fenchel duality.

result Achieves sublinear regret and constraint violation, globally optimal policy.

Reinforcement Learning Constrained Optimization Dual Variables Optimism in Face of Uncertainty Function Approximation

2024-02-16abs ↗pdf ↗

A Globally Convergent Evolutionary Strategy for Stochastic Constrained Optimization with Applications to Reinforcement Learningcs.NE

Novel evolutionary strategy solves stochastic constrained optimization problems.

problem Optimizing objective functions with stochastic constraints in reinforcement learning.

method Design of a novel optimization algorithm with a sufficient decrease mechanism for stochastic constrained problems.

result Demonstrated convergence of the algorithm on control tasks and constrained optimization problems.

Evolutionary strategy Stochastic optimization Reinforcement learning Constrained optimization Sufficient decrease

2022-02-21abs ↗pdf ↗

Convergent Policy Optimization for Safe Reinforcement Learningcs.LG

We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions. For such a problem, we construct a sequence of surrogate convex constrained optimiza…

2019-10-26abs ↗pdf ↗

On Bellman's principle of optimality and Reinforcement learning for safety-constrained Markov decision processeess.SY

Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.

problem Optimality in safety-constrained Markov decision processes with multichain structure.

method Formulated as a zero-sum game, constructed asynchronous value iteration scheme and modified Q-learning algorithm.

result Resolved Bellman's principle of optimality for multichain Markov decision processes and provided learning algorithms.

Markov decision process Reinforcement learning Safety constraints Optimality Multi-objective optimization

2023-02-25abs ↗pdf ↗

Online 3D Bin Packing with Constrained Deep Reinforcement Learningcs.LG

Solves online 3D bin packing with deep reinforcement learning under constraints.

problem Challenges of packing items immediately without information and constraints.

method Constrained deep reinforcement learning (DRL) with feasibility predictor.

result Significantly outperforms state-of-the-art methods in online 3D bin packing.

3D bin packing online packing deep reinforcement learning constrained DRL feasibility predictor

2020-06-26abs ↗pdf ↗

A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learningcs.LG

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

problem Offline constrained reinforcement learning with general function approximation.

method Primal-Dual-Critic Algorithm (PDCA) using a primal-dual approach.

result PDCA finds a near saddle point of the Lagrangian, nearly optimal for constrained RL.

offline RL constrained RL primal-dual no-regret near saddle point

2023-06-13abs ↗pdf ↗

Reinforcement Learning with Budget-Constrained Nonparametric Function Approximation for Opportunistic Spectrum Accesscs.IT

Opportunistic spectrum access is one of the emerging techniques for maximizing throughput in congested bands and is enabled by predicting idle slots in spectrum. We propose a kernel-based reinforcement learning approach coupled with a novel budget-constrained sparsification technique that efficiently captures the envir…

2017-06-14abs ↗pdf ↗

Constrained Policy Optimization with Cantelli-Bounded Value-at-Riskcs.LG

VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.

problem Optimizing VaR-constrained reinforcement learning problems.

method Combines Cantelli's inequality and trust-region framework for efficient and conservative optimization.

result Achieves zero constraint violations during training in feasible environments.

reinforcement learning value-at-risk policy optimization conservative methods trust-region

2026-01-30abs ↗pdf ↗

ClauseLens: Clause-Grounded, CVaR-Constrained Reinforcement Learning for Trustworthy Reinsurance Pricingcs.LG

ClauseLens uses reinforcement learning to price reinsurance treaties transparently and auditably.

problem Opaque and difficult-to-audit reinsurance treaty pricing practices.

method ClauseLens models treaty pricing as a Risk-Aware Constrained Markov Decision Process (RA-CMDP), incorporating legal clauses and generating interpretable explanations.

result ClauseLens reduces solvency violations and improves tail-risk performance, achieving 88.2% accuracy in clause-grounded explanations.

reinforcement learning reinsurance treaty pricing legal clauses interpretability

2025-10-09abs ↗pdf ↗

KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systemscs.LG

KCRL learns stable policies for nonlinear systems with formal guarantees.

problem Lack of stabilization guarantees in RL methods for safety-critical systems.

method KCRL uses Krasovskii's Lyapunov functions as a stability constraint and a primal-dual approach to learn stabilizing policies.

result KCRL guarantees learning a stabilizing policy in a finite number of interactions.

Reinforcement Learning Stability Guarantees Nonlinear Systems Lyapunov Functions Policy Optimization

2022-06-03abs ↗pdf ↗

Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxationstat.ML

Study uses DRL with Lagrangian relaxation to solve temporal control tasks with STL constraints.

problem Optimal control problems with temporal logic constraints.

method Extended CMDP formulation, Lagrangian relaxation, two-phase constrained DRL algorithm.

result Demonstrated learning performance of the proposed algorithm through simulations.

Deep reinforcement learning Temporal logic constraints Lagrangian relaxation Optimal control Markov decision process

2022-01-21abs ↗pdf ↗

Optimization Issues in KL-Constrained Approximate Policy Iterationcs.LG

KL-constrained API shows optimization issues and improved with regularization.

problem Optimization issues in KL-constrained API algorithms.

method Comparison of KL divergence as a constraint vs. regularizer, empirical evaluation.

result KL-constrained API is not guaranteed to converge and incurs linear regret.

reinforcement learning approximate policy iteration KL divergence optimization issues constrained algorithms

2021-02-11abs ↗pdf ↗

Foundations of Safe Online Reinforcement Learning in the Linear Quadratic Regulator: Generalized Baselinesstat.ML

This work establishes safe reinforcement learning for LQR with nonlinear baselines.

problem Safe reinforcement learning in LQR with unknown dynamics and safety constraints.

method General framework for nonlinear baselines, focusing on 1D spaces.

result Achieves optimal regret bounds for constrained reinforcement learning.

Reinforcement Learning Safe Control Linear Quadratic Regulator Nonlinear Baselines Regret Analysis

2024-10-28abs ↗pdf ↗

Mathematical methods of reinforcement learningmath.OC

Survey of mathematical foundations for reinforcement learning.

problem Design and analysis of modern reinforcement learning algorithms.

method Organizes mathematical structures from probability, optimization, and operator theory.

result Unified mathematical entry point for researchers in various fields.

reinforcement learning Markov decision processes Bellman operators stochastic approximation convex duality

2026-07-08abs ↗pdf ↗

Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tighteningcs.LG

We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…

2016-11-05abs ↗pdf ↗

Ensuring Safety in an Uncertain Environment: Constrained MDPs via Stochastic Thresholdscs.LG

New algorithm for reinforcement learning in uncertain environments with unknown thresholds.

problem Safety in reinforcement learning in unknown and uncertain environments.

method Growing-Window estimator sampling and Stochastic Pessimistic-Optimistic Thresholding (SPOT) algorithm.

result Achieves sublinear regret and constraint violation of

ilde{\mathcal{O}}(\sqrt{T})

reinforcement learning Markov decision processes uncertain environments thresholds safety

2025-04-07abs ↗pdf ↗

Constrained Policy Optimization with Cantelli-Bounded Value-at-Riskcs.LG

Canary optimizes VaR-constrained RL problems with a conservative bound using Cantelli's inequality.

problem Optimizing reinforcement learning policies under VaR constraints in dense cost regimes.

method Employing Cantelli's inequality to create a conservative and smooth bound on VaR constraints based on moments of cost returns. Extending trust-region framework for worst-case bounds on policy improvement and constraint violation.

result Canary reliably satisfies VaR constraints with fewest violations and earliest permanent satisfaction, while maintaining reward competitiveness.

reinforcement learning risk-averse optimization Value-at-Risk conservative bound trust-region

2026-01-30abs ↗pdf ↗

Efficient Reinforcement Learning in Factored MDPs with Application to Constrained RLcs.LG

FMDP-BF algorithm improves RL in factored MDPs with exponential regret reduction.

problem Efficient reinforcement learning in factored MDPs with constrained RL.

method FMDP-BF algorithm leveraging factorization structure of FMDPs.

result FMDP-BF's regret is exponentially smaller than optimal algorithms for non-factored MDPs.

Reinforcement Learning Factored MDPs Constrained RL Regret Minimization Algorithm Design

2020-08-31abs ↗pdf ↗

Robust Reinforcement Learning with Distributional Risk-averse formulationcs.LG

Paper robustifies reinforcement learning with risk-averse methods.

problem Making predictions robust to changes in system dynamics or rewards.

method Approximates Robust Reinforcement Learning using

Φ

-divergence and Risk-Averse formulation.

result Classical Reinforcement Learning can be robustified using standard deviation penalization.

Reinforcement Learning Robustness Risk-Averse Distributional Φ-divergence

2022-06-14abs ↗pdf ↗

Learning with Safety Constraints: Sample Complexity of Reinforcement Learning for Constrained MDPscs.LG

Study examines sample complexity for RL with safety constraints.

problem Ensuring safety in RL for systems with constraints.

method Analyzes two RL approaches: generative and online.

result Sample complexity increased by a logarithmic factor in constraints.

Reinforcement Learning Safety Constraints Sample Complexity Constrained Markov Decision Process PAC Learning

2020-08-01abs ↗pdf ↗

Adaptive and Regime-Aware RL for Portfolio Optimizationq-fin.PM

Study proposes adaptive RL for dynamic portfolio optimization.

problem Traditional portfolio optimization models fail to adapt to regime shifts.

method Regime-aware reinforcement learning framework with hybrid observations and constrained reward functions.

result Transformer PPO achieves highest risk-adjusted returns, while LSTM variants offer a good balance.

Reinforcement Learning Portfolio Optimization Regime Shifts Hybrid Observations Constrained Reward Functions

2025-09-17abs ↗pdf ↗

Constrained Upper Confidence Reinforcement Learningcs.LG

Constrained Markov Decision Processes are a class of stochastic decision problems in which the decision maker must select a policy that satisfies auxiliary cost constraints. This paper extends upper confidence reinforcement learning for settings in which the reward function and the constraints, described by cost functi…

2020-01-26abs ↗pdf ↗

Near-Optimal Sample Complexity for Online Constrained MDPscs.LG

New algorithm reduces sample complexity for safe reinforcement learning.

problem Safe reinforcement learning in constrained MDPs with performance and safety constraints.

method Model-based primal-dual algorithm balancing regret and bounded constraint violations.

result Proves near-optimal policies with bounded violations or zero violations in CMDPs.

reinforcement learning Markov decision processes online learning constrained optimization slater constant

2026-02-16abs ↗pdf ↗

Constrained Reinforcement Learning for Dynamic Optimization under Uncertaintycs.LG

A new RL method handles uncertainty and constraints in real-time optimization.

problem Real-time optimization under process uncertainty and constraints.

method Chance-constrained reinforcement learning to handle probabilistic state constraints.

result Satisfies process constraints with high probability in real-time.

reinforcement learning dynamic optimization uncertainty constraints bioprocess

2020-06-04abs ↗pdf ↗

Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimizationcs.LG

Reinforcement learning (RL) is attracting increasing interests in autonomous driving due to its potential to solve complex classification and control problems. However, existing RL algorithms are rarely applied to real vehicles for two predominant problems: behaviours are unexplainable, and they cannot guarantee safety…

2020-03-03abs ↗pdf ↗

Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learningcs.LG

A novel approach for safe offline RL using latent safety constraints.

problem Balancing safety constraints and reward maximization in offline RL.

method Conditional Variational Autoencoders for latent safety modeling, Constrained Reward-Return Maximization.

result Our approach maintains safety compliance while optimizing rewards, outperforming existing methods.

offline reinforcement learning safe RL latent safety constraints conditional variational autoencoders reward-advantage weighted regression

2024-12-11abs ↗pdf ↗

Continuous-time optimal investment with portfolio constraints: a reinforcement learning approachq-fin.MF

Study uses RL to optimize investment with financial constraints, showing exploration benefits.

problem Optimal investment with financial constraints in continuous time.

method Reinforcement learning framework, focusing on Gaussian and truncated Gaussian distributions.

result Exploration leads to more dispersed wealth distribution with heavier tails, especially with smaller exploration parameters.

reinforcement learning continuous time optimal investment portfolio constraints exploration

2024-12-14abs ↗pdf ↗

A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPsstat.ML

Improved RL algorithm with linear MDPs for offline learning with partial data coverage.

problem Efficient offline RL with linear MDPs under partial data coverage.

method Primal-dual algorithm with

O(ε^{-2})

sample complexity.

result First computationally efficient algorithm with

O(ε^{-2})

sample complexity for offline RL with linear MDPs under partial data coverage.

offline reinforcement learning linear MDPs primal-dual algorithm partial data coverage sample complexity

2024-02-07abs ↗pdf ↗

Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networkscs.IT

A novel method for efficient CDRL over wireless networks.

problem Challenges in collaborative deep reinforcement learning over wireless networks.

method Semantic-aware heterogeneous federated deep reinforcement learning (HFDRL) algorithm.

result Superior performance compared to state-of-the-art baselines.

semantic-aware collaborative deep reinforcement learning wireless networks federated learning deep reinforcement learning

2021-11-23abs ↗pdf ↗

A Natural Actor-Critic Algorithm with Downside Risk Constraintscs.LG

New algorithm for risk-sensitive reinforcement learning with natural policy gradients.

problem Risk-sensitive reinforcement learning with downside risk constraints.

method Introduce a new Bellman equation to estimate the lower partial moment of returns, use natural policy gradients, and extend Reward Constrained Policy Optimization.

result Sample-efficient estimation of partial moments and effective risk-sensitive control.

reinforcement learning risk-sensitive downside risk natural policy gradients Bellman equation

2020-07-08abs ↗pdf ↗

Constrained Markov Decision Processes via Backward Value Functionscs.LG

New RL approach tackles constrained Markov decision processes.

problem Applying RL to physical systems with safety constraints.

method Formulated as a Constrained Markov Decision Process (CMDP), introduced a safe policy improvement method.

result Agent learns to maximize returns while satisfying constraints.

Reinforcement Learning Markov Decision Processes Constraint Satisfaction Safety Cumulative Cost

2020-08-26abs ↗pdf ↗

Reward Constrained Policy Optimizationcs.LG

Solving tasks in Reinforcement Learning is no easy feat. As the goal of the agent is to maximize the accumulated reward, it often learns to exploit loopholes and misspecifications in the reward signal resulting in unwanted behavior. While constraints may solve this issue, there is no closed form solution for general co…

2018-05-28abs ↗pdf ↗

FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimizecs.LG

FISAR uses neural networks to optimize safe reinforcement learning with forward-invariant constraints.

problem Safe reinforcement learning with constraints in safety-critical environments.

method Imposing linear constraints on policy parameters' updating dynamics, using a DNN-based optimizer to satisfy these constraints.

result The policy decreases constraint violation and maximizes cumulative reward monotonically.

reinforcement learning safe control forward invariance deep neural networks constraint satisfaction

2020-06-19abs ↗pdf ↗

Trajectory Optimization for Unknown Constrained Systems using Reinforcement Learningstat.ML

In this paper, we propose a reinforcement learning-based algorithm for trajectory optimization for constrained dynamical systems. This problem is motivated by the fact that for most robotic systems, the dynamics may not always be known. Generating smooth, dynamically feasible trajectories could be difficult for such sy…

2019-03-13abs ↗pdf ↗

Meta-Reinforcement Learning for Trajectory Design in Wireless UAV Networkscs.LG

Meta-learning improves drone trajectory design for dynamic wireless networks.

problem Designing optimal trajectories for energy-constrained drones in dynamic network environments.

method Proposes a meta-learning algorithm to adaptively tune a reinforcement learning solution for trajectory design.

result Meta-tuned RL yields faster convergence and improved communication performance compared to baseline algorithms.

meta-reinforcement learning wireless drone networks trajectory design dynamic environments reinforcement learning

2020-05-25abs ↗pdf ↗