This paper introduces a new reward shaping method for average-reward reinforcement learning.
problem Speeding up convergence to an optimal policy in average-reward reinforcement learning tasks.
method Developed a temporal logic-based approach to automatically generate reward shaping functions.
result The optimal policy can be recovered using the proposed reward shaping framework.
Natural language guides reinforcement learning exploration in games.
problem Improving reinforcement learning in unseen environments.
method Used encoder-decoder networks to learn associations between natural language and state-action information, then applied this to guide agent exploration.
result Modified policy shaping algorithm outperformed Q-learning and baseline policy shaping in Frogger.
A new method shapes reinforcement learning environments by abstracting large state spaces.
problem Learning in large, noisy environments with sparse feedback.
method Environment shaping using state abstraction.
result Agent's policy in shaped environment preserves near-optimal behavior in original environment.
Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.
problem Difficulty in learning meta policies for sparse reward RL problems.
method Hyper-Meta RL framework with cross-environment meta state embedding and shaped meta reward.
result Improves meta policy generalization and efficiency for sparse reward RL problems.
Reward shaping speeds up human learning through IRL.
problem Slow learning in humans, especially for challenging tasks.
method Extended IRL algorithm with kernel methods, conducted experiments with online game players.
result Players learn desired policies more quickly with reward shaping.
A language for specifying complex reinforcement learning tasks.
problem Challenges in specifying and shaping reward functions for complex reinforcement learning tasks.
method Proposes a new language and algorithm for automatically generating and shaping reward functions.
result SPECTRL tool outperforms state-of-the-art baselines.
WDAIL uses Wasserstein distance for more effective reward shaping in IL.
problem Fixed reward functions in GAIL limit performance on complex tasks.
method Introduces Wasserstein distance and PPO for improved reward shaping and stability.
result Significant performance improvement in complex MuJoCo tasks.
Self-supervised reward prediction improves RL in sparse reward settings.
problem Data efficiency and sparse reward signals in reinforcement learning.
method Learning a state representation for reward prediction and using it to shape rewards.
result Self-supervised reward prediction enhances RL algorithms in single-goal environments.
Optimizes task sequencing in reinforcement learning to improve exploration and policy discovery.
problem Improving exploration and policy discovery in reinforcement learning.
method General optimization framework for task sequencing using different objective functions.
result Curriculum learning can improve initial performance, reduce suboptimal actions, and discover better policies.
New RL approach infers optimal policies via variational inference.
problem Manual design of reward functions for reinforcement learning.
method Variational inference for inferring policies achieving desired outcomes.
result Eliminates need for hand-crafted reward functions for diverse tasks.
We characterize the geometric shape of value functions in reinforcement learning.
problem Understanding the structure of value functions in reinforcement learning.
method Geometric and topological analysis of value function space.
result Value functions form a polytope, with specific properties like line segments.
Combines RL and imitation learning with oracle to optimize policies.
problem Optimizing policies in complex environments with limited oracle accuracy.
method Truncated Horizon Policy Search (THOR) using reward shaping.
result THOR achieves superior performance compared to RL and IL baselines.
Estimates personalized policies robust to shifts in target populations.
problem Estimating policies that perform well in diverse target populations.
method Develops methods for estimating robust policies considering shifts in outcomes and characteristics.
result Welfare-maximizing policies are robust to certain shifts in potential outcomes.
Dynamic pricing policy converges to Nash equilibrium with low regret.
problem Sequential price competition among sellers over multiple periods.
method Semi-parametric least-squares estimation of s-concave demand functions.
result Prices converge to Nash equilibrium with rate O(T−1/7) and sellers incur regret O(T5/7). Deep reinforcement learning models improve musculoskeletal running speed.
problem Optimizing a musculoskeletal model to run as fast as possible through an obstacle course.
method Deep reinforcement learning approaches, including DDPG, PPO, and TRPO, with modifications.
result Various modifications to known algorithms improve running performance.
New method optimizes ambiguity sets for robust MDPs, improving policy robustness.
problem Sensitivity of optimal policies to model misspecification in MDPs.
method Proposes a new approach to optimize the shape of ambiguity sets, departing from norm-bounded uniform and symmetric sets.
result Empirical results show practical promise of weighted near-optimal ambiguity sets.
NVMDP framework tackles non-stationary MDPs with varying discount rates.
problem Challenges in non-stationary environments and infinite-horizon formulations for reinforcement learning.
method Introduces NVMDP framework that accommodates non-stationarity and varying discount rates.
result NVMDPs provide a flexible mechanism to shape optimal policies without altering state or action spaces.
Physics-informed learning framework for pH systems and EB-PBC control.
problem Control of port-Hamiltonian systems from trajectory data.
method Co-learning of pH system model and EB-PBC through alternating optimization.
result Proven stability and robustness of the learned controller.
Study how untrained policies explore in RL environments.
problem Challenges in reinforcement learning, especially sparse or adversarial reward structures.
method Theoretical and empirical analysis of untrained deep neural policies in a toy model.
result Untrained policies generate correlated actions and non-trivial state-visitation distributions.
SIBRE boosts reinforcement learning convergence by rewarding improvement over past performance.
problem Improving the rate of convergence in reinforcement learning.
method SIBRE is a reward shaping approach that rewards improvement over the agent's own past performance.
result SIBRE converges faster and more stably to the optimal policy compared to baseline RL algorithms.
Study uses machine learning to estimate effective policies in settings with hidden individual actions.
problem Estimating effective policies in settings with hidden individual actions.
method Instrumental Regression and Generalized Method of Moments (GMM) estimator.
result Demonstrates how to estimate a good contract in principal-agent problems.
The paper uses potential functions to help reinforcement learning agents learn optimal policies.
problem Learning optimal stochastic policies in reinforcement learning.
method Augmenting the reward with potential functions and applying these to policy gradient algorithms.
result Potential-based reward shaping preserves optimality of stochastic policies and speeds up learning.
Introspection Learning asks policies introspective questions to improve learning.
problem Improving reinforcement learning efficiency and robustness.
method Synthesizes experience through direct policy introspection without interaction.
result Demonstrates improved training speed and robustness.
Proposes Constrained Q-learning for reinforcement learning with constraints.
problem Optimizing multiple objectives while adhering to constraints in reinforcement learning.
method Directly restricts the action space in Q-update to learn optimal Q-function for constrained MDP.
result Improves safety and optimality in high-level decision making for autonomous driving.
Optimal trading strategy with predictor and costs, derived equations and shape.
problem Optimal trading strategy in presence of price predictor, costs, and risk control.
method Path-integral method to derive equations for band edges, solved explicitly for Ornstein-Uhlenbeck predictor.
result Explicit equations and shape of the optimal band strategy derived and analyzed.
This paper investigates how policy conditioning affects reinforcement learning stability.
problem Improving stability and generalization of reinforcement learning agents.
method The authors study Jacobian conditioning behavior during policy optimization and propose a conditioning regularization algorithm.
result The proposed conditioning regularization algorithm enhances reinforcement learning agent generalization.
New method learns compact policies with fewer parameters for faster inference.
problem Training efficient policies for real hardware with limited resources.
method Structured random orthogonal matrices for gradient approximation.
result Compact policies with up to 13x fewer parameters than ES.
A novel efficient method for computing the Knowledge-Gradient policy for Continuous Parameters (KGCP) for deterministic optimization is derived. The differences with Expected Improvement (EI), a popular choice for Bayesian optimization of deterministic engineering simulations, are explored. Both policies and the Upper …
Policy shifts between Trump and Biden impact ESG investments, creating volatility.
problem Dramatic policy shifts between Trump and Biden administrations affect ESG investments.
method Analyzes contrasting policies of Trump and Biden administrations and their impacts on ESG investments.
result Policy changes significantly influence ESG investments, leading to volatility and portfolio reassessment.
GASIL encourages agents to imitate past good trajectories in reinforcement learning.
problem Long-term credit assignment in sparse and delayed reward environments.
method Generative Adversarial Imitation Learning (GASIL) framework.
result GASIL improves performance in reinforcement learning tasks with delayed rewards.
This study models FOMC policy decisions using debate-based LLMs.
problem Accurately predicting central bank policy decisions, especially FOMC's, is challenging.
method A novel framework that simulates FOMC's collective decision-making process through iterative rounds of LLMs interacting as agents.
result The debate-based approach significantly outperforms standard LLMs in prediction accuracy.
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
problem Complex thermal design challenges due to convection-diffusion equation and boundary interactions.
method Cooperative multi-agent deep reinforcement learning with continuous geometric representation.
result Framework learns optimal design strategies without shape derivation or differentiable objectives.
This paper analyzes optimal consumption strategies for loss-averse investors with multiplicative habit formation.
problem Optimal consumption strategies for loss-averse investors with multiplicative habit formation.
method The study uses a concave envelope of the S-shaped utility function and a nonlinear free boundary problem to analyze the HJB equation.
result The paper provides optimal consumption and investment policies in feedback form.
Proposes PIC and POIC for measuring task difficulty in RL.
problem Lack of metrics to measure task difficulty in RL.
method Introduces policy information capacity (PIC) and policy-optimal information capacity (POIC) as metrics based on mutual information.
result Empirically shows PIC and POIC correlate with task solvability better than alternatives.
FedSight AI predicts federal funds rate using LLMs and multi-agent reasoning.
problem Predicting Federal Open Market Committee's decisions on federal funds rate.
method Multi-agent framework with large language models, structured and unstructured inputs, and CoD extension for efficient reasoning.
result Achieved 93.75% accuracy and 93.33% stability in predicting FOMC outcomes.
Dynamic risk constraints help limit risky behavior in financial portfolios.
problem Static risk measures fail to control tail-risk-seeking traders.
method Introduces dynamic risk constraints applied throughout the trading horizon.
result Dynamic risk constraints can effectively limit risky behavior in portfolios.
AI solutions matched prosthetic control goals in a challenge.
problem Matching time-varying velocity vectors in musculoskeletal models.
method Deep reinforcement learning approaches with various modifications.
result Many solutions used similar techniques but implemented unique modifications.
Study visualizes actor-critic loss landscapes for inventory optimization.
problem Difficulties in solving multi-store dynamic inventory control problems.
method Low-dimensional visualizations of actor loss function.
result Loss landscapes favor optimal policies in reinforcement learning.
This research develops a dual-level reinforcement learning strategy to track daily VWAP accurately.
problem Inaccurate tracking of daily VWAP due to short trading horizons.
method Dual-level architecture using Transformer and LSTM models.
result Improves accuracy in approximating daily VWAP compared to previous models.
New method learns internal model from expert demonstrations to estimate rewards.
problem Lack of complete or good action information from expert demonstrations.
method Agent learns internal model of observations from expert state trajectories to estimate rewards.
result Successfully trains good policies from expert game-play videos.
Many practical environments contain catastrophic states that an optimal agent would visit infrequently or never. Even on toy problems, Deep Reinforcement Learning (DRL) agents tend to periodically revisit these states upon forgetting their existence under a new policy. We introduce intrinsic fear (IF), a learned reward…
TIR expands LLM capabilities by enabling problem-solving strategies.
problem Lack of a principled theory explaining why LLMs with tools are more capable.
method Formal proof and Advantage Shaping Policy Optimization (ASPO) algorithm.
result TIR model decisively outperforms pure-text models on challenging benchmarks.
This paper explores how hierarchical agent policies affect exploration in goal-driven navigation environments.
problem Understanding how hierarchical agent policies influence exploration in goal-driven navigation.
method Design of EscapeRoom environments, measuring complexity with hitting times of dependency graphs, evaluating PPO and hierarchical PPO.
result Analytically estimated hitting time in goal dependency graphs is a metric of environment complexity and hierarchical approaches are necessary for complex environments.
Paper uses DRL to optimize portfolios, balancing risk and return.
problem Optimizing portfolios under market uncertainty and risk constraints.
method Integrates Sharpe ratio-based reward with risk control mechanisms, uses PPO for adaptive asset allocation.
result DRL agent stabilizes volatility but sacrifices risk-adjusted returns.
New RL algorithms improve performance using multi-step greedy policies.
problem Improving model-free reinforcement learning performance.
method Developed multi-step greedy κ-Policy Iteration and κ-Value Iteration algorithms for model-free RL. result Multi-step greedy algorithms outperform DQN and TRPO on various benchmark tasks.
A new framework isolates exploration challenges in RL without explicit rewards.
problem Challenges in reinforcement learning, especially exploration.
method Reward-free RL framework, collecting trajectories without a reward function, then computing policies for various reward functions.
result Efficient algorithm that conducts exploration and computes near-optimal policies for multiple reward functions.
A new method combines human feedback with deep learning for faster policy learning.
problem Training deep neural networks for complex decision-making problems is data-intensive and time-consuming.
method Deep COACH (D-COACH) integrates human corrective feedback with deep learning models.
result The D-COACH framework can learn policies for continuous action spaces faster than traditional deep reinforcement learning.
Self-imitation learning improves RL in sparse, episodic reward settings.
problem Suboptimal performance of RL algorithms in sparse or episodic reward settings.
method Formulate policy optimization as a divergence minimization problem using Jensen-Shannon divergence, and learn shaped rewards from experience replays.
result Our algorithm performs comparably to existing algorithms in dense reward settings and significantly better in sparse and episodic reward settings.