A new method for MARL with partial observations reduces communication overhead.
problem Inefficient MARL algorithms in large-scale problems due to state and action information sharing.
method Distributed zeroth-order policy optimization with local policy gradient estimation using consensus.
result The method converges to a policy that is a stationary point of the global objective function.
Algorithm improves reinforcement learning in MDPs with partial order policies.
problem Improving reinforcement learning in MDPs with partial order policies.
method Epoch-based reinforcement learning algorithm leveraging a partial order over policy class.
result Achieves an O ( w log ( ∣ Θ ∣ ) T ) O(\sqrt{w \log(|Θ|) T}) O ( w log ( ∣Θ∣ ) T ) regret bound, independent of state and action space sizes. A new autoregressive model learns the order of graph generation tasks.
problem Generating graphs in a meaningful order when the canonical order is not obvious.
method Introduces a variant of autoregressive models that dynamically decides the autoregressive order based on data.
result Achieves state-of-the-art results on molecular graph generation benchmarks.
A new method solves bilevel optimization problems in competitive Markov games.
problem Capturing competitive structures in RL with multiple interacting policies.
method Penalty-augmented Nikaido-Isoda descent-ascent (PANDA) method.
result PANDA converges to stationary points without convexity assumptions.
Policy certifies inventory levels meeting service requirements.
problem Maintaining stock levels meeting service requirements despite unknown demand.
method Data-driven order policy using online learning and integral action.
result Valid inference method for finite samples.
In this paper, we study reinforcement learning (RL) algorithms to solve real-world decision problems with the objective of maximizing the long-term reward as well as satisfying cumulative constraints. We propose a novel first-order policy optimization method, Interior-point Policy Optimization (IPO), which augments the…
Study shows priority access in ELA auctions is less valuable due to volatility risks.
problem Impact of volatility on time-based transaction ordering policies.
method Single-round model with risk-averse bidders, tested using ETH price data.
result Priority access is discounted due to volatility and risk aversion.
Paper improves COCO problem, reducing constraint violation at the cost of slightly more regret.
problem Online Convex Optimization with adversarial constraints.
method Proposes new policies that trade off regret for reduced constraint violation.
result Achieves i l d e O ( d T + T β ) ilde{O}(\sqrt{dT}+ T^β) i l d e O ( d T + T β ) regret and i l d e O ( d T 1 − β ) ilde{O}(dT^{1-β}) i l d e O ( d T 1 − β ) CCV. Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m ! m! m ! smaller for inputs of m m m objects, improving sample efficiency. ZOSPI improves RL policies with global value function exploitation.
problem Limited sample efficiency of PG algorithms in RL.
method Zeroth-order supervised policy improvement, leveraging global value function estimation.
result ZOSPI achieves competitive results with remarkable sample efficiency.
Study uses SGD to find near-optimal execution cost policies in dynamic markets.
problem Finding optimal execution cost policies in complex markets.
method Stochastic Gradient Descent (SGD) approach to derive near-optimal policies.
result SGD-based policies offer valuable insights and are implementable in volatile markets.
ZSPO optimizes RL from unknown link functions using human feedback.
problem Designing RLHF algorithms for unknown link functions.
method Zero-order policy optimization with human preference feedback.
result ZSPO converges to a stationary policy with a polynomial rate.
Paper develops methods to optimize policies directly from human feedback without reward inference.
problem Challenges in RLHF, including reward model overfitting and distribution shift.
method Develops two algorithms for RLHF without reward inference, using zeroth-order gradient approximators.
result Establishes polynomial convergence rates and outperforms existing methods in numerical experiments.
SymNet uses neural models to solve RMDPs efficiently.
problem Limited effectiveness of generalized policies in RMDPs.
method SymNet trains shared parameters for an RDDL domain, converts instances to graphs, uses relational neural models for node embeddings, and scores actions based on embeddings.
result SymNet policies outperform random and state-of-the-art deep reactive policies on nine RDDL domains.
New method accelerates MBRL by 10x for complex quadrotor tasks.
problem Inefficient training of world models in MBRL for complex robotic tasks.
method Parallelized state-space models and privileged information for training.
result Significant speedup in world model and MBRL training times.