Optimizes antenna tilt for better QoS in cellular networks.
problem Hard to learn optimal antenna tilt policies in real networks due to risk and simulation gap.
method Uses off-policy Contextual Multi-Armed-Bandit (CMAB) techniques to learn from existing data.
result Trained policies show consistent improvements over existing logging policies.
New reinforcement learning algorithms improve policy optimization with entropy regularization.
problem Improving policy optimization in reinforcement learning.
method Soft policy gradient theorem (SPGT) and new policy optimization algorithms.
result New algorithms outperform prior works on various benchmark tasks.
Neural networks help prove PPO and TRPO can find global policy.
problem Proving global convergence of PPO and TRPO with neural networks.
method Proved convergence of neural network-based PPO and TRPO to globally optimal policy.
result Proved sublinear convergence rate to globally optimal policy.
Softmax policy gradient achieves global optimality in wide neural networks with entropy regularization.
problem Optimizing softmax policies with neural networks in the mean-field regime.
method Modeling neural networks as Wasserstein gradient flows and proving global optimality of fixed points.
result Global optimality of softmax policy gradient in wide single hidden layer neural networks with entropy regularization.
We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions that occur as a result of an underlying network structure. In our model, a bipartite graph captures the relationship between actions and a common set of unknowns such that choosing an action reveals observations…
This study analyzes economic policy uncertainty indices using visibility graphs.
problem Understanding the role of economic policy uncertainty in global economies.
method Visibility graph algorithm applied to economic policy uncertainty indices.
result The economic policy uncertainty indices exhibit persistent behavior and scale-free networks.
PPG separates policy and value function training phases for better reinforcement learning efficiency.
problem Challenges in traditional reinforcement learning methods for policy and value function optimization.
method Integrates Phasic Policy Gradient framework that splits policy and value function training into distinct phases.
result Significantly improves sample efficiency on Procgen Benchmark compared to PPO.
Policy Prediction Network improves continuous control problems with model-free and model-based learning.
problem Improving sample complexity and performance in continuous control problems.
method Integrates model-free and model-based reinforcement learning, introduces implicit model-based learning for continuous action space.
result First to introduce implicit model-based learning to Policy Gradient algorithms for continuous action space.
The study compares reinforcement learning models and finds model-based approaches superior for complex MDPs.
problem Complexity of optimal Q-functions and policies in MDPs exceeds dynamics, hindering model-free methods.
method Theoretical analysis and empirical testing of neural network expressivity for policies, Q-functions, and dynamics.
result Model-based planning yields better policies for complex MDPs, improving performance on MuJoCo tasks.
This paper addresses the question of how a previously available control policy π s π_s π s can be used as a supervisor to more quickly and safely train a new learned control policy π L π_L π L for a robot. A weighted average of the supervisor and learned policies is used during trials, with a heavier weight initially on the superv…
Paper analyzes neural network complexity for planning problems.
problem Understanding neural network complexity for planning policies.
method Circuit complexity analysis for relational neural networks.
result Three classes of planning problems identified based on network complexity.
We introduce Recurrent Predictive State Policy (RPSP) networks, a recurrent architecture that brings insights from predictive state representations to reinforcement learning in partially observable environments. Predictive state policy networks consist of a recursive filter, which keeps track of a belief about the stat…
Proposes a model for multi-agent reinforcement learning with hierarchical graph attention network.
problem Limited transferability of trained policies to new multi-agent tasks.
method Uses hierarchical graph attention network for representation learning and multi-agent actor-critic for policy learning.
result Demonstrates superior performance in mixed cooperative and competitive tasks compared to existing methods.
New method learns policies without limiting to Gaussian distributions.
problem Limitations of Gaussian parameterization in policy learning.
method Advantage weighted quantile regression for implicit policy modeling.
result Comparable or superior performance on MuJoCo benchmarks.
This paper introduces a new method to evaluate multiple policies simultaneously.
problem Estimating the value of many policies for a single set of states.
method Developed a scalable, differentiable fingerprinting mechanism to represent complex policies.
result The method can produce policies that outperform those that generated the training data, in zero-shot manner.
NPMD uses CNNs to optimize policies on low-dimensional manifolds, reducing sample complexity.
problem Explaining the effectiveness of deep policy gradient methods in high-dimensional RL.
method Neural policy mirror descent (NPMD) with CNNs, considering state spaces as low-dimensional manifolds.
result NPMD finds ε-optimal policies with O(ε^(-d/α-2)) samples, leveraging low-dimensional structure.
We present a reinforcement learning framework, called Programmatically Interpretable Reinforcement Learning (PIRL), that is designed to generate interpretable and verifiable agent policies. Unlike the popular Deep Reinforcement Learning (DRL) paradigm, which represents policies by neural networks, PIRL represents polic…
RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.
problem Efficient reinforcement learning in environments with continuous state and action spaces.
method RANDPOL uses randomized function approximation to represent policy and value functions, providing finite time guarantees and improved numerical performance.
result RANDPOL achieves better numerical performance and provides finite time guarantees compared to deep neural network based algorithms.
While bigger and deeper neural network architectures continue to advance the state-of-the-art for many computer vision tasks, real-world adoption of these networks is impeded by hardware and speed constraints. Conventional model compression methods attempt to address this problem by modifying the architecture manually …
Deep neural networks can estimate Q-values efficiently on low-dimensional state-action spaces.
problem Estimating the performance of a reinforcement learning policy using data from a different policy.
method Deep fitted Q-evaluation method leveraging manifold structure and convolutional neural networks.
result Sharp error bound for fitted Q-evaluation depends on intrinsic dimension and function space mismatch.
POPLIN improves model-based planning in complex environments.
problem Efficient planning in complex high-dimensional environments.
method Combines policy networks with online planning, optimizing parameters directly.
result POPLIN achieves state-of-the-art performance in MuJoCo benchmarks, 3x more sample efficient.
Policy-GNN optimizes GNN aggregation for diverse node iterations.
problem Optimizing GNN performance by varying aggregation iterations for different nodes.
method Policy-GNN uses a meta-policy framework with deep reinforcement learning to adaptively determine the number of aggregations for each node.
result Policy-GNN significantly outperforms state-of-the-art alternatives on real-world datasets.
New approach to understand recurrent policies as FSMs without minimization.
problem Minimization of FSMs obscures the semantics of policy decisions.
method Start with unminimized FSM, apply interpretable reductions, use attention tool.
result Reveals insights into policy decisions not previously noticed.
This paper studies learning rate policies for deep neural networks, offering metrics and tools for better tuning.
problem Effective tuning of learning rates for deep neural networks is challenging and crucial for achieving high accuracy.
method Comprehensive study of 13 learning rate functions, proposing metrics for evaluation, and developing LRBench for benchmarking and selection.
result Identification of good learning rate policies with effective ranges and step sizes for various LR update schedules.
This work learns exploration policies for unknown distributions using samples and policy gradients.
problem Learning exploration policies for unknown distributions in Bayesian bandits.
method Meta-learning approach parameterizing policies in a differentiable way and optimizing them with policy gradients.
result Effective gradient estimators and variance reduction techniques are derived.
Paper analyzes NAC with neural networks for efficient policy optimization.
problem Improving sample and iteration complexity in policy optimization.
method Entropy regularization, averaging, neural network approximation, and optimization techniques.
result Entropy regularization and averaging ensure stability and sharp sample complexity bounds.
Optimizes wireless network resource management with state-augmented policies.
problem Optimizing network-wide utility with user performance constraints.
method State-augmented parameterization of RRM policy, using dual variables.
result Superior trade-off between mean, minimum, and 5th percentile rates.
Paper certifies neural network control policies against persistent adversarial perturbations.
problem Neural networks' fragility to adversarial perturbations in control systems.
method Combining neural network certification tools with robust control theory.
result Certifies neural network policies in a control loop under l-infinity norm bounded adversarial perturbations.
This work develops agents to learn generalizable policies for dynamic network environments.
problem Real-world network topologies change due to attackers, defenders, or system failures, leading to failures in adaptive ACD systems.
method Developing agents to learn generalizable policies across dynamic network environments.
result Agents can learn robust policies for dynamic network topologies and diverse attackers.
Optimizes insurance pricing by accounting for policyholders' price sensitivity.
problem Traditional insurance pricing does not consider policyholders' price sensitivity.
method Formulates insurance pricing as a decision-making problem and uses off-policy evaluation and stochastic control.
result Neural networks outperform existing techniques for policy optimization.
Approach to optimize bidding policies offline using reinforcement learning.
problem Optimizing spending in online advertising under budget constraints.
method Offline reinforcement learning for optimizing differentiable base policies.
result Statistically significant performance gains in production bidding environments.
Proposes DGCN with trajectory sampling for data-efficient policy search in MBRL.
problem Improving data efficiency in model-based reinforcement learning.
method Combines trajectory sampling and DGCN for uncertainty propagation in probabilistic world models.
result Improves sample-efficiency over other uncertainty propagation methods and probabilistic models.
Efficient policy learning from observational data using weighted classification reductions.
problem Efficient policy evaluation does not necessarily lead to efficient estimation of policy parameters.
method Proposed an estimation approach based on generalized method of moments, efficient for policy parameters.
result Demonstrated empirical efficiency and regret benefits of a proposed method.
New algorithm optimizes interventions under network interference, scaling to large networks.
problem Optimal policy learning under network interference where one individual's treatment affects others.
method Developed a scalable Thompson sampling algorithm for dynamic networks.
result Proved a Bayesian regret bound that is sublinear in network size and rounds.
Solves POMDPs with recurrent neural networks and natural policy gradient.
problem Non-stationarity in optimal policies of POMDPs.
method Integrates recurrent neural networks into natural policy gradient and temporal difference learning.
result Non-asymptotic theoretical guarantees for global optimality up to function approximation.
Develops neural network framework for risk-reward optimization problems.
problem Multi-period risk-reward optimization with constrained policies.
method Neural network framework with two coupled feedforward networks, parametrizing two-step policies.
result Empirical optimum converges to true optimal value as network capacity and training size increase.
Improves RL algorithms with two techniques.
problem Enhance off-policy RL performance.
method Formulates RL as proximal point iteration; uses value functions for improved action value estimate.
result Significant performance improvement on RL benchmarks.
Revel tackles safe exploration in RL with verified symbolic policies.
problem Computational infeasibility of verifying neural networks in RL learning loops.
method Two policy classes: neurosymbolic with approximate gradients and symbolic policies for efficient verification. Mirror descent over policies to safely update and project policies.
result Revel discovers policies that outperform prior approaches to verified exploration.
Study how untrained policies explore in RL environments.
problem Challenges in reinforcement learning, especially sparse or adversarial reward structures.
method Theoretical and empirical analysis of untrained deep neural policies in a toy model.
result Untrained policies generate correlated actions and non-trivial state-visitation distributions.
Off-policy learning exhibits greater instability when compared to on-policy learning in reinforcement learning (RL). The difference in probability distribution between the target policy ( π π π ) and the behavior policy (b) is a major cause of instability. High variance also originates from distributional mismatch. The var…
In this work, we consider the problem of estimating a behaviour policy for use in Off-Policy Policy Evaluation (OPE) when the true behaviour policy is unknown. Via a series of empirical studies, we demonstrate how accurate OPE is strongly dependent on the calibration of estimated behaviour policy models: how precisely …
Optimal vehicle repositioning policy found for shared mobility services.
problem Matching fixed supply with spatial customer demand under uncertain and correlated demand.
method Base-stock repositioning policy, asymptotic optimality, regret analysis, adaptive repositioning algorithm.
result Surrogate Optimization and Adaptive Repositioning algorithm achieves optimal regret of O ( n 2.5 T ) O(n^{2.5} \sqrt{T}) O ( n 2.5 T ) . Lo-Hp decouples weight generation into local and global policies to improve flexibility and efficiency.
problem Over-coupling and long-horizon issues in current optimization methods.
method Hybrid-Policy Sub-Trajectory Balance objective.
result Learning local optimization policies addresses long-horizon issues and enhances global weight generation.
This paper identifies drift Lipschitz budget K as key to diffusion policy expressivity and statistical trade-offs.
problem Understanding and maximizing the expressivity of diffusion policies while managing statistical limitations.
method Identifying drift Lipschitz budget K as central, quantifying expressivity and statistical behavior, proving lower bounds, and providing practical implementation guidelines.
result Balancing expressivity and statistical complexity yields a finite-sample performance gap, with rates depending on sample size and drift type.
Novel framework learns policies for MPC with near-optimal guarantees.
problem Safe and near-optimal policy learning for MPC.
method Primal-dual neural networks for supervised learning of policies and certificate policies.
result Guaranteed feasibility and optimality of learned policies.
We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidati…
Reinforcement learning (RL) has advanced greatly in the past few years with the employment of effective deep neural networks (DNNs) on the policy networks. With the great effectiveness came serious vulnerability issues with DNNs that small adversarial perturbations on the input can change the output of the network. Sev…
Graph neural networks optimize radio resource management policies for wireless networks.
problem Optimizing user selection and power control in wireless networks with fairness constraints.
method Formulated as a Lagrangian dual problem, RRM policies are parameterized by a GNN architecture trained on channel conditions.
result The method achieves superior tradeoff between average and 5th percentile rates, demonstrating fairness.