Paper solves pendulum swing-up problem using RL.
problem Solving the classic pendulum swing-up problem.
method Deep Deterministic Policy Gradient algorithm applied to continuous action domain.
result Optimal pendulum achieved with increasing average return and decreasing loss.
The paper introduces MDP homomorphic networks for faster reinforcement learning.
problem Current reinforcement learning approaches do not exploit symmetries in the joint state-action space.
method Equivariant neural networks with group-structured symmetries (reflections, rotations).
result MDP homomorphic networks converge faster than unstructured baselines on various tasks.
Photonic quantum reinforcement learning for control problems.
problem Solving continuous control problems with noisy quantum computers.
method Proximal policy optimization for photonic variational quantum agents.
result Photonic policy learning achieves comparable performance to classical neural networks.
Adapts model-based advice to stabilize black-box policies for nonlinear control.
problem Stabilizing machine-learned policies for nonlinear control with limited model information.
method Proposes an adaptive λ-confident policy to combine black-box and model-based advice. result Proves the stability of the adaptive λ-confident policy and its competitive ratio. A lightweight FPGA-based reinforcement learning approach for edge devices.
problem Resource constraints and inefficiency of DQN on edge devices.
method OS-ELM based training algorithm and L2 regularization for stability.
result 29.77x and 89.40x faster than conventional DQN-based approach for CartPole-v0 task.
A simple DQN-based multi-agent RL system for binary actions.
problem Complexity and training issues in multi-agent reinforcement learning.
method Shared state and rewards, agent-specific actions, experience replay pool.
result Better performance and faster convergence compared to conventional methods.
This paper detects Markov violations in RL with noise, improving policy development.
problem Partial observability and sensor/actuator noise invalidate Markovian assumptions in RL.
method Combines PCMCI causal discovery with Markov Violation score (MVS).
result Even substantial noise doesn't always disrupt multi-step dependencies.
WAPPO optimizes feature distributions for better visual transfer in RL.
problem Improving visual transfer in reinforcement learning.
method WAPPO uses Wasserstein Confusion to minimize feature distribution distance.
result WAPPO outperforms previous methods in visual transfer across different environments.
Designing optimal controllers continues to be challenging as systems are becoming complex and are inherently nonlinear. The principal advantage of reinforcement learning (RL) is its ability to learn from the interaction with the environment and provide optimal control strategy. In this paper, RL is explored in the cont…
We present a data-efficient reinforcement learning algorithm resistant to observation noise. Our method extends the highly data-efficient PILCO algorithm (Deisenroth & Rasmussen, 2011) into partially observed Markov decision processes (POMDPs) by considering the filtering process during policy evaluation. PILCO conduct…
This paper proposes a novel scheme for the watermarking of Deep Reinforcement Learning (DRL) policies. This scheme provides a mechanism for the integration of a unique identifier within the policy in the form of its response to a designated sequence of state transitions, while incurring minimal impact on the nominal pe…
This paper investigates the resilience and robustness of Deep Reinforcement Learning (DRL) policies to adversarial perturbations in the state space. We first present an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DR…
Conservative Policy Iteration (CPI) is a founding algorithm of Approximate Dynamic Programming (ADP). Its core principle is to stabilize greediness through stochastic mixtures of consecutive policies. It comes with strong theoretical guarantees, and inspired approaches in deep Reinforcement Learning (RL). However, CPI …
We adapt the ideas underlying the success of Deep Q-Learning to the continuous action domain. We present an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. Using the same learning algorithm, network architecture and hyper-parameters, our algo…
WMPG reduces policy gradient variance using world models.
problem Reducing variance in policy gradient estimates.
method Trains a world model online to estimate policy gradients and uses imagined trajectories as a baseline.
result WMPG achieves better sample efficiency compared to AC and MAC.
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by usin…
Deep reinforcement learning has led to several recent breakthroughs, though the learned policies are often based on black-box neural networks. This makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned polici…
Bayesian optimization has demonstrated impressive success in finding the optimum input x* and output f* = f(x*) = max f(x) of a black-box function f. In some applications, however, the optimum output f* is known in advance and the goal is to find the corresponding optimum input x*. In this paper, we consider a new sett…
D2D-SPL uses discrete states and a classifier to train RL faster.
problem Training neural networks in RL due to correlated samples.
method Discretizes state space, uses actor-critic, selects input/target pairs, trains classifier.
result Trains faster than state-of-the-art methods.
Quantum variational circuits improve reinforcement learning efficiency.
problem Improving reinforcement learning algorithms using quantum computing.
method Investigation of quantum variational circuits for DQN and Double DQN, encoding classical data for quantum circuits.
result Quantum variational circuits can solve reinforcement learning tasks with a smaller parameter space.
New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.
problem Applying standard STDP to discrete action reinforcement learning tasks.
method Feedback-modulated TD-STDP learning rule for spiking neuron networks.
result Feedback modulation improves credit assignment in reinforcement learning.
Bayesian optimization outperforms other methods in hyperparameter tuning for reinforcement learning.
problem Finding optimal hyperparameters that generalize across random seeds in reinforcement learning.
method Benchmarked Successive Halving, Random Search, and Bayesian Optimization with and without repetitions on PPO2 algorithms for Cartpole and Inverted Pendulum tasks.
result Bayesian optimization with noise robust acquisition function is the best choice.
LM optimization outperforms other methods in deep learning tasks but at high computational cost.
problem Finding efficient optimization methods for deep learning models.
method Comparing first-order (CG, SGD, LM, L-BFGS) and higher-order optimization functions.
result Levemberg-Marquardt (LM) optimization significantly improves convergence but at a high computational cost.
AdaRL adapts quickly to new environments with minimal data.
problem Quickly adapting to new environments in reinforcement learning.
method AdaRL uses a parsimonious graphical representation to encode changes across domains.
result AdaRL can efficiently adapt policies to target domains with few samples.
CARL safely adapts RL agents for safety-critical tasks.
problem Safety hazards in RL for safety-critical tasks.
method CARL combines model-based RL and cautious adaptation.
result CARL achieves higher rewards with fewer failures in safety-critical tasks.
This project proposes using reinforcement learning to train spiking neural networks.
problem Training spiking neural networks using traditional methods is challenging due to the discrete nature of spikes.
method The project investigates two approaches: 1) treating each neuron as an RL agent, 2) applying the reparameterization trick.
result The project demonstrates that reinforcement learning can be applied to train spiking neural networks.
BCPO optimizes offline RL policies by converting uncertainty into conservative bounds.
problem Offline RL's fragility under distribution shifts and model errors.
method Bayesian approach with credible lower bounds and KL regularization.
result BCPO yields an uncertainty-calibrated policy that avoids exploiting model errors.
Paper introduces MVS to detect non-Markovian observations in reinforcement learning.
problem Real-world sensors violate Markov property, leading to suboptimal reinforcement learning performance.
method Uses prediction-based Markov Violation Score (MVS) combining random forest and ridge regression.
result MVS detects non-Markovian structure in observation trajectories, quantifying its impact.