Proposes a new Deep Recurrent Double Q-Learning model for Atari games.
problem Improving Deep Reinforcement Learning models for Atari games.
method Integrates Double Q-Learning and Recurrent Networks (LSTM, DRQN) for Atari games.
result Demonstrates improved performance in Atari games compared to existing models.
New dropout technique reduces variance and overestimation in deep Q-Learning.
problem Reduction of variance and overestimation in deep Q-Learning.
method Using Dropout techniques to reduce variance and overestimation in deep Q-Learning.
result Demonstrated effectiveness in enhancing stability and reducing both variance and overestimation.
Deep Q-learning generates directed acyclic graphs.
problem Generating DAGs with specified structures.
method Deep reinforcement learning, specifically deep Q-learning.
result Demonstrated capability of generating DAGs in sparse reward environments.
A novel Q-learning variant reduces underestimation bias in deep actor-critic methods for reinforcement learning.
problem Underestimation bias in deep actor-critic methods for reinforcement learning.
method Introduces a parameter-free Q-learning variant that combines maximum and minimum operators to bound value estimates.
result Improves state-of-the-art performance on OpenAI Gym tasks.
Improved deep Q-learning with SVRG reduces variance and stabilizes training.
problem Excessive variance in gradient estimation hinders deep Q-learning performance.
method Utilized stochastic variance reduced gradient (SVRG) techniques.
result Significantly improved performance on 18 out of 20 Atari games compared to baseline methods.
Theoretical analysis improves understanding of Deep Q-Learning's behavior.
problem Lack of formal guarantees and gaps between theory and practice of Deep Q-Learning.
method Dynamical systems perspective, focusing on realistic assumptions.
result Proves convergence of Deep Q-Learning under specific conditions.
Proposes a deep spectral Q-learning for mobile health data.
problem Personalized treatment assignment for patients with time-varying covariates.
method Integrates PCA with deep Q-learning for mixed frequency data.
result Mean return converges to optimal under estimated optimal policy.
Deep Q-learning methods are sensitive to time discretization, which this paper addresses.
problem Sensitivity of Deep Q-learning methods to time discretization in near continuous-time environments.
method Identified and formalized the problem of sensitivity to time discretization. Developed a principled off-policy RL algorithm.
result Proved that Q-learning does not exist in continuous time and developed a robust algorithm.
EBQL reduces bias in Q-learning for improved performance.
problem Over- and under-estimation biases in Q-learning degrade performance.
method Ensemble Bootstrapping to reduce both over- and under-estimation biases.
result EBQL outperforms other Q-learning methods in Atari games.
Reverse Experience Replay improves Deep Q-learning for sparse rewards.
problem Sparse rewards and reward-maximizing tasks in Deep Q-learning.
method Sampling transitions in reverse order for training.
result Significantly increased performance in tasks with limited experience and memory capacity.
Natural gradient improves deep Q-learning performance.
problem Improving deep Q-learning stability and performance.
method Integrates natural-gradient techniques into deep Q-learning.
result Natural-gradient deep Q-learning (NGDQN) outperforms standard DQN without target networks and performs similarly to DQN with target networks.
Paper analyzes Q-learning with neural networks, proving a fast convergence rate.
problem Analyzing the convergence rate of neural Q-learning.
method Finite-time analysis of neural Q-learning with a deep ReLU network.
result Neural Q-learning converges to optimal policy with O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) rate. Paper automates car negotiation in intersections using Q-learning.
problem Automated vehicles negotiate with human-driven cars in intersections.
method Deep Q-learning applied to simulated traffic with various driver behaviors.
result 98% success rate in avoiding collisions with other vehicles.
PQ-learning improves Q-learning by periodically updating target estimates.
problem Improving sample complexity in Q-learning for finding optimal policies.
method Maintains two Q-value estimates, one online and one target, updated periodically.
result PQ-learning achieves better sample complexity for finding epsilon-optimal policies.
Investigates Q-learning bottlenecks with function approximation and sampling methods.
problem Understanding and mitigating issues in Q-learning with function approximation.
method Unit testing framework with oracles to disentangle sources of error; novel sampling method based on function approximation error.
result Large neural networks improve learning stability and offer practical compensations for overfitting.
Deep Q-learning analyzes EEG for drowsiness during driving tests.
problem Estimating drowsiness during driving to improve safety.
method Adapting Q-learning to EEG data for drowsiness estimation.
result Trained model accurately tracks mind state variations in EEG data.
A new algorithm SRG-DQN reduces variance in deep Q-learning.
problem Inaccurate estimation of anchor points in SVRG for deep Q-learning.
method Introduces recursive gradient variance reduction for stochastic gradient updates.
result Demonstrates improved efficiency and effectiveness of SRG-DQN on reinforcement learning tasks.
ConQUR tackles delusional bias in deep Q-learning, improving performance in Atari games.
problem Delusional bias in deep Q-learning.
method Efficient methods to mitigate delusional bias by training Q-approximators with consistent labels and a search framework.
result Improves performance in Atari games, sometimes dramatically.
Proposes an interpretable neural network for Q-learning.
problem Limited theoretical or empirical studies on understanding deep Q-learning models.
method Key-value memories, attention, and reconstructible embeddings for global explanation.
result Comparable training rewards to state-of-the-art models but overfitting to training data.
A new algorithm for deep Q-learning with robustness to state transition uncertainty.
problem Model uncertainty in state transitions for non-tabular, continuous state spaces.
method Distributionally robust approach using worst-case transition ball and dualized Bellman operator with Sinkhorn distance.
result Optimal policy found through solving non-linear Bellman equation with neural network parameterization.
Deep Q-Learning method for Nash equilibria in stochastic games.
problem Model-free learning for multi-agent stochastic games, especially for general-sum games.
method Data-efficient Deep-Q-learning using local linear-quadratic expansion parametrized by deep neural networks.
result The algorithm learns optimal actions for stochastic games without experiencing all state-action pairs.
Deep Q-learning optimizes same-day delivery with vehicles and drones.
problem Optimizing same-day delivery with limited vehicle and drone capacities.
method Deep Q-learning approach to assign packages to vehicles or drones.
result Deep Q-learning policy outperforms benchmark policies and maintains effectiveness with changing fleet sizes.
New target-based TD learning algorithms improve deep Q-learning convergence.
problem Improving convergence of deep Q-learning algorithms.
method Introducing averaging TD, double TD, and periodic TD algorithms.
result Established asymptotic convergence analyses for averaging TD and double TD, and finite sample analysis for periodic TD.
Soft Q-learning improves sample efficiency in robotic manipulation.
problem Limited interaction time in real-world robotic tasks.
method Soft Q-learning for maximum entropy policies, with composability.
result Soft Q-learning policies are more sample efficient and can be composed.
Deep Reinforcement Learning improves with Weighted Q-Learning to reduce bias and uncertainty.
problem Overestimation and high variance in Q-Learning cause learning algorithms to diverge in complex environments.
method Deep Weighted Q-Learning (Deep WQL) uses Dropout and Monte Carlo sampling to approximate WQL's weights and reduce bias.
result Deep WQL reduces bias and improves performance on benchmarks compared to existing methods.
FSQ algorithm extends Q-learning to continuous actions with linear complexity.
problem Extending Q-learning to continuous action spaces with linear complexity.
method Discretization of the action space to maintain linear complexity.
result FSQ algorithm achieves linear complexity in the discretized problem.
Improved Q-learning for robot control with faster data-efficiency.
problem Poor data-efficiency and stochasticity in deep Q-learning.
method Composite Q-learning: Decomposes Q-function into short- and long-term predictions.
result Significant learning speedup and improved data-efficiency in robot tasks.
Deep Q-Learning model outperforms traditional methods in stock trading.
problem Optimal trade execution in stock markets.
method Model-free approach using Double Deep Q-Learning with neural networks.
result Model outperforms standard benchmark on most stocks.
A deep Q-learning method improves downlink power allocation in multi-cell networks.
problem Optimal resource allocation in dense, heterogeneous wireless networks is computationally challenging.
method Proposes a centralized DRL-based downlink power allocation scheme using deep Q-learning.
result The proposed DRL-based power allocation scheme outperforms conventional methods in multi-cell scenarios.
Paper presents a new method for Bayesian deep learning that scales to Atari games.
problem Training neural networks on complex environments like Atari games is challenging.
method Adapted temporal difference Q-learning to work with Bayesian inference.
result TAGI allows for analytical inference of neural network parameters, achieving performance comparable to gradient-based methods.
A controller learns to control a nonlinear plant with unknown model and partial observation using continuous deep Q-learning.
problem Designing a controller for a nonlinear plant with unknown model and partial sensor observation under network delays.
method Continuous deep Q-learning applied to an extended state including past control inputs and outputs.
result The controller can learn a robust control policy to network delays with partial sensor observation.
Online random forests improve Q-learning performance in specific gym environments.
problem Improving Q-learning performance in reinforcement learning tasks.
method Proposed online random forests as Q-function approximators and growing them as learning progresses.
result Improved performance over state-of-the-art Deep Q-Networks in specific gym environments.
Kernelized Q-learning achieves good performance with minimal data.
problem Efficient Q-learning in high-dimensional spaces.
method Kernelized Q-learning framework with effective dimensionality.
result Concrete regret bounds for linear and Gaussian RBF kernels.
New algorithms speed up inverse reinforcement learning by solving MDPs once.
problem Slow convergence in Maximum Entropy Inverse Reinforcement Learning.
method Deep Inverse Q-learning with constraints exploiting Q-learning.
result Up to several orders of magnitude speedup compared to existing methods.
Deep Q-Learning optimizes market making by balancing price risk and spread profits.
problem Optimizing liquidity provision in financial markets.
method Reinforcement Learning applied to a market making problem with a reward function.
result Deep Q-Learning algorithms can recover the optimal market making strategy.
Tabular Q-learning outperforms advanced RL methods in monetary policy.
problem Dynamic setting of short-term interest rates to stabilize inflation and unemployment under uncertain macroeconomic conditions.
method Discrete-action Markov Decision Process with tabular Q-learning, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning, POMDP formulations.
result Standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return) compared to advanced RL methods and traditional policy rules.
Stochastic Q-learning tackles large action spaces with reduced computation.
problem Effective decision-making in complex environments with large discrete action spaces.
method Stochastic value-based RL approaches that consider a sublinear number of actions in each iteration.
result Stochastic Q-learning achieves near-optimal returns with significantly reduced computation time.
A Deep Q-Learning framework tackles market-making by incorporating closing auctions.
problem Managing end-of-day risk in market-making models.
method Developed a Deep Q-Learning framework that anticipates closing auctions and continuously refines projected clearing prices.
result The Deep Q-Learning framework outperforms classical market-making models in simulations and real data.
Paper addresses underestimation bias in double Q-learning, proposing a method to improve learning performance.
problem Underestimation bias in double Q-learning leading to non-optimal fixed points.
method Proposes a simple approach using approximate dynamic programming to bound the target value.
result Significant improvement in learning performance over baseline algorithms in Atari benchmark tasks.
Study Whittle index learning algorithms for restless bandits with constant stepsizes.
problem Optimizing decisions in restless multi-armed bandits with constant stepsizes.
method Developed Q-learning algorithms with constant stepsizes for index learning in restless bandits, extending to DQN and function approximations.
result The algorithms learn the Whittle index effectively.
The paper investigates the effectiveness of reusing experience in Deep Q-Learning for FPS environments.
problem The high number of interactions required for reinforcement learning limits its practicality.
method The authors test the effectiveness of applying learning update steps multiple times per environmental step in the VizDoom environment.
result Updating learning steps less frequently than every 4th environmental step does not improve performance and can degrade performance.
GAN Q-learning uses GANs for distributional RL in tabular and gym environments.
problem Complex MDPs in nonlinear function approximation.
method Generative adversarial networks (GANs) for distributional reinforcement learning.
result Empirically shows GAN Q-learning is a viable alternative to traditional methods.
A deep Q-learning strategy optimizes portfolio trading efficiency.
problem Optimizing dynamic portfolio allocation schemes.
method Formulated a Markov decision process model with deep Q-learning for discrete combinatorial actions.
result Outperforms benchmark strategies in real-world trading simulations.
A new Q-learning variant reduces underestimation bias in deep reinforcement learning.
problem Underestimation bias in deep reinforcement learning policies.
method Introducing a novel, parameter-free Deep Q-learning variant.
result Significantly outperforms existing approaches and improves state-of-the-art performance.
The paper formalizes and analyzes multi-agent Q-learning with value factorization.
problem Understanding and improving the convergence of multi-agent Q-learning with value factorization.
method Formalized a multi-agent fitted Q-iteration framework for analyzing factorized multi-agent Q-learning.
result Multi-agent Q-learning with linear value factorization can converge under certain conditions.
New method reduces overestimation in actor-critic reinforcement learning.
problem Function approximation errors in actor-critic methods lead to suboptimal policies.
method Proposes novel mechanisms to minimize overestimation, including using the minimum value between critics and delaying policy updates.
result Outperforms state-of-the-art methods on OpenAI gym tasks.
New AI algorithm improves multi-layer optical film design efficiency.
problem Traditional algorithms converge to local optima, limiting global optimal solutions.
method Deep Q-learning for global optimal multi-layer optical film design.
result Deep Q-learning model converges global optimum of optical thin film structure.
We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems. Our agents explore via Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop neural network. Our algorithm learns much faster than common exploration strategies such as …