Paper proposes a user model for RL in recommendation systems.
problem Challenges in applying RL to recommendation systems due to unclear reward and dynamics.
method Develops a generative adversarial network to model user behavior and reward function, then uses this model in a novel RL algorithm.
result The RL policy based on the proposed user model leads to better user reward and higher click rates.
Theoretical analysis of DQN and its variants.
problem Understanding the theoretical foundations of deep Q-learning.
method Theoretical analysis of DQN and Minimax-DQN algorithms under mild assumptions.
result Established rates of convergence for action-value functions and provided justifications for DQN techniques.
Langevin DQN achieves deep exploration using Gaussian noise.
problem Deep exploration in reinforcement learning.
method Developed Langevin DQN, a variation of DQN with Gaussian noise.
result Langevin DQN achieves deep exploration.
NROWAN-DQN improves stability and exploration in noisy networks.
problem Noisy networks struggle with stable exploration in complex tasks.
method Noise reduction and online weight adjustment for stable actions.
result NROWAN-DQN outperforms prior algorithms in stability and exploration.
DQN struggles to generalize, but regularization helps.
problem DQN's poor generalization in similar environments.
method Evaluation protocol on Atari games, dropout, and ℓ2 regularization. result Regularization improves DQN's generalization and sample efficiency.
MB-DQN uses different backup lengths for improved reinforcement learning.
problem Improving reinforcement learning with multi-step returns.
method Integrates multi-step returns into bootstrapped DQN with different backup lengths.
result MB-DQN maintains advantages of different backup lengths.
Paper combines DQN and return-based RL for improved policy performance.
problem Improving policy performance in reinforcement learning.
method Integrates DQN and return-based reinforcement learning, introduces two measurements to quantify policy discrepancy.
result The proposed measurements accurately express trace coefficient and improve approximation to return.
Decorrelating features from deep neural networks improves reinforcement learning performance.
problem Learning effective representations for high-dimensional data in reinforcement learning.
method Adding a regularized loss that penalizes correlation in latent features learned by DRL algorithms.
result Our decorrelation method significantly boosts reinforcement learning performance, achieving 40% better results than DQN on 49 Atari games.
DQNs can approximate optimal Q-functions with high accuracy on compact sets.
problem Approximating optimal Q-functions in continuous-time Markov Decision Processes.
method Stochastic control, FBSDEs, residual network approximation theorems, large deviation bounds, viscosity solutions.
result DQNs can approximate optimal Q-functions on compact sets with arbitrary accuracy and high probability.
Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy explorat…
A simple DQN-based multi-agent RL system for binary actions.
problem Complexity and training issues in multi-agent reinforcement learning.
method Shared state and rewards, agent-specific actions, experience replay pool.
result Better performance and faster convergence compared to conventional methods.
RS-DQN protects RL agents from adversarial attacks.
problem Adversarial attacks can disrupt deep RL training and evaluation.
method Online robustness training with RS-DQN combining state-of-the-art adversarial and provably robust training.
result RS-DQN makes RL agents resilient to strong attacks.
This paper improves DQN agents' robustness to adversarial perturbations.
problem Improving DQN agents' robustness to adversarial perturbations.
method Adversarial training and a novel AGE mechanism based on ε-greedy and Boltzmann exploration. result AGE mechanism enhances DQN agents' robustness to adversarial perturbations.
SF-DQN improves RL transfer by learning successor features.
problem Transfer RL with shared dynamics but different reward functions.
method Decomposes Q-function into SF and reward mapping; uses GPI for policy improvement.
result SF-DQN with GPI converges faster and generalizes better than traditional RL methods.
MP-DQN improves deep Q-learning for complex action spaces.
problem Learning with discrete actions and continuous parameters in reinforcement learning.
method Multi-pass deep Q-networks (MP-DQN) to handle parameterised actions.
result Significantly outperforms P-DQN and other methods in data efficiency and policy performance.
Optimized RL algorithms perform well on offline datasets, outperforming fully trained agents.
problem Improving reinforcement learning performance on offline datasets.
method Random Ensemble Mixture (REM) algorithm for Q-learning, trained on DQN replay dataset.
result Offline REM outperforms strong RL baselines and fully trained DQN agent.
Adaptive synchronization improves deep reinforcement learning performance.
problem Fixed step size synchronization can cause loss of properly learned networks.
method Adaptive synchronization based on recent agent behavior.
result Adaptive synchronization leads to better performance in games.
Combines Hebbian and DQN for better POMDP problem solving.
problem Difficult POMDP problems with TD errors.
method Modulated Hebbian plus Q network architecture (MOHQA) integrating Hebbian and DQN.
result Improved DQN performance and outperformed other algorithms on some POMDPs.
Natural gradient improves deep Q-learning performance.
problem Improving deep Q-learning stability and performance.
method Integrates natural-gradient techniques into deep Q-learning.
result Natural-gradient deep Q-learning (NGDQN) outperforms standard DQN without target networks and performs similarly to DQN with target networks.
DQN with model-based exploration improves learning in sparse reward environments.
problem Poor sample efficiency in sparse reward environments.
method Combines model-free and model-based approaches for better exploration.
result Improves performance in environments with sparse rewards.
On many social networking web sites such as Facebook and Twitter, resharing or reposting functionality allows users to share others' content with their own friends or followers. As content is reshared from user to user, large cascades of reshares can form. While a growing body of research has focused on analyzing and c…
DQN optimizes traffic light control policies in ITSs.
problem Challenges in scalable real-time actuation mechanisms for smart traffic management.
method Exploration of Deep Q-Networks (DQN) for traffic light control policies.
result DQN algorithms produce intelligent behavior, such as greenwave patterns.
DQN outperforms traditional stock market strategies by 30%.
problem Optimizing portfolio management in the stock market.
method Deep Q-Network applied to portfolio management, with discretization and neural network enhancements.
result DQN strategy yields 30% higher profit and lower risk compared to traditional strategies.
Instability and variability of Deep Reinforcement Learning (DRL) algorithms tend to adversely affect their performance. Averaged-DQN is a simple extension to the DQN algorithm, based on averaging previously learned Q-values estimates, which leads to a more stable training procedure and improved performance by reducing …
Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN) have achieved state-of-the-art results in a variety of challenging, high-dimensional domains. This success is mainly attributed to the power of deep neural networks to learn rich domain representations for approximating the value function or pol…
ARLD uses active deep Q-learning to learn faster with expert demonstrations.
problem Efforts required to obtain enough expert demonstrations for RL training.
method ARLD framework with Active Deep Q-Network (ADQN) for active demonstration queries.
result ARLD learns faster and reaches super-expert performance with expert demonstrations.
Study shows DQN's performance degrades with temporal dependence in data.
problem Temporal dependence in replayed data affects DQN's performance.
method Modelled τ-mixing data, derived risk bounds, and empirical validation. result Temporal dependence leads to a degradation in DQN's performance rate.
Method predicts diffusion reach probabilities using node embeddings.
problem Estimating diffusion reach probabilities with limited cascades and network information.
method Representation learning on node embeddings for cascade prediction.
result Proposed method outperforms using available cascade data.
This paper optimizes power grid protection settings to maximize network degradation due to cascading attacks.
problem Cascading attacks on power grids and their undetected nature.
method Constrained Bayesian Optimization applied to transmission line protection settings.
result Even limited misconfiguration of protection settings can cause severe cascading attacks.
New method reduces SBL complexity from cubic to linear, improving scalability.
problem Sparse Bayesian Learning's high computational complexity for large feature spaces.
method DQN-SBL, a diagonal Quasi-Newton method for SBL.
result DQN-SBL achieves competitive generalization with sparse models, scaling well to large-scale problems.
Model infers diffusion networks from heterogeneous cascade data.
problem Understanding and predicting diffusion processes in interconnected populations.
method Double mixture directed graph model with layer-specific constraints.
result Convex formulation allows for statistical and computational guarantees.
New insights into cascade feedback linearization of control systems.
problem Obtaining a cascade feedback linearization for invariant control systems.
method Introducing truncated versions of operators from the calculus of variations to prove new theorems.
result Established new geometry and foundational theorems for future work.
A new algorithm SRG-DQN reduces variance in deep Q-learning.
problem Inaccurate estimation of anchor points in SVRG for deep Q-learning.
method Introduces recursive gradient variance reduction for stochastic gradient updates.
result Demonstrates improved efficiency and effectiveness of SRG-DQN on reinforcement learning tasks.
The paper improves theoretical guarantees for Thompson Sampling in cascading bandits.
problem Optimizing online recommender systems with cascading bandits.
method Develops and analyzes new Thompson Sampling algorithms for cascading bandits.
result Establishes the first theoretical guarantees on Thompson Sampling for cascading bandits.
New research reveals diverse cascade sizes in finite networks, challenging traditional risk assessments.
problem Predicting the size of cascades in finite networks is difficult due to uncertain parameters and missing information.
method Derived explicit closed-form solutions for cascade size distributions in complete and star networks.
result Broad and even bimodal cascade size distributions in finite networks, not centered around the average.
Modeling cascading behavior in complex systems using CTBNs.
problem Understanding which states trigger cascading events in complex systems.
method Continuous-time Bayesian networks (CTBNs) for modeling and identifying likely sentry states.
result Identification of likely sentry states that may lead to cascading behavior.
Deep learning agent improves pedestrian navigation in urban environments.
problem Autonomous driving among pedestrians in urban areas.
method Multi-objective deep reinforcement learning using a deep Q-learning variant.
result The multi-objective DQN agent outperforms single-objective DQN in various environments.
Improved stability in DQN using neural networks with modifications.
problem Stability issues in DQN with neural networks.
method Simple modifications to neural network structure.
result Improved stability in DQN learning.
Improved reinforcement learning with deep learning.
problem Extending MultiGrid Reinforcement Learning to work with deep learning.
method Combining potential-based reward shaping with a learned potential function from interaction, and adapting it for deep learning algorithms.
result DQN augmented with the approach performs significantly better on continuous control tasks.
Study on information cascade fragility under mismatched revealing probabilities.
problem Analyzing the fragility of information cascades in decision-making processes with imperfect knowledge of revealing probabilities.
method Examined sequential decision-making models with players having private information and imitating previous decisions. Studied the effect of a mismatch between players' beliefs and actual revealing probabilities.
result Derived closed-form expressions for optimal learning rates and identified phase transitions in the behavior of asymptotic learning rates.
A search engine usually outputs a list of K web pages. The user examines this list, from the first web page to the last, and chooses the first attractive page. This model of user behavior is known as the cascade model. In this paper, we propose cascading bandits, a learning variant of the cascade model where the obje…
Adaptive traffic control uses deep RL to improve decision-making.
problem Improving traffic control using deep RL.
method Integrates recent deep RL techniques into a novel DQN-based algorithm (TC-DQN+) for traffic control.
result Proposes a new reward function for traffic control.
Paper uses RL to mitigate cascading failures in power systems.
problem Mitigating multi-stage cascading failures in power grids.
method Reinforcement Learning applied to DC-OPF for power flow optimization.
result Reduced system collapse rates through RL-based optimization.
Random play trains a DQN to win at Sungka.
problem Optimizing game strategies through self-play.
method Training a DQN agent with random play.
result DQN trained with random play converges fast and consistently wins.
The paper introduces a new reinforcement learning model that improves sample efficiency and performance.
problem Improving sample efficiency and performance in reinforcement learning.
method The paper proposes a transcoder network that includes model learning in the DQN loss function, providing a richer training signal.
result The proposed method leads to superior results on Atari games compared to vanilla DQN.
Structured prediction tasks pose a fundamental trade-off between the need for model complexity to increase predictive power and the limited computational resources for inference in the exponentially-sized output spaces such models require. We formulate and develop the Structured Prediction Cascade architecture: a seque…
DBS improves convergence in reinforcement learning.
problem Softmax operator convergence issues in reinforcement learning.
method Dynamic Boltzmann Softmax (DBS) updates value function.
result DBS enables better value function estimation and convergence.
Proposes a deep RL framework for optimizing task allocation in crowdsourcing platforms.
problem Optimizing task allocation in crowdsourcing platforms for both workers and requesters.
method Uses Deep Q-Network (DQN) to estimate long-term rewards, considering immediate and future rewards simultaneously.
result Demonstrates superior performance in synthetic and real datasets.