New model-free DR-RL algorithm with finite sample complexity.
problem Limited model-free DR-RL methods with convergence guarantees or sample complexities.
method Integrates Multi-level Monte Carlo (MLMC) technique with threshold mechanism.
result First model-free DR-RL approach with finite sample complexity for total variation and Chi-square divergence.
Two new algorithms improve model-free RL for infinite-horizon MDPs.
problem Learning in infinite-horizon average-reward MDPs.
method Two model-free algorithms for infinite-horizon average-reward MDPs.
result Improved regret bounds for model-free RL in MDPs.
A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.
problem Model-free reinforcement learning for infinite-horizon average-reward MDPs.
method Exploration Enhanced Q-learning (EE-QL) for weakly communicating MDPs.
result Achieves O ( T ) O(\sqrt{T}) O ( T ) regret bound for general weakly communicating MDPs. New hybrid RL algorithm outperforms model-free and model-based methods.
problem Improving reinforcement learning algorithms for MDPs.
method Combines model-free and model-based learning, with a PAC analysis.
result Outperforms both model-free and model-based methods in most cases.
Efficient Q-learning for metric spaces without planning oracles.
problem Efficient reinforcement learning in continuous state-action spaces.
method Model-free Q-learning adapted for metric spaces without a planning oracle.
result Sample efficiency in continuous state-action spaces.
Unified reinforcement learning methods using hybrid inference.
problem Combining model-based and model-free reinforcement learning approaches.
method Control as Hybrid Inference (CHI) framework.
result CHI algorithm balances model-based and model-free learning.
FORK improves model-free reinforcement learning performance.
problem Improving model-free reinforcement learning performance.
method Introducing a new forward-looking Actor (FORK) for Actor-Critic algorithms.
result FORK significantly improves performance in various environments.
New RL algorithms show model-based methods are more efficient than model-free ones in complex decision processes.
problem Efficient reinforcement learning in contextual decision processes with strategic exploration.
method Design of new model-based RL algorithms with sample complexity governed by witness rank.
result Exponential separation between model-based and model-free RL in some rich-observation settings.
New model-free algorithm achieves similar LQR regret guarantees.
problem Model-free control of linear dynamical systems under quadratic costs.
method Online policy gradient scheme with policy space cost analysis.
result Achieves regret scaling with √T, matching model-based methods.
Improved model-free RL algorithm with reduced sample complexity.
problem Learning optimal policies in discounted MDPs efficiently.
method Proposes new algorithms with improved sample complexity bounds.
result Achieves better sample complexity than existing methods.
Optimizes control of noisy discrete systems without system matrix knowledge.
problem Optimal control of discrete-time systems with additive and multiplicative noises.
method Stochastic Lyapunov and Riccati equations, model-free reinforcement learning.
result Model-free reinforcement learning algorithm converges to optimal control policy.
Proposes a model-free control method for chaotic systems using deep Q-learning.
problem Chaos control in systems without precise mathematical models.
method Data-based control policy with two steps: region determination and policy learning.
result Efficient control policy learning in a limited region of state space.
New model-free algorithms learn representations for low-rank MDPs efficiently.
problem Learning representations in reinforcement learning for low-rank MDPs.
method Developed minimax representation learning objective and interleaved with reward-free exploration.
result Proven sample efficiency and scalability to complex environments.
New algorithm borrows future randomness to stabilize model-free control.
problem Double sampling problem in model-free control with nonlinear approximations.
method Borrowing from the future (BFF) algorithm to approximate re-sampling of next states.
result BFF is close to unbiased SGD under smooth dynamics, validated by simulations.
RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.
problem Minimizing regret in episodic MDPs with model-free methods.
method Randomized Q-learning with learning rate randomization.
result RandQL achieves optimal regret bounds in both tabular and metric state-action spaces.
New RL method finds near-optimal policies efficiently.
problem Efficient exploration in model-free RL.
method Modifies Delayed Q-learning with long-term exploration bonus.
result Polynomial time PAC-MDP solution for near-optimal policy finding.
Model-free RL agents can learn effective planning without explicit model.
problem Challenging domains with combinatorial complexity.
method Model-free RL with inductive bias in neural networks.
result Model-free RL can exhibit planning characteristics without explicit model.
DREAM learns optimal strategies in imperfect games without needing a simulator.
problem Learning optimal strategies in imperfect-information games with multiple agents.
method DREAM is a deep reinforcement learning algorithm that converges to Nash Equilibria and coarse correlated equilibria.
result DREAM achieves state-of-the-art performance in benchmark games and is competitive with simulator-based algorithms.
Paper develops efficient algorithms for zero-sum Markov games with general function classes.
problem Challenging settings in zero-sum Markov games with parameterized value functions or models.
method Developed new model-free and model-based algorithms for decoupled and coordinated settings.
result Improved sample complexity and regret bounds for various settings.
Improves sample efficiency in RL by matching model-based gradients.
problem Lack of sample efficiency in model-free RL.
method Gradient matching between model-based and model-free RL components.
result Improves sample efficiency without increasing asymptotic bias.
STEVE improves sample efficiency in reinforcement learning.
problem Combining model-free and model-based reinforcement learning with low sample complexity.
method Dynamic interpolation between model rollouts of various horizon lengths.
result STEVE achieves an order-of-magnitude increase in sample efficiency.
Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. This is especially true with high-capacity parametric function approximators, such as deep networks. In this paper, we study how to bridge th…
Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…
Q-learning achieves optimal regret with UCB exploration.
problem Proving sample efficiency of model-free RL algorithms.
method Q-learning with UCB exploration, proving regret bound.
result Q-learning achieves optimal regret bound of i l d e O ( H 3 S A T ) ilde{O}(\sqrt{H^3 SAT}) i l d e O ( H 3 S A T ) . New algorithms for model selection in off-policy evaluation of reinforcement learning.
problem Hyperparameter tuning for off-policy evaluation methods in reinforcement learning.
method Developed new model-free and model-based selectors with theoretical guarantees and a new experimental protocol.
result New model-free selector, LSTD-Tournament, demonstrates promising empirical performance.
Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of c…
The paper presents a model-free method for stabilizing unknown control systems.
problem Stabilizing unknown control systems in engineering.
method Solving discounted LQR problems with increasing discount factors.
result The method efficiently recovers a stabilizing controller for linear and smooth nonlinear systems.
New algorithm improves sample efficiency for zero-sum Markov games.
problem Improving sample efficiency for model-free algorithms in zero-sum Markov games.
method Proposes a model-free stage-based Q-learning algorithm using variance reduction techniques.
result Achieves optimal sample complexity for finding ε-optimal Nash Equilibrium.
Proposes model-free learning for solving optimization problems with constraints.
problem Hard or impossible to derive expressions for objective function or constraints in optimization problems.
method Uses neural networks to parameterize the function and constraints, trains all networks simultaneously, and connects to reinforcement learning.
result Demonstrates the efficiency of model-free learning in power control problems.
POME combines model-free and model-based methods to improve reinforcement learning performance.
problem Combining model-free and model-based reinforcement learning methods to improve performance.
method POME uses a model-free component for action value estimation and a model-based component for state value prediction, adding the error of these estimations as exploration value.
result POME outperforms PPO on 33 out of 49 Atari 2600 games.
Improved model-free RL from images with stable training.
problem Training RL agents from high-dimensional images is challenging.
method Used variational autoencoders with auxiliary losses to stabilize training.
result Proposed techniques improve sample efficiency and robustness.
Improved model-free reinforcement learning with decision-estimation coefficient.
problem Interactive decision making, including structured bandits and reinforcement learning.
method Combining Estimation-to-Decisions with optimistic estimation to achieve better regret bounds.
result Regret bounds for model-free reinforcement learning with value function approximation.
New method tackles reinforcement learning of complex ω-regular objectives without models.
problem Learning ω-regular objectives in unknown MDPs.
method Constructive reduction to almost-sure reachability, compilation to limit-deterministic Buechi automata.
result Optimal strategies computed from MDP observations using reinforcement learning.
Simple image augmentation boosts deep RL from pixels.
problem Training deep RL models from raw pixel inputs is challenging.
method Applying standard computer vision data augmentation to reinforcement learning.
result Our method significantly improves SAC's performance on DeepMind control suite.
New algorithm learns optimal policies with minimal memory and time.
problem Learning optimal policies in discounted MDPs with short burn-in time.
method Variance reduction and adaptive policy switching.
result First regret-optimal model-free algorithm with low burn-in time.
Simple object representations improve model-free RL performance.
problem Current reinforcement learning agents lack object recognition.
method Used simple, feature-engineered object representations with the Rainbow model.
result Object representations significantly boost performance on Atari games.
MOOSE improves offline RL robustness by using dynamics models.
problem Low robustness of model-free offline RL algorithms in industrial settings.
method MOOSE uses dynamics models to assess policy performance, keeping policies within data support.
result MOOSE outperforms state-of-the-art model-free offline RL algorithms in robust performance.
UCB-Advantage learns MDPs with O ( H 2 S A T ) O(\sqrt{H^2SAT}) O ( H 2 S A T ) regret.
problem Model-free reinforcement learning in finite-horizon MDPs.
method Reference-Advantage decomposition for low regret.
result Achieves i l d e O ( H 2 S A T ) ilde{O}(\sqrt{H^2SAT}) i l d e O ( H 2 S A T ) regret, matching best known bounds. Algorithm learns NE in imperfect information games with imperfect feedback.
problem Learning Nash equilibrium in imperfect information games with bandit feedback.
method IXOMD algorithm for model-free learning with 1 / T 1/\sqrt{T} 1/ T convergence rate. result IXOMD achieves 1 / T 1/\sqrt{T} 1/ T convergence rate to NE. Study shows model-based methods require fewer samples than model-free methods for LQR tasks.
problem Comparing model-based and model-free methods in reinforcement learning for continuous control tasks.
method An asymptotic analysis of sample complexity for policy evaluation in LQR tasks.
result Model-based methods require asymptotically less samples than model-free methods for policy evaluation in LQR tasks.
PicCoLO improves reinforcement learning by correcting model errors.
problem Improving reinforcement learning algorithms with model corrections.
method Predictor-corrector framework that combines model predictions with environment feedback.
result PicCoLO algorithms converge faster and are less biased than previous methods.
Policy Prediction Network improves continuous control problems with model-free and model-based learning.
problem Improving sample complexity and performance in continuous control problems.
method Integrates model-free and model-based reinforcement learning, introduces implicit model-based learning for continuous action space.
result First to introduce implicit model-based learning to Policy Gradient algorithms for continuous action space.
AOP combines model-based planning with model-free learning to handle lifelong learning challenges.
problem Learning control in an online reset-free lifelong learning scenario where mistakes can compound and dynamics change.
method Adaptive Online Planning (AOP) that combines model-based planning with model-free learning, approximating uncertainty to call upon planning only when necessary.
result Achieves strong performance in lifelong learning challenges, gracefully adapting behaviors in the face of unpredictable changes.
New model-free RL algorithm tackles robust average-reward problems with finite sample complexity analysis.
problem Long-term decision-making in environments with varying dynamics.
method Proposes Robust Halpern Iteration (RHI) algorithm based on a black-box sampling oracle and multi-level Monte-Carlo estimator.
result Achieves ε-optimal robust policy with sample complexity of O(1/ε^(2+o(1))) under generative model setting.
New algorithm improves RL performance across different environments.
problem Improving reinforcement learning performance across various environments.
method Designing a fully model-free DRRL algorithm that learns from a single trajectory.
result Demonstrates superior robustness and sample efficiency compared to existing methods.
Study compares model-free valuation to actual financial outcomes, finds it slightly conservative.
problem Evaluating the quality of model-free valuation approaches for financial derivatives.
method Empirical analysis using historical option prices from S&P 500 constituents.
result Model-free valuation approaches are only marginally more conservative than industry-standard models.
Unified framework for policy improvement in RL with benefits in data efficiency and computation.
problem Improving data efficiency and computation in reinforcement learning for continuous control.
method Local, regularized policy improvement with tree search for continuous action spaces.
result Improves data efficiency and reduces wall-clock time in high-dimensional domains.
End-to-end Dyna-AIL combines planning and learning for faster convergence.
problem Efficient imitation learning with minimal interactions.
method Dyna-like framework integrating model-based planning and model-free learning.
result Our approach converges to optimal policies with fewer interactions.