End-to-end Dyna-AIL combines planning and learning for faster convergence.
problem Efficient imitation learning with minimal interactions.
method Dyna-like framework integrating model-based planning and model-free learning.
result Our approach converges to optimal policies with fewer interactions.
Deep belief networks improve Dyna-style planning in large state spaces.
problem Lack of real data and difficulty in learning a good generative model for large state spaces.
method Used deep belief networks to learn an environment model for Dyna-style planning.
result Deep belief networks significantly outperform linear expectation models in empirical validation.
The paper addresses Dyna-style RL's value hallucination issue by proposing a new algorithm.
problem Value hallucination in Dyna-style RL due to bootstrapping simulated states.
method Introduces a new Dyna algorithm using predecessor models with multi-step updates.
result Evidence supports the Hallucinated Value Hypothesis (HVH), suggesting predecessor models with multi-step updates are promising.
New algorithms accelerate value function convergence in MDPs.
problem Accelerating convergence of value functions in Markov Decision Processes (MDPs).
method Operator Splitting Value Iteration (OS-VI) and OS-Dyna.
result Achieves much faster convergence rate with accurate models.
A new search-control strategy improves Dyna's efficiency.
problem Improving sample efficiency in model-based reinforcement learning.
method Proposes a novel search-control strategy by sampling high frequency regions of the value function.
result Empirically shows that high frequency regions require more samples to approximate, suggesting a better search-control strategy.
Selective planning with imperfect models reduces harmful effects of model inadequacy.
problem Harmful effects of using an imperfect model in reinforcement learning.
method Selective planning with heteroscedastic regression to estimate predictive uncertainty from model inadequacy.
result Effective selective planning requires considering both parameter uncertainty and model inadequacy.
Self-consistent models improve reinforcement learning by aligning predictions with future values.
problem Improving reinforcement learning by aligning model predictions with future values.
method Proposes multiple self-consistency updates to encourage a learned model and value function to be consistent with each other.
result Self-consistency helps both policy evaluation and control in both tabular and function approximation settings.
Proposes using Hill Climbing on value estimates for search-control in Dyna to improve sample efficiency.
problem Improving sample efficiency in model-based reinforcement learning.
method Proposes using Hill Climbing on current value estimates for search-control in Dyna, deriving a noisy projected natural gradient algorithm.
result HC-Dyna can obtain significant sample efficiency improvements on four classical domains.
Climate-Dyna learns residual climate HVA for XVAs using model-based RL.
problem Tackles residual climate HVA in XVAs not inferred from stand-alone stress loss.
method Combines residual climate HVA with model-based reinforcement learning.
result Reduces climate charge from 1.517 to 0.831 in EU ETS study.
The study questions the necessity of accurate models in MBRL.
problem The difficulty of achieving and utilizing accurate models in complex or changing environments.
method Introduces a meta-learning algorithm focused on model usefulness rather than accuracy.
result Meta-learning models enable faster learning than accurate models built with domain-specific knowledge.
Hybrid approach improves reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Approximate model from Model-Free RL, use Mean First Passage Times for reachability, design new algorithms.
result Hybrid approaches converge with fewer iterations and samples.
DynAE improves deep clustering by dynamically shifting from reconstruction to centroid construction.
problem Lack of clear cost functions in unsupervised learning for capturing variations and similarities.
method Dynamic Autoencoder (DynAE) that gradually eliminates reconstruction in favor of centroid construction.
result DynAE achieves state-of-the-art results in deep clustering compared to other methods.
This paper develops source traces for faster TD learning.
problem Improving temporal difference learning speed and generalization.
method Introduces source traces as a backward view of successor representations, enabling TD errors to be propagated to potential causal states.
result Demonstrates faster generalization and improved performance of source traces compared to previous methods.
Dyna optimizes momentum for stochastic optimization of neural networks.
problem Optimizing neural networks with momentum for stochastic optimization.
method Introduces fictitious mass to regularize adaptive stepsize in momentum gradient descent.
result Promises improved performance and convergence in preliminary trials.
A framework schedules hyperparameters for model-based reinforcement learning, improving performance.
problem Inadequate scheduling of hyperparameters in model-based reinforcement learning.
method Theoretical analysis and AutoMBPO framework to automatically schedule real data ratio and other hyperparameters.
result Training with hyperparameters scheduled by AutoMBPO significantly improves performance.
New hybrid RL algorithm outperforms model-free and model-based methods.
problem Improving reinforcement learning algorithms for MDPs.
method Combines model-free and model-based learning, with a PAC analysis.
result Outperforms both model-free and model-based methods in most cases.
DPN combines model-based and model-free reinforcement learning for efficient planning.
problem Efficiently plan actions in reinforcement learning environments.
method Combines model-based and model-free reinforcement learning, dynamically constructing plans using a learned state-transition model.
result Reduces the number of state transitions during planning by up to 96%, improving data efficiency and performance.
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
New method designs fairer transport plans with uncertainty.
problem Designing fair and balanced mass transport plans.
method Hierarchical fully probabilistic design (HFPD) for transport plans.
result Optimal hyperprior for transport plans with uncertain marginals.
MPNet uses neural networks for efficient motion planning.
problem Exponential increase in computational complexity with motion planning problem dimensionality.
method MPNet encodes workspaces from point cloud measurements and generates collision-free paths.
result MPNet is computationally efficient and generalizes to unseen environments.
Study on human planning and re-planning in unknown stochastic environments.
problem Understanding how humans adjust plans in unfamiliar environments.
method Grid world task, 12 different models, model-based reinforcement learning approach.
result Model-based reinforcement learning approach best explains human re-planning behavior.
Unified framework integrates symbolic planning and HRL for robust decision-making.
problem Combining reinforcement learning and symbolic planning for robust decision-making in dynamic environments.
method Integrates symbolic planning with hierarchical reinforcement learning to guide task execution and improve planning.
result Unified framework leads to rapid policy search and robust symbolic plans in complex domains.
We aim to reduce the burden of programming and deploying autonomous systems to work in concert with people in time-critical domains, such as military field operations and disaster response. Deployment plans for these operations are frequently negotiated on-the-fly by teams of human planners. A human operator then trans…
Generative model learns goal-directed visual plans from raw data.
problem Learning goal-directed visual plans from high-dimensional observations.
method Combines representation learning and planning, maximizing mutual information.
result Generative model learns low-dimensional representation explaining causal data.
First model-based planner learns to construct, evaluate, and execute plans.
problem Challenges in model-based planning, especially in constructing plans.
method Imagination-based Planner that learns to construct, evaluate, and execute plans.
result Can learn to solve challenging continuous control problems and elaborate planning strategies.
Generative adversarial networks improve radiation therapy planning.
problem Creating optimal radiation therapy plans.
method Using a GAN to predict 3D dose distributions directly.
result Significantly outperforms previous methods in clinical satisfaction and metrics.
Study integrates reliability constraints into generation planning models.
problem Challenges in integrating reliability constraints with generation planning models.
method Leverages a weighted oblique decision tree (WODT) technique to embed reliability verification constraints.
result Demonstrates effectiveness in achieving reliable and optimal planning solutions.
Information planning speeds up text data learning with fewer examples.
problem Learning with limited training examples for text data.
method Entropy and mutual information-based information planning for three supervised models.
result Information planning accelerates learning for text data.
New approach improves black-box planning efficiency by discovering focused macros.
problem Difficulty of deterministic planning increases exponentially with depth.
method Discovering macro-actions with focused effects to improve goal-count heuristics.
result Focused macros dramatically improve black-box planning efficiency.
UPNs embed planning within a goal-directed policy for effective visuomotor control.
problem Learning abstract representations for visuomotor control and generalization.
method Differentiable planning within a latent space, gradient descent trajectory optimization, end-to-end learning of representations.
result UPNs can transfer visuomotor planning strategies across robots with different morphologies and actuation capabilities.
Study motion planning for points avoiding obstacles in a plane.
problem Avoiding collisions for multiple points in a plane with unknown obstacles.
method Algebraic and topological tools for motion planning.
result New topological complexity for planar motion planning.
CoMPNetX uses neural networks to efficiently solve constrained motion planning problems.
problem Finding collision-free paths on constraint manifolds efficiently.
method Neural generator and discriminator with neural gradients-based projection operator.
result CoMPNetX finds path solutions with high success rates and lower computation times.
This article asks how planning scholarship may effectively gain impact in planning practice through media exposure. In liberal democracies the public sphere is dominated by mass media. Therefore, working with such media is a prerequisite for effective public impact of planning research. Using the example of megaproject…
New method combines heuristics and search techniques to speed up cooperative planning for autonomous vehicles.
problem Efficient cooperative planning for autonomous vehicles in complex traffic scenarios.
method Combining learned heuristics with Monte Carlo Tree Search (MCTS) to guide search towards promising actions.
result Better solutions at lower computational costs achieved through accelerated planning.
This paper presents a unifying framework for reinforcement learning and planning.
problem Sequential decision making in AI, formalized as MDP optimization.
method A unifying algorithmic framework (FRAP) for reinforcement learning and planning.
result Identifies common dimensions in MDP planning and learning algorithms.
A planning approach learns skills from interactions, balancing exploration and exploitation.
problem Learning robust high-level skills in noisy environments with unknown pre-conditions.
method Formulates skills as high-level policies, learns plans via bandit problems, balances exploration and exploitation.
result A planner capable of learning robust high-level skills in high-dimensional state spaces.
Paper evaluates how forecast errors affect optimal utilisation in production planning.
problem Forecast errors impact optimal utilisation in production planning.
method Simulation and mixed integer programming for stochastic demand.
result Forecast errors significantly affect optimal costs in production planning.
Survey of integrating planning and learning in model-based reinforcement learning.
problem Sequential decision making in AI, formalized as MDP optimization.
method Systematic coverage of dynamics model learning and planning-learning integration.
result Broad conceptual overview of model-based reinforcement learning.
New method uses autoregressive models for dynamic planning.
problem Planning in dynamic environments with moving obstacles and goals.
method Conditional autoregressive generative models in a discrete latent space.
result Method nearly matches true environment performance for planning.
New approach for obstacle avoidance in robotics using learned representations.
problem Challenges in sensor-based motion planning for new and dynamic environments.
method Proposes a new obstacle representation using PointNet architecture trained jointly with policies for obstacle avoidance.
result Significant improvements in accuracy and efficiency compared to state of the art.
This work clarifies the role of inference types in planning.
problem Lack of consistency in using inference types for planning.
method Variational framework and loopy belief propagation.
result All inference types correspond to different weights in variational problems.
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.
Improved motion planning for dynamic environments using RL.
problem Efficient motion planning in changing environments.
method Deep reinforcement learning (DDPG) tailored for motion planning.
result DDPG-MP significantly improves motion planning accuracy.
PlaNet learns latent dynamics from images for better planning in unknown environments.
problem Leveraging planning in unknown environments with accurate dynamics models.
method Deep Planning Network (PlaNet) learns dynamics from images using latent space and multi-step variational inference.
result PlaNet achieves high performance in continuous control tasks with contact dynamics and sparse rewards.
This work defines a complexity measure for BAMDP planning and introduces state abstraction for more efficient approximate planning.
problem The computational intractability of exact BAMDP planning solutions.
method Define a complexity measure for BAMDP planning, introduce state abstraction, and develop an approximate planning algorithm.
result Introduces a computationally tractable approximate planning algorithm using state abstraction.
Model-free RL agents can learn effective planning without explicit model.
problem Challenging domains with combinatorial complexity.
method Model-free RL with inductive bias in neural networks.
result Model-free RL can exhibit planning characteristics without explicit model.
Hierarchical Foresight improves robot vision tasks by planning long-term goals.
problem Compounding uncertainty and scalability issues in long horizon video prediction.
method Subgoal generation and planning using hierarchical visual foresight (HVF).
result Achieves nearly 200% performance improvement in vision-based manipulation tasks.
PALM learns abstract models for efficient planning and task transfer.
problem Efficiently learning and transferring hierarchical models for planning.
method PALM uses a new formal structure (L-AMDP) to learn independent, modular models at multiple levels of abstraction.
result PALM integrates planning and execution, facilitating rapid learning of abstract models.