HPSDE learns hierarchical policies from multi-modal rewards.
problem Learning optimal policies from a multi-modal reward function.
method Density estimation with return-weighted importance sampling for identifying modes.
result HPSDE successfully learns option policies corresponding to modes of the return function.
Unified framework integrates symbolic planning and HRL for robust decision-making.
problem Combining reinforcement learning and symbolic planning for robust decision-making in dynamic environments.
method Integrates symbolic planning with hierarchical reinforcement learning to guide task execution and improve planning.
result Unified framework leads to rapid policy search and robust symbolic plans in complex domains.
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
Robotic table tennis learns efficient policies to return balls at 100Hz.
problem Developing efficient robotic table tennis strategies.
method Model-free reinforcement learning using evolutionary search on CNN-based policies.
result Robots can develop multi-modal styles (forehand and backhand) with 80% return rate.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
The paper introduces an adjacency constraint to improve goal-conditioned HRL.
problem Training inefficiency in goal-conditioned HRL due to large action space.
method Restricting the high-level action space to a k-step adjacent region of the current state.
result The adjacency constraint preserves optimal hierarchical policies and improves HRL performance.
A new HRL method learns hierarchical policies using mutual information maximization.
problem Learning hierarchical policies in reinforcement learning for structured tasks.
method Mutual information maximization for latent variable learning, advantage-weighted importance sampling for option policies, deterministic policy gradient for optimization.
result Enhanced performance in continuous control tasks through learned hierarchical policies.
PGS uses neural networks to improve policies online without search trees.
problem Limited scalability of Monte Carlo Tree Search (MCTS) for high branching factor games.
method Adapts a neural network simulation policy via policy gradient updates, avoiding search trees.
result PGS achieves comparable performance to MCTS and defeats strong Hex agents.
The paper introduces a method to optimize decision-making with computational constraints.
problem Optimizing decision-making under computational limitations.
method Introduces a free-energy principle that considers both value and computational cost.
result Solves the variational equations using DP and rate distortion theory.
MSOL learns hierarchical policies for multitask tasks with soft options.
problem Training hierarchical policies for multiple tasks with stability and flexibility.
method MSOL uses separate variational posteriors for each task, regularized by a shared prior, to avoid instabilities and fine-tune options for new tasks.
result MSOL significantly outperforms hierarchical and flat transfer-learning baselines.
The paper tackles hierarchical reinforcement learning by approximating optimal solutions for the Traveling Salesman Problem.
problem Approximating optimal solutions for the Traveling Salesman Problem using hierarchical reinforcement learning.
method Mapping the problem into a Reward Discounted Traveling Salesman Problem and deriving approximate solutions using local policies.
result Three stochastic policies are proposed that guarantee better performance than any deterministic policy.
Hierarchical NMF organizes COVID-19 literature into a searchable tree.
problem Organizing and searching for relevant COVID-19 research.
method Hierarchical nonnegative matrix factorization.
result Identified 8 major and 52 subtopics related to COVID-19.
New A* algorithm improves hierarchical clustering quality.
problem Improving hierarchical clustering quality in large search spaces.
method Combining A* search with a trellis data structure.
result Achieves higher quality results than baselines in particle physics and other benchmarks.
Develops a new method for optimizing policies in hierarchical models.
problem Optimizing complex policies in hierarchical models.
method Applies second-order methods in the space of state-action paths.
result The natural path gradient method can be computed exactly and reflects state-space hierarchy.
Model learns sub-goals and low-level policies for hierarchical reinforcement learning.
problem Determining appropriate low-level policies in hierarchical reinforcement learning.
method Unsupervised learning scheme based on asymmetric self-play.
result Obtains performance gains over non-hierarchical approaches.
A new framework scales active search for large datasets.
problem Scaling active search for large, high-dimensional data sets.
method Hierarchical Batch Bandit Search (HBBS) framework.
result HBBS improves performance and scalability for batch search.
Proposes a model for multi-agent reinforcement learning with hierarchical graph attention network.
problem Limited transferability of trained policies to new multi-agent tasks.
method Uses hierarchical graph attention network for representation learning and multi-agent actor-critic for policy learning.
result Demonstrates superior performance in mixed cooperative and competitive tasks compared to existing methods.
HiPPO adapts skills and higher-level policies together for better transfer in hierarchical RL.
problem Sub-optimality in skill transfer when lower-level skills are fixed.
method HiPPO: a novel hierarchical policy gradient method that trains all levels of the hierarchy jointly.
result Improved robustness of skills to environment changes through training time-abstractions.
Trust-region methods and natural gradients are equivalent in certain policy search scenarios.
problem Improving policy search methods in continuous control tasks.
method Introducing compatible policy search (COPOS) that uses natural parameterization and compatible value function approximation to control entropy loss.
result COPOS yields state-of-the-art results in challenging tasks and reduces entropy loss.
Study on kernels for optimizing functions with hierarchical variables.
problem Optimizing functions with hierarchical variables requires efficient surrogate models.
method Investigate and propose alternative kernels for hierarchical variables in surrogate models.
result Different kernels and assumptions affect model quality and search performance.
Bayesian approach for policy search in stochastic domains.
problem Policy search in stochastic domains.
method Nested probabilistic programs, Lightweight Metropolis-Hastings (LMH) adaptation.
result Similar quality policies learned with simpler algorithm.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
New algorithm learns sub-task policies from unsegmented demonstrations.
problem Challenges in learning hierarchical policies from unsegmented demonstrations.
method Generative adversarial imitation learning framework with directed information maximization.
result Automatic learning of sub-task policies from unsegmented demonstrations.
New hierarchical search algorithm improves neural architecture design across different operator sets.
problem DARTS's performance drops when search space changes due to operator correlation and optimization complexity.
method Operator clustering and optimization complexity matching in a hierarchical search algorithm.
result The algorithm consistently finds high-performance architectures across various search spaces, outperforming other methods.
Bayesian optimization for contextual policy search improves robot learning.
problem Scalable robot learning with limited data.
method Factored contextual representation using target and environment contexts.
result Experience can be generalized over target contexts, leading to faster learning and better generalization.
HM-NAS improves neural architecture search by learning optimal architectures.
problem Limited flexibility in architecture candidates due to hand-designed heuristics.
method Incorporates multi-level encoding and hierarchical masking to automatically learn optimal architectures.
result Achieves better architecture search performance and competitive model accuracy.
New method learns search policies by inspecting and improving past roll-outs.
problem Learning good search policies for complex combinatorial spaces.
method Retrospective imitation learning, improving policy through past roll-outs.
result Policy can iteratively scale up to larger problems.
Beam search policies learned via imitation learning.
problem Beam search policies are not explicitly learned by models during training.
method Developed a meta-algorithm for learning beam search policies using imitation learning.
result Showed no-regret guarantees for learning beam search policies.
Efficient neural architecture search discovers top-performing models.
problem Finding optimal neural architectures efficiently.
method Hierarchical genetic representation and expressive search space.
result Discovered architectures outperform manually designed models.
Enhances RL performance with a population-guided parallel learning scheme.
problem Improving off-policy reinforcement learning performance.
method Population-guided parallel learning scheme with shared experience replay buffer and soft policy update.
result Monotone improvement of the expected cumulative return proved theoretically and demonstrated in practice.
A framework uses POMDPs to assess hierarchical clustering quality.
problem No agreed methodology for evaluating hierarchical clustering without ground-truth labels.
method Modelled as a POMDP, assessing search support for hierarchical structures.
result Proposes a novel measure for assessing hierarchical clustering quality.
CF-GPS learns policies from logged data by considering counterfactual outcomes.
problem Learning policies from limited real experience in complex environments.
method Assumes logged real experience and models counterfactual outcomes. Uses structural causal models for evaluation.
result Improves policy evaluation and search results on a grid-world task.
Automates organizing diverse web data into a hierarchical topic model.
problem Manual classification of all scientific and popular scientific knowledge is impractical.
method Proposes an algorithm to aggregate multiple collections into a single hierarchical topic model.
result Demonstrates a web service for topical exploratory search.
Variational inference improves hierarchical imitation learning of control programs.
problem Learning structured control policies from demonstrations.
method Variational inference for discovering hierarchical structure in observation-action traces.
result Variational inference leads to more efficient and generalized control policies.
New approach learns walk and trot gaits from simulated quadruped using strategic exploration.
problem Learning symmetric gaits (walk and trot) from high-dimensional action spaces.
method Introduced symmetry properties into initial covariance of Gaussian search distribution for strategic exploration. Used episode-based likelihood ratio policy gradient and relative entropy policy search.
result Significant performance enhancement in learning walk and trot gaits compared to random gaits.
Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…
MTL-NAS combines NAS with GP-MTL for task-agnostic multi-task learning.
problem Designing architectures for diverse tasks with varying priors.
method Disentangled GP-MTL networks, hierarchical feature sharing, and gradient-based search.
result General-purpose model trained once can adapt to multiple tasks.
EffAcTS uses active learning to select model parameters for robust policy search.
problem Learning robust policies that perform well in unseen environments.
method EffAcTS is an active learning framework that selects model parameters using Linear Bandits to minimize data collection.
result Efficiency gains in sample collection and improved policy performance.
Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference. This is unsatisfactory in many applications where the reference policy is suboptimal and the goal of learning is to improve upon it. Ca…
This paper explores how hierarchical agent policies affect exploration in goal-driven navigation environments.
problem Understanding how hierarchical agent policies influence exploration in goal-driven navigation.
method Design of EscapeRoom environments, measuring complexity with hitting times of dependency graphs, evaluating PPO and hierarchical PPO.
result Analytically estimated hitting time in goal dependency graphs is a metric of environment complexity and hierarchical approaches are necessary for complex environments.
Paper shows MCTS approximates policy optimization, proposing an improved variant.
problem Improving AI performance through better MCTS algorithms.
method Shows MCTS approximates policy optimization problem, proposes a new algorithm.
result Proposed algorithm reliably outperforms original AlphaZero in multiple domains.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
problem Improving predictive performance through test-time data augmentation.
method Greedy policy search (GPS) for learning test-time augmentation policies.
result Augmentation policies learned with GPS achieve superior predictive performance and robustness.
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
A new framework for personalized ad retrieval in e-commerce search.
problem Difficulty in measuring ads retrieved using multiple signals (e.g. user profiles, clicks).
method Employing historical click data to initialize a hierarchical network representing signals, keys, and ads. Training a model to learn weights of edges and selecting the best edges.
result Framework achieves good performance, improving RPM/CTR.
Survey on algorithms for quick robot learning.
problem Efficiently learn robot controllers with limited data.
method Leverage prior knowledge and data-driven models.
result Combining prior knowledge and surrogate models improves learning.
Introduces GAMPS for better model-based policy learning.
problem Misspecified model classes lead to poor policy estimates.
method Exploits current policy to learn approximate transition model, focusing on relevant parts of the environment.
result Empirically validated GAMPS on benchmark domains, demonstrating improved properties.
Efficient neural architecture search by sampling structure and operations.
problem Efficiently searching for optimal neural architectures.
method Decouples structure and operation search, using reinforcement learning with policy vectors.
result Significantly improved efficiency compared to traditional methods.
A new reinforcement learning approach using competitive primitives that specialize and specialize based on information needs.
problem Complex environments require efficient and specialized decision-making.
method Decomposes policy into competitive primitives that decide based on information needs, regularized to use minimal information.
result Improves generalization over flat and hierarchical policies.