EffAcTS uses active learning to select model parameters for robust policy search.
problem Learning robust policies that perform well in unseen environments.
method EffAcTS is an active learning framework that selects model parameters using Linear Bandits to minimize data collection.
result Efficiency gains in sample collection and improved policy performance.
Robust optimization and statistical robustness improve robot navigation policies.
problem Efficiently finding optimal robot navigation policies in uncertain environments.
method Combining robust optimization and statistical robustness with improved Bayesian optimization techniques.
result Safe and repeatable robot navigation policies are achieved with improved robust optimization methods.
New framework improves sample efficiency and robustness in RL with smooth policies.
problem Sample inefficiency and lack of robustness in deep reinforcement learning.
method SR^2L framework, smoothness-inducing regularization.
result Improved sample efficiency and robustness in both on-policy and off-policy RL algorithms.
Bayesian approach for policy search in stochastic domains.
problem Policy search in stochastic domains.
method Nested probabilistic programs, Lightweight Metropolis-Hastings (LMH) adaptation.
result Similar quality policies learned with simpler algorithm.
FiDi-RL combines deep RL with FiDi policy search for efficient continuous control.
problem Challenges in continuous control tasks using reinforcement learning.
method Combines Deep Deterministic Policy Gradients (DDPG) with Augment Random Search (ARS) using FiDi policy search.
result Improves data efficiency and stability of ARS.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
problem Improving predictive performance through test-time data augmentation.
method Greedy policy search (GPS) for learning test-time augmentation policies.
result Augmentation policies learned with GPS achieve superior predictive performance and robustness.
Proposes DGCN with trajectory sampling for data-efficient policy search in MBRL.
problem Improving data efficiency in model-based reinforcement learning.
method Combines trajectory sampling and DGCN for uncertainty propagation in probabilistic world models.
result Improves sample-efficiency over other uncertainty propagation methods and probabilistic models.
Paper learns domain randomization distributions for robust robot policies.
problem Finding good domain randomization parameters for simulation without real data.
method Gradient-based search methods to learn domain randomization distribution.
result Improvements in jump-start and asymptotic performance when transferring policies.
PIPPS solves deep learning's exploding gradient problem by reparameterization gradients.
problem Exploding gradients in deep learning and model-based RL.
method Develops PIPPS framework, a flexible policy search method robust to chaos-like gradients.
result PIPPS improves over reparameterization gradients by up to 10^6 times.
Bayesian regularization improves policy performance in noisy MDPs.
problem Suboptimal policies from estimated model parameters.
method Bayesian regularization of MDP objective function with prior information.
result Regularized policies show better robustness against model noise.
PGS uses neural networks to improve policies online without search trees.
problem Limited scalability of Monte Carlo Tree Search (MCTS) for high branching factor games.
method Adapts a neural network simulation policy via policy gradient updates, avoiding search trees.
result PGS achieves comparable performance to MCTS and defeats strong Hex agents.
The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recent…
A new estimator reduces bias and variance in ranking policy evaluation.
problem Estimating ranking policies using logged data in recommender systems.
method Cascade Doubly Robust estimator based on the cascade assumption.
result The estimator reduces bias and variance compared to existing methods.
DR-MCTS improves decision quality and sample efficiency in complex environments.
problem Improving decision quality and sample efficiency in complex environments.
method Integrates Doubly Robust off-policy estimation into Monte Carlo Tree Search (MCTS).
result DR-MCTS achieves superior performance in Tic-Tac-Toe and VirtualHome tasks.
Trust-region methods and natural gradients are equivalent in certain policy search scenarios.
problem Improving policy search methods in continuous control tasks.
method Introducing compatible policy search (COPOS) that uses natural parameterization and compatible value function approximation to control entropy loss.
result COPOS yields state-of-the-art results in challenging tasks and reduces entropy loss.
Prognosticator improves performance in non-stationary MDPs.
problem Real-world applications often violate the stationary assumption in reinforcement learning.
method A policy gradient algorithm that maximizes a forecast of future performance.
result Prognosticator outperforms online adaptation techniques in non-stationary MDPs.
MOOSE improves offline RL robustness by using dynamics models.
problem Low robustness of model-free offline RL algorithms in industrial settings.
method MOOSE uses dynamics models to assess policy performance, keeping policies within data support.
result MOOSE outperforms state-of-the-art model-free offline RL algorithms in robust performance.
Bayesian optimization for contextual policy search improves robot learning.
problem Scalable robot learning with limited data.
method Factored contextual representation using target and environment contexts.
result Experience can be generalized over target contexts, leading to faster learning and better generalization.
Beam search policies learned via imitation learning.
problem Beam search policies are not explicitly learned by models during training.
method Developed a meta-algorithm for learning beam search policies using imitation learning.
result Showed no-regret guarantees for learning beam search policies.
Enhances RL performance with a population-guided parallel learning scheme.
problem Improving off-policy reinforcement learning performance.
method Population-guided parallel learning scheme with shared experience replay buffer and soft policy update.
result Monotone improvement of the expected cumulative return proved theoretically and demonstrated in practice.
CF-GPS learns policies from logged data by considering counterfactual outcomes.
problem Learning policies from limited real experience in complex environments.
method Assumes logged real experience and models counterfactual outcomes. Uses structural causal models for evaluation.
result Improves policy evaluation and search results on a grid-world task.
New approach learns walk and trot gaits from simulated quadruped using strategic exploration.
problem Learning symmetric gaits (walk and trot) from high-dimensional action spaces.
method Introduced symmetry properties into initial covariance of Gaussian search distribution for strategic exploration. Used episode-based likelihood ratio policy gradient and relative entropy policy search.
result Significant performance enhancement in learning walk and trot gaits compared to random gaits.
Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…
Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference. This is unsatisfactory in many applications where the reference policy is suboptimal and the goal of learning is to improve upon it. Ca…
Paper certifies neural network control policies against persistent adversarial perturbations.
problem Neural networks' fragility to adversarial perturbations in control systems.
method Combining neural network certification tools with robust control theory.
result Certifies neural network policies in a control loop under l-infinity norm bounded adversarial perturbations.
Paper shows MCTS approximates policy optimization, proposing an improved variant.
problem Improving AI performance through better MCTS algorithms.
method Shows MCTS approximates policy optimization problem, proposes a new algorithm.
result Proposed algorithm reliably outperforms original AlphaZero in multiple domains.
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
Survey on algorithms for quick robot learning.
problem Efficiently learn robot controllers with limited data.
method Leverage prior knowledge and data-driven models.
result Combining prior knowledge and surrogate models improves learning.
Introduces GAMPS for better model-based policy learning.
problem Misspecified model classes lead to poor policy estimates.
method Exploits current policy to learn approximate transition model, focusing on relevant parts of the environment.
result Empirically validated GAMPS on benchmark domains, demonstrating improved properties.
Efficient neural architecture search by sampling structure and operations.
problem Efficiently searching for optimal neural architectures.
method Decouples structure and operation search, using reinforcement learning with policy vectors.
result Significantly improved efficiency compared to traditional methods.
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
Early stopping method saves up to 75% computation time in policy search tasks.
problem Lengthy evaluation times in optimization problems, especially in robotics.
method A generalized early stopping criterion that only uses objective value at each time step.
result The method saves up to 75% computation time compared to no stopping.
Bayesian optimization improves policy search in reinforcement learning.
problem Finding optimal policies with high variance estimates from random samples.
method Develops an algorithm combining Bayesian optimization and policy gradients.
result Improves sample complexity and reduces variance in empirical evaluations.
Paper proposes an efficient bandit-based algorithm for hyperparameter optimization.
problem Efficiently evaluating hyperparameters in deep learning models with large search spaces.
method Sub-Sampling (SS) algorithm combined with Bayesian Optimization (BOSS).
result Theoretical proof of optimality and empirical validation of superior performance.
InfoTree improves reinforcement learning by optimizing tool use with a greedy submodular approach.
problem Maximizing information from tool use in reinforcement learning with limited resources.
method Formalizes Rollout Informativeness, recasts state selection as submodular maximization, and uses UUCB and ABA.
result InfoTree outperforms existing methods across various benchmarks, improving performance by 18.2% on average.
This paper improves self-play learning in games by manipulating experience distributions.
problem Improving self-play learning in games through better experience sampling.
method Three approaches: weighted sampling, Prioritized Experience Replay, and diversifying trajectories.
result Major improvements in early training performance in some games, minor improvements overall.
Transformer learns to search through reinforcement learning, mimicking DFS.
problem Understanding how transformers learn search capabilities in RL.
method Two-head transformer, depth-wise curriculum, discounted returns.
result Transformer policy generalizes depth and prioritizes high-probability branches.
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
problem Reward-free learning in high-dimensional, continuous-control domains.
method Maximum Entropy POLicy optimization (MEPOL) algorithm that maximizes a non-parametric state entropy estimate.
result MEPOL learns a maximum-entropy exploration policy that facilitates learning various reward-based tasks.
Contextual policy search allows adapting robotic movement primitives to different situations. For instance, a locomotion primitive might be adapted to different terrain inclinations or desired walking speeds. Such an adaptation is often achievable by modifying a small number of hyperparameters. However, learning, when …
Efficiently find near-optimal medical treatments with less trial and error.
problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.
Genie optimizes search marketplaces by estimating policy impacts without risky experiments.
problem Optimizing search marketplaces with frequent policy changes and limited randomized experiments.
method Genie uses an open box simulation engine and click calibration model to estimate KPI impacts.
result Genie outperforms existing approaches in optimizing Bing Ads Marketplace.
Paper develops efficient nonmyopic active search methods for drug and materials discovery.
problem Efficiently identifying many members of a class in high-throughput screening.
method Bayesian decision framework, nonmyopic approximations, efficient computation.
result Proposed policy outperforms sequential and batch settings in drug and materials discovery.
Improves sample efficiency in evolutionary policy search methods.
problem Worse sample efficiency in evolutionary policy search methods.
method Importance mixing to reuse more samples.
result Improves sample efficiency but still less than deep reinforcement learning.
Study improves policy search in continuous control by using heavy-tailed distributions.
problem Challenges in continuous space policy search due to non-convexity and myopic-farsighted incentives.
method Introduced heavy-tailed policy parameterizations and analyzed convergence rates and stability.
result Convergence rate to stationarity depends on policy's tail index and exploration tolerance.
Maximizes robustness in Bayesian experimental design under model uncertainty.
problem Brittleness of Bayesian experimental design under model misspecification.
method Formulates as a max--min game, uses Sibson's α-MI, and adopts PAC-Bayes framework.
result Establishes robust belief update and conditional information gain measure.
New method transfers robotic control policies to unknown environments using a family of policies.
problem Transfer of robotic control policies trained in simulation to real hardware is difficult due to environment differences.
method Simultaneously learns a family of policies with different behaviors; searches for the best policy based on task performance.
result Demonstrates superior performance in unknown environments compared to other methods, overcoming larger modeling errors.
New method learns better branching policies for MILP problems.
problem Improving branch and bound search for solving MILP problems.
method Imitates strong branching rule with parameterized state of B&B search tree.
result Generalized policies outperform current state-of-the-art.
ProMP improves Meta-RL by better handling credit assignment.
problem Poor credit assignment in Meta-RL leads to suboptimal performance.
method Develops a novel meta-learning algorithm controlling pre-adaptation and adapted policies.
result Significantly improves sample-efficiency and asymptotic performance.