This paper proposes a new method to automatically learn optimal return functions in reinforcement learning.
problem Learning optimal policies in reinforcement learning can be slow and inefficient.
method The authors propose a general mathematical form for the return function and use meta-learning to automatically learn the optimal form.
result Their method significantly speeds up the learning of optimal policies in reinforcement learning.
AutoAugment learns optimal data augmentation policies automatically.
problem Improving image classifier accuracy through better data augmentation.
method AutoAugment uses a search algorithm to find the best augmentation policies in a defined search space.
result AutoAugment achieves state-of-the-art accuracy on multiple datasets.
PID control architectures are widely used in industrial applications. Despite their low number of open parameters, tuning multiple, coupled PID controllers can become tedious in practice. In this paper, we extend PILCO, a model-based policy search framework, to automatically tune multivariate PID controllers purely bas…
Extract fairness policies from legal documents using machine learning.
problem Extract fairness policies from legal documents for AI applications.
method Two approaches based on semantic relatedness: Wordnet-based similarity and vector-based similarity.
result Vector-based similarity outperforms classical Wordnet-based similarity.
DisCoRL combines state learning and policy distillation for continual reinforcement learning.
problem Learning multiple policies with a single model and inferring the correct policy at test time.
method Combining state representation learning and policy distillation.
result The policy can solve all tasks and automatically infer the correct one to run.
Automatically finds effective security strategies through reinforcement learning and self-play.
problem Finding effective security strategies for intrusion prevention.
method Modeling interaction as a Markov game, evolving attack and defense strategies through reinforcement learning and self-play.
result Effective security strategies emerge from self-play, reflecting common-sense knowledge.
Robot solves Rubik's cube using simulation-trained models.
problem Solving Rubik's cube with a robot hand.
method Automatic domain randomization (ADR) and a custom robot platform.
result Control policies and vision state estimators trained with ADR exhibit improved sim2real transfer.
Pipeline selects best RL policy from limited data.
problem Selecting best RL policy from small datasets.
method Task- and method-agnostic pipeline using multiple data splits.
result Pipeline produces higher-performing policies.
MORL uses program synthesis to improve reinforcement learning policies.
problem Difficult to interpret and impose constraints on learned policies from black-box neural networks.
method Iterative framework combining program synthesis and behavior cloning.
result Programmatic representation allows for high-level modifications leading to improved learning.
Paper develops a method to automatically create learning curricula for reinforcement learning.
problem Designing an effective sequence of simpler tasks to prepare for a more complex task.
method Formulated as a Markov Decision Process, extended to handle multiple transfer learning algorithms.
result A curriculum policy can be learned from experience, improving training speed for reinforcement learning agents.
Model learns sub-goals and low-level policies for hierarchical reinforcement learning.
problem Determining appropriate low-level policies in hierarchical reinforcement learning.
method Unsupervised learning scheme based on asymmetric self-play.
result Obtains performance gains over non-hierarchical approaches.
Safe reinforcement learning with logical constraints for optimal policy synthesis.
problem Ensuring safety during reinforcement learning while maximizing goal satisfaction.
method Adaptive safe padding that synthesizes optimal control policies satisfying temporal logic formulas.
result The proposed method handles the trade-off between exploration and safety with theoretical guarantees.
New algorithm learns sub-task policies from unsegmented demonstrations.
problem Challenges in learning hierarchical policies from unsegmented demonstrations.
method Generative adversarial imitation learning framework with directed information maximization.
result Automatic learning of sub-task policies from unsegmented demonstrations.
Decima uses machine learning to automatically generate efficient scheduling policies.
problem Scheduling data processing jobs on distributed clusters is complex and requires tuning for each workload.
method Decima employs reinforcement learning and neural networks to learn workload-specific scheduling policies without human intervention.
result Decima improves average job completion time by at least 21% compared to hand-tuned heuristics.
New model learns better policies from expert demonstrations with higher efficiency.
problem Learning accurate policies from expert demonstrations with high efficiency.
method Generative adversarial imitation learning (GAIL) model that learns f-divergence automatically. result Learns better policies with higher data efficiency in physics-based control tasks.
Logit dynamics formula reveals self-regulation in softmax policy gradient methods.
problem Understanding the stability and convergence of softmax policy gradient methods.
method Deriving the exact formula for the L2 norm of the logit update vector.
result Logit update magnitudes are modulated by action probability and policy concentration.
This paper proposes a method to safely adjust exploration in RL to satisfy constraints.
problem Unsafe exploration in reinforcement learning violates constraints on controlled object states.
method Automatic adjustment of exploration inputs and variance-covariance matrix for safety.
result The method guarantees satisfaction of joint chance constraints with specified probability.
New method transfers robotic control policies to unknown environments using a family of policies.
problem Transfer of robotic control policies trained in simulation to real hardware is difficult due to environment differences.
method Simultaneously learns a family of policies with different behaviors; searches for the best policy based on task performance.
result Demonstrates superior performance in unknown environments compared to other methods, overcoming larger modeling errors.
Automatically finds strong neural network topologies for continuous control tasks.
problem Handcrafted neural network architectures limit the performance of Deep Reinforcement Learning.
method Combines Neuroevolution with off-policy training and proposes a novel architecture mutation operator.
result The proposed Actor-Critic Neuroevolution algorithm often outperforms strong baseline methods.
Algorithm design is a laborious process and often requires many iterations of ideation and validation. In this paper, we explore automating algorithm design and present a method to learn an optimization algorithm, which we believe to be the first method that can automatically discover a better algorithm. We approach th…
Proposes a new framework to optimize policies while accounting for model uncertainty.
problem Model-based reinforcement learning's vulnerability to model bias and inefficiency.
method Uncertainty-aware model-based policy optimization framework.
result Significantly lower sample complexity and competitive asymptotic performance.
ENAS discovers efficient neural architectures quickly and cheaply.
problem Automatic model design and neural architecture search.
method Parameter sharing and policy gradient for subgraph selection.
result ENAS discovers novel architectures achieving state-of-the-art performance.
HPSDE learns hierarchical policies from multi-modal rewards.
problem Learning optimal policies from a multi-modal reward function.
method Density estimation with return-weighted importance sampling for identifying modes.
result HPSDE successfully learns option policies corresponding to modes of the return function.
New method optimizes treatment policies to avoid winner's curse.
problem Winner's curse in treatment policy optimization.
method Inference-aware policy optimization.
result Optimizes for both estimated performance and downstream evaluation.
GENE tackles sparse reward in RL by generating states to explore and exploit.
problem Sparse reward in reinforcement learning.
method Generative Exploration and Exploitation (GENE) method.
result GENE significantly outperforms existing methods in tasks with binary rewards.
HOOF learns hyperparameters efficiently for policy gradients.
problem Efficient hyperparameter tuning for policy gradient methods.
method Gradient-free algorithm that optimizes hyperparameters directly through the gradient using existing trajectories.
result HOOF leads to faster learning and improved performance.
Galen algorithm compresses neural networks for specific hardware with reduced latency.
problem Finding optimal compression policies for neural networks on specific hardware.
method Reinforcement learning using pruning and quantization to optimize inference latency.
result Compressed ResNet18 for ARM processor reduced inference latency by 80%.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
problem Exploration-exploitation dilemma in reinforcement learning.
method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.
result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.
ReSkill reconciles RL skill creation with policy optimization.
problem RL policies lack reusable strategies across tasks.
method Integrates skill creation into RL loop with three mechanisms.
result Consistently outperforms existing methods, especially on unseen tasks.
Deep Reinforcement Learning (DRL) is a trending field of research, showing great promise in many challenging problems such as playing Atari, solving Go and controlling robots. While DRL agents perform well in practice we are still missing the tools to analayze their performance and visualize the temporal abstractions t…
AI-Interpret transforms opaque policies into simple, interpretable decision rules.
problem Designing effective decision aids for professionals to mitigate decision-making biases.
method Combining imitation learning, program induction, and clustering to transform learned policies into interpretable descriptions.
result Providing interpretable decision rules as flowcharts significantly improves people's planning strategies and decisions.
Oriental ink painting, called Sumi-e, is one of the most appealing painting styles that has attracted artists around the world. Major challenges in computer-based Sumi-e simulation are to abstract complex scene information and draw smooth and natural brush strokes. To automatically find such strokes, we propose to mode…
Automatic debiasing for causal and policy effects using Neural Nets and Random Forests.
problem Estimating causal and policy effects from high-dimensional or non-parametric regression functions.
method Automatic learning of Riesz representation using Neural Nets and Random Forests.
result Automatic debiasing method performs well compared to state-of-the-art algorithms.
Algorithm optimizes lockdown policies balancing health and economy.
problem Balancing health and economic impacts of lockdowns during pandemics.
method Reinforcement learning to automatically compute lockdown policies.
result Algorithm learns optimal lockdown policies from disease and population data.
New method improves deep RL efficiency by adaptively setting accuracy requirements.
problem Improving efficiency in deep reinforcement learning.
method Accuracy-based curriculum learning using adaptive selection of accuracy requirements.
result Adaptive accuracy requirements lead to better learning efficiency than random selection.
Optimizes Thompson sampling policies using policy gradient methods.
problem Improving Thompson sampling in bandit problems.
method Applies policy gradient algorithms to optimize Thompson sampling policies.
result Direct policy search on Thompson sampling improves performance.
Implicit Q-learning and SARSA adjust step-sizes automatically, improving stability and performance.
problem Numerical instability and slow progress in Q-learning and SARSA due to step-size calibration.
method Reformulate iterative updates as fixed-point equations, scaling step-sizes inversely with feature norms.
result Implicit methods maintain stability over broader step-size ranges and achieve comparable convergence rates.
A planning approach learns skills from interactions, balancing exploration and exploitation.
problem Learning robust high-level skills in noisy environments with unknown pre-conditions.
method Formulates skills as high-level policies, learns plans via bandit problems, balances exploration and exploitation.
result A planner capable of learning robust high-level skills in high-dimensional state spaces.
A language for specifying complex reinforcement learning tasks.
problem Challenges in specifying and shaping reward functions for complex reinforcement learning tasks.
method Proposes a new language and algorithm for automatically generating and shaping reward functions.
result SPECTRL tool outperforms state-of-the-art baselines.
Automates bias control in reinforcement learning algorithms.
problem Overestimation bias in reinforcement learning algorithms.
method Data-driven approach for automatic selection of bias control hyperparameters.
result Significant reduction in the number of interactions while maintaining performance.
Automatically designs CNN architectures for medical image segmentation.
problem Manual design of deep network architectures is time-consuming and resource-intensive.
method Policy gradient reinforcement learning with dice index reward function.
result Efficacy demonstrated with low computational cost compared to state-of-the-art networks.
A framework schedules hyperparameters for model-based reinforcement learning, improving performance.
problem Inadequate scheduling of hyperparameters in model-based reinforcement learning.
method Theoretical analysis and AutoMBPO framework to automatically schedule real data ratio and other hyperparameters.
result Training with hyperparameters scheduled by AutoMBPO significantly improves performance.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
RDMM integrates RL and deep learning for better trading performance.
problem Improving financial trading performance with complex market dynamics.
method Reinforced Deep Markov Model (RDMM) integrating RL and deep learning.
result RDMM outperforms benchmarks in optimal execution problems.
Sayer uses implicit feedback to optimize system policies.
problem Leveraging implicit feedback to improve system policies is difficult due to bias and incompleteness.
method Sayer combines randomized exploration and unbiased counterfactual estimators to evaluate and train new policies using implicit feedback.
result Sayer can accurately evaluate and train new policies that outperform existing ones.
DeepSynth synthesizes automata to guide deep RL agents through sparse, non-Markovian rewards.
problem Training deep RL agents with sparse, non-Markovian rewards and unknown high-level objectives.
method Employing a novel algorithm for synthesizing compact automata to uncover sequential structure from trace data.
result Reduces the number of iterations required for policy synthesis by two orders of magnitude and improves scalability.
New estimator uses clustering to improve off-policy evaluation accuracy.
problem Improving off-policy evaluation accuracy when logging and evaluation policies differ.
method Proposes an estimator that shares information across similar contexts using clustering.
result Clustering contexts improves estimation accuracy, especially in deficient information settings.
This paper introduces a new reward shaping method for average-reward reinforcement learning.
problem Speeding up convergence to an optimal policy in average-reward reinforcement learning tasks.
method Developed a temporal logic-based approach to automatically generate reward shaping functions.
result The optimal policy can be recovered using the proposed reward shaping framework.