Meta-learning algorithms use past experience to learn to quickly solve new tasks. In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks. The performance of meta-learning algorithms depe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “meta reinforcement learning”
Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.
problem Difficulty in learning meta policies for sparse reward RL problems.
method Hyper-Meta RL framework with cross-environment meta state embedding and shaped meta reward.
result Improves meta policy generalization and efficiency for sparse reward RL problems.
Study OOD generalization in meta-reinforcement learning using information theory.
problem Understanding how meta-reinforcement learning handles distribution shifts.
method Information-theoretic analysis of Markov Decision Processes and gradient-based algorithms.
result Established fine-grained generalization bounds for meta-reinforcement learning.
Paper tackles offline meta-reinforcement learning with a new algorithm.
problem Performing reinforcement learning on limited data from a new task.
method Meta-Actor Critic with Advantage Weighting (MACAW) algorithm.
result Achieves notable gains over prior methods on offline meta-RL benchmarks.
Study on meta-reinforcement learning generalization in high-dimensional tasks.
problem Generalization performance of meta-reinforcement learning algorithms in high-dimensional tasks.
method High-dimensional, procedurally generated environments.
result Meta-reinforcement learning algorithms exhibit strong overfitting on challenging tasks.
New symmetries improve meta-reinforcement learning's generalization.
problem Black-box meta reinforcement learning struggles with generalization.
method Introduce symmetries to a black-box meta RL system.
result Incorporating symmetries improves generalization to new environments.
Gradient-based meta-RL fails with incorrect task distributions, leading to instability and poor performance.
problem Gradient-based meta-RL's sensitivity to task distributions causes instability and poor performance.
method Proposes meta Active Domain Randomization (meta-ADR) to learn task distributions for gradient-based meta-RL.
result Meta-ADR improves stability and generalization of MAML on simulated locomotion and navigation tasks.
Meta-reinforcement learning improves fault-adaptive control efficiency.
problem Adaptive control under abrupt system faults with strict time constraints.
method Model-agnostic meta learning (MAML) with a fault library of prior policies.
result Improved sample efficiency and quick adaptation to new faults.
Proves efficient learning of hierarchical structure in meta-reinforcement learning.
problem Lack of provable guarantees for learning hierarchical structures in reinforcement learning.
method Analyzed HRL in meta-RL setting with tabular transition dynamics, providing diversity conditions and regret bounds.
result Sample-efficient recovery of natural hierarchical structure with provable guarantees.
Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.
problem Improving meta-reinforcement learning by providing full posterior distributions.
method Augmenting point estimates with Laplace approximation for full posterior distributions.
result Our method performs similarly to variational baselines with fewer parameters.
New method improves meta-reinforcement learning efficiency.
problem Sample inefficiency in meta-reinforcement learning.
method Hindsight Foresight Relabeling (HFR) method.
result HFR improves performance on various meta-reinforcement learning tasks.
Meta-reinforcement learning algorithms can enable robots to acquire new skills much more quickly, by leveraging prior experience to learn how to learn. However, much of the current research on meta-reinforcement learning focuses on task distributions that are very narrow. For example, a commonly used meta-reinforcement…
FOCAL tackles offline meta-reinforcement learning with efficient task inference and behavior regularization.
problem Efficiently adapt RL algorithms to unseen tasks without interactions, addressing bootstrapping errors and robust task inference.
method FOCAL combines behavior regularization, a deterministic context encoder, and a negative-power distance metric for efficient task inference.
result FOCAL outperforms prior algorithms on meta-RL benchmarks, demonstrating computational efficiency.
Novel Bayesian meta-reinforcement learning framework improves traffic signal control robustness.
problem Lack of robustness and stability in adaptation for traffic signal control.
method Value-based Bayesian meta-reinforcement learning framework BM-DQN with fast-adaptation variation and DQN fast-update advantage.
result Framework adapts more quickly and robustly to new scenarios than previous methods.
Biological evolution has distilled the experiences of many learners into the general learning algorithms of humans. Our novel meta reinforcement learning algorithm MetaGenRL is inspired by this process. MetaGenRL distills the experiences of many complex agents to meta-learn a low-complexity neural objective function th…
Algorithm learns new tasks efficiently from past experience.
problem Lack of robustness to distributional shift in meta-reinforcement learning.
method Model Identification and Experience Relabeling (MIER) using dynamics models.
result Efficient extrapolation to out-of-distribution tasks.
FinRL-Meta creates diverse market environments for DRL in finance.
problem Inaccurate financial data and diverse market environments challenge DRL in finance.
method Open-source data processing tools, hundreds of market environments, and multiprocessing.
result FinRL-Meta improves DRL accuracy and speed in financial simulations.
New method improves convergence of RL meta-learning.
problem Improving convergence in model-agnostic meta-reinforcement learning.
method Proposes Stochastic Gradient Meta-Reinforcement Learning (SG-MRL) to find -first-order stationary points.
result Derives iteration and sample complexity for SG-MRL.
Minimum attention improves reinforcement learning performance in high-dimensional dynamics.
problem Improving reinforcement learning performance in high-dimensional nonlinear dynamics.
method Applying minimum attention as a regularization technique in reinforcement learning, including model-based and model-free approaches.
result Minimum attention outperforms state-of-the-art algorithms in few-shot adaptation and variance reduction.
Novel meta-RL strategy improves efficiency in learning novel tasks.
problem Efficiency in learning novel tasks using deep RL.
method Decomposes meta-RL into task-exploration, task-inference, and task-fulfillment; uses deep networks and a task encoder.
result Improves sample efficiency and mitigates meta-overfitting.
AdMRL improves meta-reinforcement learning by minimizing worst-case sub-optimality gap.
problem Meta-reinforcement learning's sensitivity to task distribution shift.
method Model-based adversarial approach with minimax objective and alternating optimization.
result Efficacy in worst-case performance, generalization to out-of-distribution tasks, and sample efficiency.
Most meta reinforcement learning (meta-RL) methods learn to adapt to new tasks by directly optimizing the parameters of policies over primitive action space. Such algorithms work well in tasks with relatively slight difference. However, when the task distribution becomes wider, it would be quite inefficient to directly…
FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learningq-fin.TR
FinRL-Meta offers market environments and benchmarks for financial reinforcement learning.
problem Challenges in creating high-quality market environments and benchmarks for financial reinforcement learning.
method DataOps paradigm, automatic pipeline, community-wise competitions, Jupyter/Python demos.
result Openly accessible FinRL-Meta library for data-driven financial reinforcement learning.
Meta-learning balances task-specific modeling and optimization complexity.
problem Balancing accurate task-specific modeling with ease of optimization in meta-learning.
method Theoretical and empirical analysis of trade-off between modeling and optimization in meta-learning.
result Explicit bounds on modeling and optimization errors for non-convex and linear regression problems.
HyperX uses reward bonuses to enable efficient exploration in meta-learning.
problem Catastrophic failure of meta-learning with sparse rewards.
method HyperX uses novel reward bonuses to explore in approximate hyper-state space.
result HyperX meta-learns better task-exploration and adapts more successfully to new tasks.
This work extends HiP-MDPs to robust state abstractions for multi-task and meta-reinforcement learning.
problem Limited observability of state in HiP-MDPs for real-world scenarios with rich observation spaces.
method Inspired by Block MDPs, the work extends HiP-MDPs to enable robust state abstractions for multi-task and meta-reinforcement learning.
result Transfer and generalization bounds based on task and state similarity, and sample complexity bounds that depend on the aggregate number of samples across tasks.
CARML uses meta-learning to avoid obstacles in 2D vehicle navigation.
problem Collision avoidance in 2D vehicle navigation.
method Model-Agnostic Meta-Learning for multi-objective reinforcement learning.
result CARML outperforms a baseline TD3 solution in obstacle avoidance.
Meta-learning improves drone trajectory design for dynamic wireless networks.
problem Designing optimal trajectories for energy-constrained drones in dynamic network environments.
method Proposes a meta-learning algorithm to adaptively tune a reinforcement learning solution for trajectory design.
result Meta-tuned RL yields faster convergence and improved communication performance compared to baseline algorithms.
Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.
problem Efficient and robust adaptation to new tasks with uncertainty assessment.
method Extends Bayesian meta-learning with gradient-EM algorithm, decoupling inner-update from meta-update.
result Improves accuracy with less computation cost and enhanced robustness to uncertainty.
The paper proposes a learning algorithm that improves adaptability and generalization.
problem Improving adaptability and generalization in learning models.
method Learning to meta-learn by meta-finetuning on related tasks before adapting to specific tasks.
result Learning to meta-learn improves adaptability and generalization across various tasks.
The capacity of meta-learning algorithms to quickly adapt to a variety of tasks, including ones they did not experience during meta-training, has been a key factor in the recent success of these methods on few-shot learning problems. This particular advantage of using meta-learning over standard supervised or reinforce…
Meta-AAD uses deep reinforcement learning to improve anomaly detection by selecting the most informative instances.
problem High false-positive rate in anomaly detection, especially in high-stake applications.
method Meta-AAD leverages deep reinforcement learning to train a meta-policy for query selection, optimizing the number of discovered anomalies.
result Meta-AAD significantly outperforms state-of-the-art re-ranking strategies and unsupervised baselines on 24 benchmark datasets.
Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learningcs.LG
CCM improves context for Meta-RL by contrastive learning.
problem Improving context for Meta-RL to enable task generalization.
method CCM framework using contrastive learning for context encoding and information-gain-based trajectory collection.
result CCM outperforms state-of-the-art algorithms in benchmarks and sparse-reward environments.
Deep reinforcement learning algorithms require large amounts of experience to learn an individual task. While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality. Current methods rely heavi…
New meta-RL method avoids exploration-exploitation trade-off.
problem Learning to explore and exploit simultaneously in meta-RL.
method Developed new objectives for exploration and exploitation.
result DREAM outperforms existing methods on complex tasks.
Meta-DRL improves resource allocation in O-RAN networks.
problem Dynamic resource allocation in O-RAN networks.
method Meta Deep Reinforcement Learning (Meta-DRL) inspired by MAML.
result 19.8% improvement in network management performance.
Unified RL meta-learning framework for few-shot optimization.
problem Few-shot learning optimization problems.
method Generic RL meta-learning framework that learns optimal optimization algorithms.
result Significantly improved performance on few-shot tasks.
Meta-gradient RL learns to learn from experience.
problem Learning efficiency and adaptability in reinforcement learning.
method Meta-gradient descent to discover its own objective function.
result Meta-gradient RL adapts to learn more efficiently over time.
Meta-agent learns effective exploration from offline data.
problem Design a meta-agent to quickly maximize reward in unseen tasks.
method Bayesian RL approach with adaptive neural belief estimate.
result Meta-agent learns effective exploration behavior from diverse tasks.
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
problem Improving sample efficiency in continuous control tasks.
method Meta-critic that meta-learns an additional loss for the actor.
result Online meta-critic learning leads to improved performance in various continuous control environments.
FLAP adapts policies quickly to new tasks using shared linear representations.
problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.
ContraBAR uses contrastive learning to learn Bayes-optimal policies in RL.
problem Learning optimal policies for unknown tasks sampled from a known distribution.
method Proposes ContraBAR, a meta RL algorithm using contrastive predictive coding (CPC) for belief inference.
result ContraBAR achieves comparable performance to state-of-the-art methods and is computationally efficient.
Prior-Aligned Meta-RL: Thompson Sampling with Learned Priors and Guarantees in Finite-Horizon MDPscs.LG
New meta-reinforcement learning method improves performance in finite-horizon MDPs.
problem Improving meta-reinforcement learning in finite-horizon MDPs with shared optimal action-value functions.
method Proposes MTSRL and MTSRL+ algorithms with learned priors and covariance, coupled with prior-alignment technique for meta-regret guarantees.
result Achieves meta-regret guarantees with learned priors and covariance, outperforming prior-independent RL and bandit-only meta-baselines.
Meta-SAGE improves deep RL scalability for CO tasks by adapting pre-trained models to larger-scale problems.
problem Improving scalability of deep reinforcement learning models for combinatorial optimization tasks.
method Meta-SAGE combines a scale meta-learner and scheduled adaptation with guided exploration to adjust model parameters for larger-scale problems.
result Meta-SAGE outperforms previous methods and significantly improves scalability in CO tasks.
We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…
Reinforcement learning (RL) algorithms have demonstrated promising results on complex tasks, yet often require impractical numbers of samples since they learn from scratch. Meta-RL aims to address this challenge by leveraging experience from previous tasks so as to more quickly solve new tasks. However, in practice, th…
Proposes active learning for meta-learning in graph node response prediction.
problem Difficulty in improving performance with meta-learning due to unbalanced observations.
method Combines graph convolutional neural networks and reinforcement learning for both prediction and node selection.
result Can predict responses and select nodes even for unseen response variables.
This research tackles intervention-centric causal reasoning in learning agents by using meta-learning.
problem Learning agents lack the concept of interventions, making causal learning challenging.
method A meta-reinforcement learning algorithm is used to learn causal relationships from observational data.
result The approach enables agents to learn and manipulate the environment effectively.