A new EM framework for goal-conditioned RL improves performance on sparse reward tasks.
problem Handling sparse rewards in goal-conditioned reinforcement learning.
method A graphical model framework with an EM algorithm that includes a learning-in-hindsight E-step and a supervised M-step.
result hEM significantly outperforms model-free baselines on goal-conditioned benchmarks with sparse rewards.
DAGR improves navigation by refining goal representations conditioned on the current state.
problem Goal-conditioned reinforcement learning lacks state awareness, leading to inefficient policy recovery.
method DAGR refines static goal embeddings into state-conditioned ones using gated cross-attention with a state-goal discrepancy map.
result DAGR improves navigation tasks on OGBench, matching or outperforming base methods.
A novel framework uses goal-conditioned reinforcement learning to generate diverse samples.
problem Generating high-quality, diverse samples from generative models.
method Two agents: GC-agent learns to reconstruct the training set, S-agent learns to imitate GC-agent without knowing the goals.
result Empirically, the method generates diverse and high-quality samples in image synthesis.
A novel approach learns goal-conditioned policies for locomotion using batch RL.
problem Training goal-conditioned policies for rotation invariant locomotion.
method Data augmentation and Siamese framework for invariance.
result Our approach outperforms existing RL algorithms on 3D locomotion agents.
Goal-conditioned policies are used in order to break down complex reinforcement learning (RL) problems by using subgoals, which can be defined either in state space or in a latent feature space. This can increase the efficiency of learning by using a curriculum, and also enables simultaneous learning and generalization…
This work improves imitation learning and goal-conditioned RL by estimating value densities.
problem Effective solutions for imitation and goal-conditioned reinforcement learning require reliably reaching specified states or demonstrations.
method The approach uses recent advances in density estimation to learn value functions efficiently and without hindsight bias.
result The method achieves state-of-the-art demonstration sample-efficiency in imitation learning and is both efficient and bias-free in goal-conditioned reinforcement learning.
Generative neural nets learn deep policies conditioned on goals.
problem Learning optimal policies for specific goals in reinforcement learning.
method Goal-conditioned neural nets that generate deep neural policies.
result Single learned policy generator can achieve any desired return.
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.
Physics-informed GCRL tackles sparse feedback learning with hybrid dynamics.
problem Sparse feedback learning with high-dimensional, hybrid, or contact-dependent dynamics.
method Introduces physics-informed inductive biases into goal-conditioned value learning.
result Contact-rich manipulation tasks degrade existing Pi-GCRL methods.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Eikonal-Constrained QRL improves goal-reaching in reinforcement learning.
problem Reward design and out-of-distribution generalization in reinforcement learning.
method Eikonal-Constrained Quasimetric Reinforcement Learning (Eik-QRL) using the Eikonal PDE.
result Eik-QRL achieves state-of-the-art performance in offline goal-conditioned navigation and manipulation tasks.
Advances in Deep Reinforcement Learning have led to agents that perform well across a variety of sensory-motor domains. In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction. Final goals are specified to our agent via images o…
Agent learns goals and rewards through language and curiosity.
problem Autonomous agents lack intrinsic motivations and reward functions.
method LE2 algorithm using NL interactions and intrinsic motivations.
result Agent autonomously discovers and grounds goals in real behavior.
Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent to perform goal-directed tasks is to use Hindsight Learning approaches. In thes…
New approach decouples skill learning and language grounding for autonomous agents.
problem Autonomous acquisition of skills without external instructions and feedback.
method Language-Goal-Behavior (LGB) architecture with semantic representation.
result Decouples skill learning and language grounding, enabling diversity and strategy switching.
Automatically generates curricula for reinforcement learning agents.
problem Learning in dynamic, sparse reward environments.
method Setter-solver paradigm focusing on goal validity, feasibility, and coverage.
result Demonstrated success in 2D and 3D environments with varying goals.
We propose a novel framework for multi-task reinforcement learning (MTRL). Using a variational inference formulation, we learn policies that generalize across both changing dynamics and goals. The resulting policies are parametrized by shared parameters that allow for transfer between different dynamics and goal condit…
The paper introduces an adjacency constraint to improve goal-conditioned HRL.
problem Training inefficiency in goal-conditioned HRL due to large action space.
method Restricting the high-level action space to a k-step adjacent region of the current state.
result The adjacency constraint preserves optimal hierarchical policies and improves HRL performance.
KEMP predicts long-term trajectories for autonomous driving using keyframes.
problem Predicting future trajectories of road agents for autonomous driving.
method Keyframe-based hierarchical end-to-end deep learning framework.
result Ranked 1st on Waymo Open Motion Dataset Leaderboard.
Agent learns causal relationships from visual data to perform tasks.
problem Performing tasks in novel environments with latent causal structures.
method Learning-based approach to induce causal graphs from visual observations, using attention mechanisms.
result Effective generalization to new tasks with unseen causal structures.
New method tackles parcel routing with AI.
problem Routing parcels efficiently through a network of hubs.
method Combines graph neural networks with model-free RL.
result Extracts small feature graphs from the environment state.
In Multi-Goal Reinforcement Learning, an agent learns to achieve multiple goals with a goal-conditioned policy. During learning, the agent first collects the trajectories into a replay buffer, and later these trajectories are selected randomly for replay. However, the achieved goals in the replay buffer are often biase…
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. Towards these capabilities, we present a probabilistic forecasting model of future interactions b…
Unified algorithm tackles various RL goals like reward-free and preference-based learning.
problem Unified approach to multiple RL learning goals.
method Decision-Estimation Coefficient (DEC) framework.
result Unified algorithm handles various learning goals with a single framework.
For an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires. Furthermore, to provide the requisite level of generality, these skills must handle raw sensory input such as images. In this paper, we propose an algo…
Autonomous agents that must exhibit flexible and broad capabilities will need to be equipped with large repertoires of skills. Defining each skill with a manually-designed reward function limits this repertoire and imposes a manual engineering burden. Self-supervised agents that set their own goals can automate this pr…
Many AI problems, in robotics and other domains, are goal-directed, essentially seeking a trajectory leading to some goal state. In such problems, the way we choose to represent a trajectory underlies algorithms for trajectory prediction and optimization. Interestingly, most all prior work in imitation and reinforcemen…
Consider mutli-goal tasks that involve static environments and dynamic goals. Examples of such tasks, such as goal-directed navigation and pick-and-place in robotics, abound. Two types of Reinforcement Learning (RL) algorithms are used for such tasks: model-free or model-based. Each of these approaches has limitations.…
Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent s…
Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward function…
AMIGo uses adversarial intrinsic goals to teach RL agents new skills.
problem Learning in sparse reward environments.
method Adversarial intrinsic goals to generate a curriculum for a student policy.
result AMIGo enables agents to learn new skills without extrinsic rewards.
PlanGAN uses GANs to plan efficient trajectories for multi-goal tasks in sparse reward environments.
problem Learning with sparse rewards in multi-goal environments.
method PlanGAN combines GANs to generate trajectories leading to specified goals, then combines these into a planning algorithm.
result PlanGAN achieves comparable performance to model-free RL but is 4-8 times more sample efficient.
Method solves long-horizon robotic tasks via imitation and reinforcement learning.
problem Long-horizon robotic tasks with complex sequences of actions.
method Two-phase approach: imitation learning followed by reinforcement learning.
result Method can scale to challenging long-horizon tasks.
Paper tackles goal-directed generation of discrete structures using conditional generative models.
problem Challenges in generating structured discrete data, especially for problems like program synthesis and materials design.
method Investigates conditional generative models to directly model the distribution of discrete structures given properties of interest. Introduces a novel approach to optimize a reinforcement learning objective.
result Improvements over maximum likelihood estimation and other baselines in generating molecules and identifying short python expressions.
Efficient exploration is necessary to achieve good sample efficiency for reinforcement learning in general. From small, tabular settings such as gridworlds to large, continuous and sparse reward settings such as robotic object manipulation tasks, exploration through adding an uncertainty bonus to the reward function ha…
AI bias arises from human-defined goals, not algorithmic flaws.
problem AI bias due to human-defined goals in LLMs.
method Purpose-conditioned cognition and revealing downstream use of LLM outputs.
result AI bias can be reduced by purpose-aware prompting but not fully by regularization.
Pre-trained language model boosts RL efficiency.
problem Low sample efficiency in RL, especially in lifelong learning.
method Use a pre-trained task-independent language model for transfer learning.
result Goal-conditional RL agents become more sample efficient.
Paper analyzes neural network complexity for planning problems.
problem Understanding neural network complexity for planning policies.
method Circuit complexity analysis for relational neural networks.
result Three classes of planning problems identified based on network complexity.
In this note, we study the prescribed mean curvature equation with Neumann boundary conditions on Riemannian product manifold Mn×R. The main goal is to establish the boundary gradient estimates for solutions by the maximum principle. As a consequence, we obtain an existence result.
We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal. Our approach, Myopic Posterior Sampling (MPS), is inspired by the classical posterior (Thompson) sampling algorithm for multi-armed…
Proposes Mv-TCNN for continual learning without known tasks in advance.
problem Catastrophic forgetting in continual learning.
method Multi-view Task Conditional Neural Networks (Mv-TCNN).
result Outperforms state-of-the-art continual learning models.
The goal of this short paper is to give condition for the completeness of the Binet-Legendre metric in Finsler geometry. The case of the Funk and Hilbert metrics in a convex domain are discussed.
Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
problem Bayesian Optimization
method Bayesian Optimization with Conditional Diffusion Models
result DMS outperforms standard BO baselines
Despite its popularity, it is widely recognized that the investigation of some theoretical aspects of clustering has been relatively sparse. One of the main reasons for this lack of theoretical results is surely the fact that, whereas for other statistical problems the theoretical population goal is clearly defined (as…
Paper explores how text generation quality and diversity metrics relate to distribution fitting.
problem Unclear relation between text generation quality and diversity metrics and distribution fitting.
method Theoretical approach to prove a linear combination of quality and diversity metrics can be a divergence metric.
result CR/NRR proposed as a better substitute for BLEU/Self-BLEU metrics.
This study analyzes convergence and stability of reinforcement learning algorithms.
problem Understanding the conditions under which reinforcement learning algorithms converge and remain stable.
method Theoretical analysis of convergence and stability of Episodic Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning, and Online Decision Transformers.
result The algorithms can achieve near-optimal behavior if the transition kernel is close to a deterministic kernel.
The paper develops methods to predict the probability of achieving a user goal in a task, ensuring the system alerts when the probability falls below a threshold.
problem Ensuring an autonomous system achieves the user's goal with calibrated probability estimates.
method Invertible conformal prediction using Probability-space Conformalized Quantile Regression (PCQR) to produce well-calibrated conditional prediction intervals.
result The method produces well-calibrated probabilities that the cumulative reward will fall within a user-specified target interval, with finite-sample guarantees.
DFI maps covariates to latent representations for feature importance.
problem Feature importance when predictors are statistically dependent.
method Disentangled Feature Importance (DFI) using entropic optimal transport.
result DFI yields stable, interpretable, uncertainty-quantified attributions of shared predictive signal.