This work improves Hindsight Learning for goal-directed tasks in reinforcement learning.
problem Sparse reward problems in reinforcement learning, especially goal-directed tasks.
method Improves Hindsight Experience Replay and Hindsight Policy Gradients for robotic tasks.
result Improved learning in Hindsight Experience Replay and Hindsight Policy Gradients.
UPNs embed planning within a goal-directed policy for effective visuomotor control.
problem Learning abstract representations for visuomotor control and generalization.
method Differentiable planning within a latent space, gradient descent trajectory optimization, end-to-end learning of representations.
result UPNs can transfer visuomotor planning strategies across robots with different morphologies and actuation capabilities.
New RL approach infers optimal policies via variational inference.
problem Manual design of reward functions for reinforcement learning.
method Variational inference for inferring policies achieving desired outcomes.
result Eliminates need for hand-crafted reward functions for diverse tasks.
Agent learns causal relationships from visual data to perform tasks.
problem Performing tasks in novel environments with latent causal structures.
method Learning-based approach to induce causal graphs from visual observations, using attention mechanisms.
result Effective generalization to new tasks with unseen causal structures.
Agents need world models to generalize multi-step tasks.
problem The necessity of world models for flexible, goal-directed behavior.
method Formal analysis and demonstration of the necessity of world models for agents to generalize multi-step tasks.
result World models are necessary for agents to generalize to multi-step goal-directed tasks.
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
GCPN uses reinforcement learning to generate molecules optimizing desired properties.
problem Generating novel molecules with desired properties while obeying physical laws.
method Graph Convolutional Policy Network (GCPN) trained with reinforcement learning.
result GCPN achieves significant improvements in molecule optimization tasks.
GARIM theory explains how conscious manipulation of internal representations enhances goal-directed behavior.
problem Limited understanding of how consciousness supports flexible goal-directed cognition.
method Extending a three-component theory of flexible cognition, proposing GARIM theory.
result Conscious states actively manipulate internal representations to align with goals, enhancing flexibility.
New method learns optimal environment and goal difficulty for reinforcement learning.
problem Lack of robust transfer in reinforcement learning.
method Self-Supervised Active Domain Randomization (SS-ADR).
result Optimal domain randomization strategy improves transfer in goal-directed tasks.
Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for …
Paper tackles goal-directed generation of discrete structures using conditional generative models.
problem Challenges in generating structured discrete data, especially for problems like program synthesis and materials design.
method Investigates conditional generative models to directly model the distribution of discrete structures given properties of interest. Introduces a novel approach to optimize a reinforcement learning objective.
result Improvements over maximum likelihood estimation and other baselines in generating molecules and identifying short python expressions.
Chance-constrained ActInf allows for small violations of constraints to drive goal-directed behavior.
problem Goal-directed behavior constrained by prior beliefs.
method Introducing chance constraints to ActInf, allowing for small violations of constraints.
result Chance-constrained ActInf allows for a trade-off between robust control and chance constraint violation.
Hybrid approach improves reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Approximate model from Model-Free RL, use Mean First Passage Times for reachability, design new algorithms.
result Hybrid approaches converge with fewer iterations and samples.
Paper proposes a method to make learned models focus on task-relevant information.
problem Mismatch between model objective and downstream task objective.
method Direct prediction towards task-relevant information, self-supervised.
result Model more effectively models relevant parts of the scene conditioned on the goal.
Paper proposes Imitative Models combining IL and planning for flexible goal achievement.
problem Difficult to direct IL to arbitrary goals and specify reward functions for goal-directed planning.
method Imitative Models are probabilistic predictive models that plan interpretable expert-like trajectories.
result Imitative Models outperform IL and planning approaches in a dynamic autonomous driving task.
Proposes a deep learning approach for attributed graph clustering.
problem Suboptimal performance in graph clustering due to two-step frameworks.
method Goal-directed deep learning approach using attention networks and inner product decoders.
result Superior performance compared to state-of-the-art algorithms.
Generative model learns goal-directed visual plans from raw data.
problem Learning goal-directed visual plans from high-dimensional observations.
method Combines representation learning and planning, maximizing mutual information.
result Generative model learns low-dimensional representation explaining causal data.
A new trajectory representation method for AI problems.
problem Trajectory prediction and optimization in AI problems.
method Sub-goal trees, recursively partitioning trajectories into sub-segments.
result Sub-goal trees predict trajectories faster and more accurately.
MERLIN solves tasks with hidden sensor information.
problem Partial observability challenges AI agents.
method MERLIN integrates memory formation with predictive modeling.
result MERLIN solves complex tasks without memory limitations.
Robot learns from multiple teachers to efficiently achieve various motor skill outcomes.
problem Efficiently learning motor skills from multiple teachers and strategies.
method Hierarchical active decisions based on empirical evaluation of learning progress.
result Significantly more efficient learning and coherent strategy selection.
PFAx extends PFA for global navigation in multiroom environments.
problem Global navigation in multiroom environments.
method SFA-based algorithm that decomposes tasks into subgoals, each solvable by PFAx.
result Stable global navigation in multiroom environments.
New method predicts vehicle trajectories using map lane centers.
problem Accurate long-term vehicle trajectory prediction.
method Uses map lane centers to generate goal paths and predict trajectories.
result Model outperforms state-of-the-art approaches for 6-second horizon predictions.
DISTANA improves weather prediction by inferring hidden factors from temperature data.
problem Inferring hidden factors in spatiotemporal processes without supervision.
method Enhanced DISTANA architecture for spatiotemporal data, active tuning for latent state inference.
result DISTANA achieves more accurate predictions than other methods, inferring hidden factors from temperature data.
Active inference minimizes expected free energy for optimal behavior.
problem Understanding and optimizing behavior in complex systems.
method Combines Bayesian decision theory, optimal Bayesian design, and the free energy principle.
result Active inference emerges as a unified framework for information-seeking, utility maximization, and goal-directed behavior.
GraphAF generates chemically valid molecules efficiently and accurately.
problem Generating chemically valid molecular structures while optimizing chemical properties.
method Flow-based autoregressive model combining autoregressive and flow-based approaches.
result GraphAF generates 68% chemically valid molecules without chemical knowledge rules and 100% with rules, achieving state-of-the-art performance.
A reinforcement learning algorithm improves graph construction for robustness.
problem Improving graph construction for specific objectives.
method Reinforcement learning and graph neural networks.
result The approach outperforms existing methods in robustness.
A neural network model mimics body functions for movement tasks.
problem Solving inverse and forward kinematics, dynamics for a redundant manipulator.
method Recurrent neural network with Mean of Multiple Computations principle, dynamic extension.
result Neural network solves inverse tasks and shows prototypical population-coding.
Unified framework for hybrid learning and optimization via active inference.
problem Sequential decisions in black-box evaluations requiring both task improvement and uncertainty reduction.
method Pragmatic Curiosity (PraC) framework that evaluates queries by balancing information gain and pragmatic value.
result Unified approach reduces decision risk and improves coverage of critical regions without task-specific rules.
Fine-tunes diffusion models to generate diverse samples with high genuine rewards.
problem Reward collapse in finetuning diffusion models.
method Entropy-regularized control against pretrained diffusion models.
result Efficient generation of diverse samples with high genuine rewards.
NIPA aims to translate brain learning mechanisms into scalable Bayesian inference.
problem Scalable Bayesian inference for large-scale statistical machine learning problems.
method Neural-inspired algorithm combining model-based, model-free, and episodic-control modules.
result Advances Bayesian methods and facilitates their application to deep learning.
Adapts Floyd-Warshall algorithm for RL to improve multi-goal task learning.
problem Limited transfer of learned information in model-free RL for dynamic goal tasks.
method Adapts Floyd-Warshall algorithm for RL to learn goal-conditioned action-value functions.
result FWRL achieves higher reward strategies in multi-goal tasks with fewer samples.
Universal AI seeks high-optionality states through empowerment and curiosity.
problem Understanding and optimizing AI behavior in uncertain environments.
method Unified framework combining AIXI and variational empowerment, showing how universal AI agents balance goal-directed behavior with uncertainty reduction curiosity.
result Self-AIXI asymptotically converges to AIXI performance and exhibits power-seeking behavior due to intrinsic motivations.
New method corrects state distribution mismatch for off-policy policy optimization.
problem Mismatch between behavior and evaluation policy state distributions.
method Off-policy policy gradient with state distribution correction.
result Significantly improved policy quality in simulations.
Adapts GRPO for off-policy RL, improving reward.
problem Improving training stability and efficiency in RL.
method Adapts GRPO to off-policy setting, uses clipped surrogate objectives.
result Off-policy GRPO outperforms on-policy GRPO in empirical tests.
The paper shows how to improve policies on- and off-policy using bounds.
problem Improving reinforcement learning policies on- and off-policy.
method Lower bounding the performance difference of two policies to ensure monotonic improvement from mixture samples.
result An optimization procedure that applies the proposed bound can be seen as an off-policy natural policy gradient method.
Paper improves off-policy evaluation by estimating behavior policy.
problem Evaluating policies with data from a different behavior policy.
method Importance sampling with an estimated behavior policy.
result Estimating behavior policy reduces mean squared error.
Paper tackles efficient evaluation of natural stochastic policies in offline RL.
problem Efficiency issues in evaluating natural stochastic policies due to unknown evaluation policy.
method Derive efficiency bounds for tilting and modified treatment policies, propose nonparametric estimators.
result Proposed estimators attain efficiency bounds under lax conditions and enjoy partial double robustness.
Study shows accurate OPE depends on calibrated behaviour policy models.
problem Estimating a behaviour policy for OPE when true policy is unknown.
method Empirical studies comparing parametric vs non-parametric models.
result Simple non-parametric k-nearest neighbors model produces better calibrated behaviour policy estimates.
Guided Learning improves end-to-end modeling for multi-stage decision-making.
problem Challenges in training unified neural networks for multi-stage decision-making.
method Guided Learning framework with a guide function and utility function.
result Significant improvement in performance over traditional methods.
New framework studies policy learning problems under data scarcity.
problem Learning improving policies when data is insufficient.
method Developed a mathematical framework for policy learning problems.
result Reduced policy learning problems to simpler ones in sample complexity.
New algorithms improve policy evaluation in reinforcement learning.
problem Off-policy stability and on-policy efficiency issues in policy evaluation.
method Introduced novel algorithms using oblique projection method.
result Demonstrated both off-policy stability and on-policy efficiency.
DE via conjugate policies improves exploration and policy performance.
problem Effective exploration in policy gradient methods.
method DE via conjugate policies.
result DE improves policy performance and exploration effectiveness.
Stabilizes policy optimization with off-policy data using divergence augmentation.
problem Premature convergence and instability in policy optimization with off-policy data.
method Incorporates Bregman divergence between behavior and current policies to ensure safe policy updates.
result Empirically shows better performance in data-scarce scenarios compared to other algorithms.
New method estimates state-action stationary distribution for better off-policy policy evaluation.
problem Accurately estimating state-action stationary distribution for off-policy policy evaluation.
method Estimated Mixture Policy (EMP) for state and state-action stationary distribution corrections.
result Empirical validation shows improved accuracy over state-of-the-art methods.
New methods estimate policy value and gradients for deterministic policies from off-policy data.
problem Estimating policy value and gradients for deterministic policies from off-policy data.
method Proposed new doubly robust estimators based on kernelization approaches.
result Demonstrated a rate independent of horizon length for policy value and gradient estimation.
Study designs logging policies to minimize off-policy evaluation error.
problem Minimizing OPE error with logging policies for target policies.
method Characterizes reward-coverage tradeoff, proposes a unifying framework, derives optimal policies.
result Provides actionable guidance for firms choosing recommendation systems.
DSPI connects natural policy gradient to policy iteration, proving global convergence.
problem Optimizing policies in reinforcement learning.
method DSPI framework, combining smoothed policy iteration and natural policy gradient.
result DSPI achieves geometric convergence and optimal complexity for policy optimization.
New method improves off-policy policy evaluation by balancing policy values.
problem Accurate off-policy policy evaluation in reinforcement learning.
method Developed an MDP model with balanced representation to estimate both individual and average policy values.
result Substantially lower Mean Squared Error (MSE) in various benchmarks and a real-world HIV treatment simulation.