VTA learns hierarchical temporal structure for sequential data.
problem Learning interpretable temporal structure in sequential data.
method Hierarchical recurrent state space model with variational approach.
result VTA models 2D and 3D visual sequences with hierarchical structure.
Develops models for temporally abstract reasoning and attention.
problem Temporal abstraction and attention in reinforcement learning.
method Defines affordances for options and develops partial option models.
result Identifies trade-offs between estimation and approximation error.
New method learns temporal abstractions by defining interest functions.
problem Learning temporal abstractions with limited, variable durations.
method Introduced interest functions to define initiation sets, enabling gradient-based learning.
result Demonstrated effectiveness in discrete and continuous environments.
TD-VAE learns beliefs about future states in complex environments.
problem Lack of models that can build abstract states, form beliefs, and perform temporal abstraction.
method Temporal Difference Variational Auto-Encoder (TD-VAE) trained on pairs of time points using temporal difference learning.
result TD-VAE generates representations with beliefs about future states, allowing direct rolling out without single-step transitions.
HO2 learns options from data efficiently, improving robot manipulation tasks.
problem Learning options from raw pixel inputs in 3D robot manipulation tasks.
method HO2 infers likely option choices and trains all policy components off-policy.
result HO2 outperforms existing methods on 3D robot manipulation tasks.
ASI allows models to skip unnecessary time intervals for sequential predictions.
problem The need for precise time intervals in sequential predictions.
method Adaptive Skip Intervals (ASI) approach to dynamically adjust prediction intervals.
result Improvement in both computational efficiency and prediction accuracy.
Paper proposes a Hellinger distance regularizer to disentangle options in reinforcement learning.
problem Temporal abstraction in reinforcement learning, specifically the mutual exclusivity of learned options.
method Introduces a Hellinger distance regularizer to disentangle options.
result Demonstrates the effectiveness of the Hellinger distance regularizer in disentangling options.
Recurrent Ladder Networks improve iterative inference and temporal modeling.
problem Complex learning tasks requiring iterative inference and temporal modeling.
method Proposes a recurrent extension of Ladder networks.
result Shows close-to-optimal results on temporal modeling of video data and competitive results on music modeling.
Language helps RL agents learn complex tasks by composing sub-skills.
problem Solving complex, temporally-extended tasks in RL.
method Using language as an abstraction to learn and reuse sub-skills.
result Agents can learn and solve diverse, temporally-extended tasks.
Enhanced LSTM model learns complex temporal dependencies.
problem Modeling long-term dependencies in sequential data.
method Gamma-LSTM with hierarchical memory units and gates.
result Gamma-LSTM outperforms regular and stacked LSTMs in sequence prediction.
A method uses RL to learn abstractions for planning, improving robot navigation and manipulation tasks.
problem Planning requires suitable abstractions for states and transitions, which RL struggles with for temporally extended tasks.
method Goal-conditioned policies learned with RL are incorporated into planning, with a latent variable model representing valid states.
result Our method significantly outperforms prior work on image-based robot navigation and manipulation tasks.
Improved RL with hindsight for faster learning.
problem Poor sample efficiency in RL with delayed rewards.
method Combines universal value functions and hindsight learning.
result Significantly accelerates learning in various tasks.
This work extends the options framework to learn at multiple temporal resolutions.
problem Autonomous creation of temporal abstractions from data in reinforcement learning.
method Developed a hierarchical option-critic architecture capable of learning at multiple temporal resolutions.
result Derived policy gradient theorems for a deep hierarchy of options.
The paper introduces a method to learn Markov state abstractions for reinforcement learning.
problem Learning Markov state representations in complex environments.
method The paper introduces a novel set of conditions and a training procedure combining inverse model estimation and temporal contrastive learning.
result The approach learns representations that capture the underlying structure of the domain and improve sample efficiency.
Paper proposes robust methods for learning temporally abstract actions in reinforcement learning.
problem Learning robust reinforcement learning policies in uncertain environments.
method Robust Options Policy Iteration (ROPI) and Robust Options Deep Q Network (RO-DQN) for learning robust options.
result ROPI and RO-DQN algorithms demonstrate robustness in solving multiple tasks and generalizing over a broader range of dynamics.
SG-NTF completes HDI tensors with spectral mapping and spatio-temporal gating.
problem High-dimensional and incomplete tensor completion.
method Spectra-Guided Neural Tucker Factorization (SG-NTF) with Spatio-Temporal Co-Gating (STCG).
result Maintains competitive completion accuracy with parameter efficiency.
MAOP learns object dynamics from raw visual data.
problem Efficient learning of dynamics from raw visual data for multiple objects.
method Three-level learning architecture with spatial-temporal relational reasoning.
result Significantly outperforms previous methods in sample efficiency and generalization.
Advantage amplification helps RL in slow-evolving latent-state environments.
problem Challenges in reinforcement learning for long-horizon latent-state environments.
method Temporal abstraction and aggregation methods to overcome belief state error and small action advantage.
result Proven advantage amplification in settings with slowly evolving latent states.
Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
Introduces a new RL formalism focusing on event desirability.
problem Limited consideration of performance distribution in RL.
method Introduces micro-objective reinforcement learning.
result New formalism allows for prior knowledge and event desirability.
Agents compose pre-trained policies for complex tasks, improving zero-shot performance.
problem Challenges in long-horizon predictions and estimating visitation distributions induced by policy sequences.
method Learn predictive jumpy world models of multi-step dynamics, enhancing predictions with a consistency objective.
result Compositional planning with jumpy world models yields, on average, a 200% relative improvement over primitive actions on long-horizon tasks.
This paper presents a way of solving Markov Decision Processes that combines state abstraction and temporal abstraction. Specifically, we combine state aggregation with the options framework and demonstrate that they work well together and indeed it is only after one combines the two that the full benefit of each is re…
h-DQN integrates hierarchical value functions with intrinsic motivation for efficient exploration.
problem Sparse feedback and insufficient exploration in reinforcement learning.
method Hierarchical-DQN framework combining temporal abstraction and intrinsic motivation.
result Demonstrated efficiency in exploration and task-solving on sparse feedback problems.
This paper solves inverse reinforcement learning with options framework.
problem Reinforcement learning in complex environments with expert demonstrations.
method Gradient method for parametrized options, deducing Q-feature and reward feature spaces, optimal reward function selection.
result Recovered rewards provide effective solution to IRL problem and accelerate transfer learning.
Proves efficient learning of hierarchical structure in meta-reinforcement learning.
problem Lack of provable guarantees for learning hierarchical structures in reinforcement learning.
method Analyzed HRL in meta-RL setting with tabular transition dynamics, providing diversity conditions and regret bounds.
result Sample-efficient recovery of natural hierarchical structure with provable guarantees.
ST-UNet models spatio-temporal graphs by pooling and unpooling operations.
problem Lack of effective means to extract dynamic features from spatio-temporal graphs.
method Designing a multi-scale architecture, Spatio-Temporal U-Net (ST-UNet), with paired sampling operations.
result Achieves substantial improvements in spatio-temporal prediction tasks.
UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.
problem Degrading performance of action repetition in reinforcement learning, especially with sub-optimal actions.
method UTE uses ensemble methods to measure uncertainty during action extension, allowing strategic exploration or certainty.
result UTE outperforms existing action repetition algorithms, significantly enhancing policy learning efficiency.
Generative Link Sequence Modeling predicts future links in evolving networks.
problem Predicting future links in networks with evolving structures.
method Sequence modeling framework with self-tokenization to capture temporal link formation patterns.
result GLSM achieves best performance on AUC metrics compared to existing methods.
Neuro-symbolic agent learns systematic generalisation from formal instructions.
problem Achieving zero-shot generalisation of formally specified tasks.
method Combines deep reinforcement learning with temporal logic.
result Systematic learning emerges with convolutional layers and abstract operators.
CAWs learn temporal network dynamics without node identities or edge attributes.
problem Learning temporal network dynamics without node identities or edge attributes.
method Causal Anonymous Walks (CAWs) using temporal random walks and hitting counts.
result CAW-N outperforms previous methods in predicting links over 6 real temporal networks.
This review explores methods for detecting Adverse Drug Events in EHRs.
problem Detecting Adverse Drug Events (ADEs) in EHRs using temporal data.
method Examines 5 main approaches: temporal abstraction, graph-based, learning weights, and time series data.
result Challenges remain in exploiting heterogeneous temporal data for ADE detection.
THGFM models dynamic relational systems with cross-type and temporal fusion.
problem Learning on temporal heterogeneous graphs with diverse node and relation types.
method Dual-Path Architecture with Shared-Space and Relational Type-Partitioned Temporal Attention.
result THGFM outperforms baseline models on academic graph benchmarks.
Convolutional-Restricted-Boltzmann-Machine learns relational order among time-related inputs.
problem Learning optimal relational order among multiple time-related inputs.
method Extended Convolutional-Restricted-Boltzmann-Machine with multiplicative units and reinforcement learning.
result The machine can learn the optimal relational order among inputs.
Mid-training improves RL by identifying compact action abstractions.
problem Unlocking full potential of RL with large language models.
method RA3 algorithm that optimizes sequential variational lower bound and discovers latent structures via RL.
result Improves performance by 8 points on HumanEval and 4 points on MBPP.
OP3 models entities for better task generalization in reinforcement learning.
problem Generalizing to unseen physical tasks with combinatorial complexity.
method Object-centric perception, prediction, and planning (OP3) framework.
result OP3 outperforms oracle models and state-of-the-art video prediction models.
Proposes RMN for learning long-term dependencies in feed-forward networks.
problem Complicated training of deep RNN architectures.
method Residual Memory Neural Network (RMN) with residual and time-delayed connections.
result RMN and BRMN outperform LSTM and BLSTM networks in learning long-term and hierarchical information.
Graph Convolutional Reinforcement Learning improves cooperation in dynamic multi-agent environments.
problem Learning cooperation in dynamic multi-agent environments is challenging.
method Graph Convolutional Reinforcement Learning adapts to dynamic graphs and captures interplay between agents.
result Our method substantially outperforms existing methods in cooperative scenarios.
GAT-AGNN learns stock trends using graph and attention mechanisms.
problem Predicting dynamic stock trends in a complex market.
method Sequential graph structure with attention mechanisms.
result GAT-AGNN outperforms state-of-the-art methods in stock trend prediction.
Method trains policies for MDPs that satisfy LTL properties.
problem Training policies for MDPs that meet complex temporal logic constraints.
method Convert LTL to automaton, construct synchronised MDP, adapt reward function, apply neural Q-iteration.
result Synthesised policies satisfy specified LTL properties.
Unified multilinear model for causal factor disentanglement.
problem Disentangling causal factors from complex data without direct manipulation.
method Hierarchical block multilinear factorization (M-mode Block SVD) and incremental approach.
result Interpretable object representation robust to occlusion and reduced training data.
Study analyzes OR/MS literature using topic models.
problem Understanding the content and evolution of OR/MS literature.
method Developed a topic model using Latent Dirichlet Allocation (LDA).
result Identified temporal dynamics and unique scopes of journals.
Detects changes in topic proportions over time in large text datasets.
problem Unsupervised detection of structural changes in topic distributions over time.
method Specialised temporal topic model with changepoint detection, approximate inference using sample splitting and likelihood ratio statistic.
result Automated detection of changepoints in topic proportions, facilitating interpretable results.
Paper uses weakly-supervised clustering to automatically create network protocol abstractions.
problem Manual definition of abstraction by domain experts is time-consuming.
method Weakly supervised clustering algorithm for automatic abstraction.
result The method successfully matches the reference abstraction with minimal labeled examples.
Abstracts index for ML4H workshop at NeurIPS 2019.
problem No specific problem stated; index of accepted abstracts.
method Not specified; index of accepted abstracts.
result No specific result stated.
DSARF models complex spatio-temporal data with deep switching auto-regressive factors.
problem Forecasting complex spatio-temporal data with recurring patterns.
method Deep switching auto-regressive factorization (DSARF) with stochastic variational inference.
result DSARF outperforms state-of-the-art methods in long- and short-term prediction accuracy.
The paper aims to mathematically define and learn abstractions from data.
problem Defining and learning abstractions from data.
method Characterize abstractions as summaries for answering queries, define leakiness as a loss function, and generalize classical statistics.
result A mathematical theory of abstraction can be learned from data.
Unified framework for causal models at different levels of abstraction.
problem Relating causal models at varying levels of abstraction.
method Categorical framework using natural transformations between Markov functors.
result Generalized and unified causal abstractions with categorical proofs.
Abstract MDPs enable strategic exploration and fast reward transfer in complex environments.
problem Challenging to learn accurate MDPs for high-dimensional states.
method Learn an abstract MDP over low-dimensional coarse states, using an abstraction function.
result Achieves superhuman performance on Pitfall! and higher reward with fewer samples.