MSOL learns hierarchical policies for multitask tasks with soft options.
problem Training hierarchical policies for multiple tasks with stability and flexibility.
method MSOL uses separate variational posteriors for each task, regularized by a shared prior, to avoid instabilities and fine-tune options for new tasks.
result MSOL significantly outperforms hierarchical and flat transfer-learning baselines.
Novel online algorithm for hierarchical imitation learning.
problem Scalability issue in reinforcement learning and options discovery.
method Online Baum-Welch algorithm for hierarchical imitation learning.
result The online algorithm outperforms the batch version in both discrete and continuous environments.
A new HRL method learns hierarchical policies using mutual information maximization.
problem Learning hierarchical policies in reinforcement learning for structured tasks.
method Mutual information maximization for latent variable learning, advantage-weighted importance sampling for option policies, deterministic policy gradient for optimization.
result Enhanced performance in continuous control tasks through learned hierarchical policies.
Proves EM algorithm guarantees for hierarchical imitation learning.
problem Learning hierarchical policies from expert demonstrations.
method Converts hierarchical imitation learning to latent variable model inference, analyzes EM algorithm.
result Proves EM algorithm converges to true parameter under regular conditions.
Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
This work extends the options framework to learn at multiple temporal resolutions.
problem Autonomous creation of temporal abstractions from data in reinforcement learning.
method Developed a hierarchical option-critic architecture capable of learning at multiple temporal resolutions.
result Derived policy gradient theorems for a deep hierarchy of options.
Option Encoder compresses reinforcement learning options into a policy basis.
problem Redundant options in reinforcement learning frameworks.
method Auto-encoder framework with constrained weights to discover a policy basis.
result Option Encoder reduces the number of options while maintaining performance.
Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gati…
Efficiently price high-dimensional Bermudan options using tensor compression.
problem High-dimensional option pricing with computational complexity.
method Hierarchical tensor compression for Monte Carlo and dual martingale methods.
result Tensor compression alleviates the curse of dimensionality for Bermudan option pricing.
Paper extends option-critic architecture to estimate natural gradient for reinforcement learning.
problem Estimating natural gradient in hierarchical reinforcement learning.
method Introduces natural option critic algorithm to estimate natural gradient for option's policy and termination function.
result Improves over vanilla gradient approach in experimental results.
This work improves metric learning models by incorporating class hierarchies.
problem Class hierarchies are often ignored in classification-based metric learning models.
method Trained softmax classifier and metric learning models with predefined class hierarchies.
result ProxyDR model performs better in hierarchical inference and hierarchy-informed performance.
We improve learning sub-tasks in hierarchical reinforcement learning using hyperbolic embeddings.
problem Learning meaningful sub-tasks in hierarchical reinforcement learning remains challenging.
method Combining routing in computer networks and graph-based skill discovery, we use hyperbolic embeddings to define sub-goals.
result Hyperbolic embeddings enforce a global topology on states, enabling the learning of meaningful sub-tasks.
New method speeds up option pricing for rough Bergomi model.
problem Time-consuming option pricing under rough Bergomi model.
method Hierarchical adaptive sparse grids and quasi Monte Carlo.
result Substantial computational gains over standard Monte Carlo.
In this paper we combine one method for hierarchical reinforcement learning - the options framework - with deep Q-networks (DQNs) through the use of different "option heads" on the policy network, and a supervisory network for choosing between the different options. We utilise our setup to investigate the effects of ar…
DAC combines actor-critic methods for learning options.
problem Learning hierarchical policies in reinforcement learning.
method Double Actor-Critic (DAC) architecture using two parallel augmented MDPs.
result DAC outperforms existing methods in robot simulation tasks.
PODNet discovers plannable options from unstructured demonstrations.
problem Learning from unstructured, multi-objective demonstrations.
method Custom categorical variational autoencoder, recurrent option inference network, option-conditioned policy network, and option dynamics model.
result PODNet enables learning from demonstration for multiple tasks and planning.
Unified coagent network theory with execution paths.
problem Hierarchical reinforcement learning exploration-exploitation trade-off.
method Formalized coagent networks, introduced execution paths, revisited coagent network theory.
result Shorter proof of policy gradient theorem, generalizable to asynchronous coagents.
A new HRL algorithm learns and exploits multiple subgoals for faster exploration.
problem Sparse reward problem in reinforcement learning.
method Multi-goal HRL algorithm with Manager and Worker policies.
result Significantly improved exploration efficiency with reduced training time.
New algorithms improve learning of long-term actions in reinforcement learning.
problem Violation of parameter independence assumption in deep function approximation.
method Reconsidered option-critic and hierarchical option-critic training for deep settings.
result Significantly improved stability and faster convergence in Atari games.
This paper presents a way of solving Markov Decision Processes that combines state abstraction and temporal abstraction. Specifically, we combine state aggregation with the options framework and demonstrate that they work well together and indeed it is only after one combines the two that the full benefit of each is re…
This paper solves inverse reinforcement learning with options framework.
problem Reinforcement learning in complex environments with expert demonstrations.
method Gradient method for parametrized options, deducing Q-feature and reward feature spaces, optimal reward function selection.
result Recovered rewards provide effective solution to IRL problem and accelerate transfer learning.
New method smooths integrands for efficient option pricing.
problem Improving numerical performance of option pricing methods.
method Combining hierarchical adaptive sparse grids, quasi-Monte Carlo, and numerical smoothing.
result Improved efficiency of ASGQ and QMC methods for high-dimensional problems.
The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. …
New hierarchical RL agent learns to generalize in complex multi-agent games.
problem Current RL methods struggle to generalize to unseen opponents in multi-agent games.
method Proposed a hierarchical RL architecture grounded in game-theoretic structure.
result Hierarchical agent generalizes to unseen opponents, while baselines fail.
Eigenoptions improve credit assignment in reinforcement learning.
problem Improving credit assignment in reinforcement learning models.
method Investigated eigenoptions for credit assignment in model-free RL, comparing pre-specified and online discovery methods.
result Pre-specified eigenoptions aid exploration and credit assignment, while online discovery can hinder learning.
New framework estimates staged tree models using hierarchical clustering on the probability simplex.
problem Estimating staged tree models with context-specific dependencies.
method Hierarchical clustering on the probability simplex, using simplex-based divergences and linkage methods.
result Total Variation divergence with Ward.D2 linkage produces staged trees with better model fit, structure recovery, and computational efficiency.
Study uses SABR model to create implied volatilities from sparse quotes.
problem Creating accurate implied volatility surfaces from limited market data.
method Multitask Gaussian process with SABR model embeddings and hierarchical regularization.
result Model produces more accurate volatilities than single-task methods.
DSE learns transferable skills across changing dynamics and goals.
problem Learning transferable skills across different reinforcement learning tasks.
method Variational inference for multi-task reinforcement learning with shared and task-specific latent spaces.
result Policies can generalize to unseen dynamics and goals conditions.
Numerous empirical proofs indicate the adequacy of the time discrete auto-regressive stochastic volatility models introduced by Taylor in the description of the log-returns of financial assets. The pricing and hedging of contingent products that use these models for their underlying assets is a non-trivial exercise due…
A new method optimizes Fourier pricing for multi-asset options using adaptive quadrature.
problem Efficiently pricing multi-asset options in Lévy models.
method Optimized damping parameters and hierarchical adaptive quadrature.
result Significant speed-up in computational time for up to six dimensions.
A new reinforcement learning approach using competitive primitives that specialize and specialize based on information needs.
problem Complex environments require efficient and specialized decision-making.
method Decomposes policy into competitive primitives that decide based on information needs, regularized to use minimal information.
result Improves generalization over flat and hierarchical policies.
CPGAs converge to locally optimal policies for coagent networks.
problem Training stochastic neural networks using reinforcement learning.
method Proved convergence of CPGAs and extended prior theory to asynchronous and recurrent networks.
result CPGAs converge to locally optimal policies for coagent networks, including asynchronous and recurrent networks.
New method explains ML performance gaps without causal knowledge.
problem Understanding why ML algorithms perform differently across domains.
method Nonparametric hierarchical decomposition framework.
result Detailed variable-level explanations for performance gaps.
AIRL learns robust, generalizable reward functions from demonstrations.
problem Learning robust reward functions from demonstrations for changing environments.
method Adversarial Inverse Reinforcement Learning (AIRL) with hierarchical disentangled rewards.
result Generalizable policies and comparable results to state-of-the-art methods.
FOSC-X: An extended framework for extracting multiple optimal flat clusterings from hierarchical cluster trees
problem Extracting multiple optimal flat clusterings from hierarchical cluster trees
method Dynamic programming with lower and upper feasibility bounds
result Guaranteed optimal rankings of top-M solutions with linear-time complexity
MTL-NAS combines NAS with GP-MTL for task-agnostic multi-task learning.
problem Designing architectures for diverse tasks with varying priors.
method Disentangled GP-MTL networks, hierarchical feature sharing, and gradient-based search.
result General-purpose model trained once can adapt to multiple tasks.
CREAM models enable concept-grounded predictions and interpretability.
problem Designing models that can encode and extend prior knowledge about concept-concept and concept-task relationships.
method Proposes a flexible and efficient framework (CREAMs) that encodes arbitrary C−C and CoY relationships, incorporating a side-channel for incomplete concept sets. result CREAM models achieve competitive task performance while encouraging concept-grounded predictions, avoiding concept leakage and achieving black-box-level performance.
We present a numerical method for the frequent pricing of financial derivatives that depends on a large number of variables. The method is based on the construction of a polynomial basis to interpolate the value function of the problem by means of a hierarchical orthogonalization process that allows to reduce the numbe…
Paper introduces hierarchical softmax for global hierarchical classification tasks.
problem Improving classification accuracy in tasks with class hierarchies.
method Global hierarchical neural networks using hierarchical softmax.
result Hierarchical softmax outperforms regular softmax in multiple datasets.
This study compares hierarchical and non-hierarchical models for open-domain multi-turn dialog generation.
problem Which kind of models (hierarchical or non-hierarchical) is better for open-domain multi-turn dialog generation?
method Systematically compared nearly all representative hierarchical and non-hierarchical models over the same experimental settings.
result Nearly all hierarchical models are worse than non-hierarchical models in open-domain multi-turn dialog generation, except for HRAN.
We survey agglomerative hierarchical clustering algorithms and discuss efficient implementations that are available in R and other software environments. We look at hierarchical self-organizing maps, and mixture models. We review grid-based clustering, focusing on hierarchical density-based approaches. Finally we descr…
This paper provides fast estimates for complex option types.
problem Estimating prices for constrained multiple exercise American options.
method Lookahead search for lower estimates and nearest-neighbor martingale for upper estimates.
result Probabilistic convergence guarantees for the algorithms.
Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.
problem Tuning hierarchical topic models, especially determining the number of topics at each level, is challenging.
method The paper introduces a Renyi entropy-based metric for quality assessment and a practical tuning concept.
result The proposed method can estimate the number of topics for two hierarchical levels in hARTM model.
Study bounds for prices of European and American options with optional termination.
problem Bounding prices of options with potential termination.
method Duality results linking upper prices of vulnerable options to American options with constrained exercise times.
result Linking upper prices of vulnerable options to American options and game options.
Neural NMF discovers hierarchical topics in multilayer data.
problem Detecting latent hierarchical structure in multilayer data.
method Recursive application of nonnegative matrix factorization (NMF) in layers with backpropagation optimization.
result Neural NMF outperforms other hierarchical NMF methods in synthetic and real-world datasets.
The paper develops a decision support system for hierarchical text classification of conference proceedings.
problem Classifying documents with a fixed hierarchical structure of topics.
method Developed a weighted hierarchical similarity function to calculate topic relevance, using entropy of words to estimate weights.
result The weighted hierarchical similarity function improves ranking accuracy compared to other methods.
New option pricing formulas for American and Bermudan options.
problem Traditional option pricing models assume constant volatility and interest rate.
method Relaxing assumptions, using square root of Brownian motion, providing closed-form formulas.
result Simple, closed-form pricing formulas for American and Bermudan options.
Path integral method calculates barrier option prices.
problem Barrier option pricing in finance.
method Path integral method applied to trapezoid and square potential barriers.
result Analytical expressions for option pricing derived.