The paper proposes a principle for dynamically adjusting the granularity of reinforcement learning abstractions.
problem Lack of general principles for dynamically adjusting the granularity of reinforcement learning abstractions.
method The paper proposes a principle based on rate-distortion theory, formalized through a performance certificate decomposing value error into learning and abstraction error bounds.
result Soft state-action abstractions can achieve near-optimal performance under substantial lossy compression of state and action information.
moment maps arise as a generalization of genuine moment maps on symplectic manifolds when the symplectic structure is discarded, but the relation between the mapping and the action is kept. Particular examples of abstract moment maps had been used in Hamiltonian mechanics for some time, but the abstract notion originat…
Develops models for temporally abstract reasoning and attention.
problem Temporal abstraction and attention in reinforcement learning.
method Defines affordances for options and develops partial option models.
result Identifies trade-offs between estimation and approximation error.
Mid-training improves RL by identifying compact action abstractions.
problem Unlocking full potential of RL with large language models.
method RA3 algorithm that optimizes sequential variational lower bound and discovers latent structures via RL.
result Improves performance by 8 points on HumanEval and 4 points on MBPP.
HO2 learns options from data efficiently, improving robot manipulation tasks.
problem Learning options from raw pixel inputs in 3D robot manipulation tasks.
method HO2 infers likely option choices and trains all policy components off-policy.
result HO2 outperforms existing methods on 3D robot manipulation tasks.
We prove the rigidity of presymplectic actions of a compact semisimple Lie algebra on a presymplectic manifold of constant rank in the local and global case. The proof uses an abstract normal form theorem we had stated in a previous work, based on an iterative process of Nash-Moser type. In order to use correctly this …
Paper learns meaningful state and action representations from MDP trajectories.
problem Learning good state and action representations from MDP trajectories.
method Tensor decomposition, kernelization, importance sampling, low-Tucker-rank approximation.
result The learned state/action abstractions provide accurate approximations to latent block structures.
A new concept of causality for abstract phenomena.
problem Unclear definition of causality in real-life variables.
method Introduces 'phenomenological causality' based on elementary actions.
result Defines causal structure without hard-wired links.
Equivariant cohomology simplifies symplectic manifold integrals with group actions.
problem Complex integrals on symplectic manifolds with Lie group actions.
method Atiyah-Bott Localization Theorem
result Simplifies integrals by revealing hidden structure.
SPEDER extracts state-action abstraction from dynamics for reinforcement learning.
problem Curse of dimensionality and limited applicability of spectral methods.
method Spectral Decomposition Representation (SPEDER) that extracts state-action abstraction from dynamics without policy dependence.
result Theoretical analysis establishes sample efficiency in online and offline settings.
The paper studies quaternionic structures on GKM graphs and their relation to torus actions on quaternionic projective spaces.
problem Understanding quaternionic structures on GKM graphs and their implications for torus actions.
method Introducing quaternionic structures on GKM graphs and analyzing their properties in the context of torus actions.
result Abstract GKM graphs with specific 2-face structures correspond to torus actions on quaternionic projective spaces or Grassmannians.
Abstract MDPs enable strategic exploration and fast reward transfer in complex environments.
problem Challenging to learn accurate MDPs for high-dimensional states.
method Learn an abstract MDP over low-dimensional coarse states, using an abstraction function.
result Achieves superhuman performance on Pitfall! and higher reward with fewer samples.
Abstract: Characterizes spaces with positive scalar curvature.
problem Spaces with positive scalar curvature and invariant metrics.
method Cohomogeneity one manifolds and homogeneous spaces with compact Lie group actions.
result Characterization of spaces with positive scalar curvature.
Deep neural network learns discrete state abstractions for efficient planning.
problem Efficient sequential decision making in large state spaces.
method Information bottleneck method for learning approximate bisimulations using deep neural encoders and action-conditioned HMM.
result Trained method efficiently plans for unseen goals in multi-goal reinforcement learning.
Survey examines challenges and solutions in sim-to-real transfer for robotics.
problem Challenges in transferring robotic systems from simulation to real-world environments.
method Leveraging techniques like domain randomization, real-to-sim transfer, state and action abstractions, and sim-real co-training.
result Promising results in closing the reality gap across various robotic domains.
Study circle actions on 4-manifolds, deriving formulas and graphs.
problem Understanding circle actions on 4-dimensional manifolds.
method Derive the Atiyah-Hirzebruch formula and associate graphs to fixed point data.
result Show existence of 4D oriented S^1-manifolds from satisfying graphs.
We investigate compact Kahler manifolds, which are acted on by a semisimple compact Lie group G of isometries with one hypersurface orbit. In case of ordinary action and projectable complex structure, we set up a one to one correspondence between such manifolds and abstract models. The Ricci tensor is then computed and…
We present a K-theoritic approach to the Guillemin-Sternberg conjecture, about the commutativity of geometric quantization and symplectic reduction, which was proved by Meinrenken and Tian-Zhang. Besides providing a new proof of this conjecture for the full non-abelian group action case, our methods lead to a generalis…
We give a systematic treatment of the stability theory for action of a real reductive Lie group G on a topological space. More precisely, we introduce an abstract setting for actions of non-compact real reductive Lie groups on topological spaces that admit functions similar to the Kempf-Ness function. The point of this…
UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.
problem Degrading performance of action repetition in reinforcement learning, especially with sub-optimal actions.
method UTE uses ensemble methods to measure uncertainty during action extension, allowing strategic exploration or certainty.
result UTE outperforms existing action repetition algorithms, significantly enhancing policy learning efficiency.
A new estimator reduces variance in slate bandit OPE.
problem Large action spaces in slate bandits cause high variance in OPE.
method Develops Latent IPS (LIPS) to optimize slate abstractions for low variance and bias.
result LIPS substantially outperforms existing estimators in scenarios with non-linear rewards and large slate spaces.
We study contextual bandit learning with an abstract policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-…
We survey the use of dynamics of SL(2,R)-actions to understand gap distributions for various sequences of subsets of [0,1), particularly those arising from special trajectories of various two-dimensional dynamical systems. We state and prove an abstract theorem that gives a unified explanation for some of the ex…
Planning methods can solve temporally extended sequential decision making problems by composing simple behaviors. However, planning requires suitable abstractions for the states and transitions, which typically need to be designed by hand. In contrast, model-free reinforcement learning (RL) can acquire behaviors from l…
Abstract: Proves generic torus diffeomorphisms act parabolically and non-properly on fine curve graph and have generalized rotation sets.
problem Generic torus diffeomorphisms on fine curve graph.
method Proves generic torus diffeomorphisms act parabolically and non-properly on fine curve graph.
result Generic torus diffeomorphisms have generalized rotation sets of any point-symmetric compact convex homothety type.
A planning approach learns skills from interactions, balancing exploration and exploitation.
problem Learning robust high-level skills in noisy environments with unknown pre-conditions.
method Formulates skills as high-level policies, learns plans via bandit problems, balances exploration and exploitation.
result A planner capable of learning robust high-level skills in high-dimensional state spaces.
Nonlinear optimal control problems are often solved with numerical methods that require knowledge of system's dynamics which may be difficult to infer, and that carry a large computational cost associated with iterative calculations. We present a novel neurobiologically inspired hierarchical learning framework, Reinfor…
In our book on cohomological methods in transformation groups the minimal Hirsch-Brown model was used to good effect. The construction there, however, was rather abstract. Here, for smooth compact connected Lie group actions on smooth closed manifolds, we give a much more explicit construction of the minmal Hirsch-Brow…
One way to interpret smoothness of a measure in infinite dimensions is quasi-invariance of the measure under a class of transformations. Usually such settings lack a reference measure such as the Lebesgue or Haar measure, and therefore we can not use smoothness of a density with respect to such a measure. We describe h…
We consider the orientation-preserving actions of finite groups G on pairs (S3,Γ), where Γ is a connected graph of genus g>1, embedded in S3. For each g we give the maximum order mg of such G acting on (S3,Γ) for all such Γ⊂S3. Indeed we will classify all graphs Γ⊂S3 which re…
We prove an extension of a celebrated equivariant bifurcation result of J. Smoller and A. Wasserman, in an abstract framework for geometric variational problems. With this purpose, we prove a slice theorem for continuous affine actions of a (finite-dimensional) Lie group on Banach manifolds. As an application, we discu…
Agents compose pre-trained policies for complex tasks, improving zero-shot performance.
problem Challenges in long-horizon predictions and estimating visitation distributions induced by policy sequences.
method Learn predictive jumpy world models of multi-step dynamics, enhancing predictions with a consistency objective.
result Compositional planning with jumpy world models yields, on average, a 200% relative improvement over primitive actions on long-horizon tasks.
New framework for AI to learn causal models through experience.
problem Lack of guidance for variable choice and interventions in causal models for AI.
method Defines actions as state space transformations, introduces causal variables, and identifies interventions.
result Clarifies the concept of interventions and makes causal representation learning clearer.
Let (M,ω) be a Kähler manifold and let K be a compact group that acts on M in a Hamiltonian fashion. We study the action of KC on probability measures on M. First of all we identify an abstract setting for the momentum mapping and give numerical criteria for stability, semi-stability and polystabili…
Abstract: Mapping class groups act on cohomology of surfaces via Hochschild cohomology.
problem Understanding the action of mapping class groups on cohomology of surfaces.
method Associate cochain complexes to surfaces, with mapping class groups acting projectively on cohomology.
result Projective action of mapping class groups on Hochschild cohomology of Hopf algebras.
Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
This work uses action equivariance to learn structured latent spaces for reinforcement learning.
problem Learning structured latent spaces for reinforcement learning.
method Introduced a contrastive loss function to enforce action equivariance on learned representations.
result Optimal policies in the abstract MDP can be successfully lifted to the original MDP.
Systematic reviews, which summarize and synthesize all the current research in a specific topic, are a crucial component to academia. They are especially important in the biomedical and health sciences, where they synthesize the state of medical evidence and conclude the best course of action for various diseases, path…
Object-based approaches for learning action-conditioned dynamics has demonstrated promise for generalization and interpretability. However, existing approaches suffer from structural limitations and optimization difficulties for common environments with multiple dynamic objects. In this paper, we present a novel self-s…
Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functio…
We shall give an axiomatic construction of Wess-Zumino-Witten actions valued in (G=SU(N)), (N\geq 3). It is realized as a functor ({WZ}) from the category of conformally flat four-dimensional manifolds to the category of line bundles with connection that satisfies, besides the axioms of a topological field theory, the …
This paper describes how to define and work with differential equations in the abstract setting of tangent categories. The key notion is that of a curve object which is, for differential geometry, the structural analogue of a natural number object. A curve object is a preinitial object for dynamical systems; dynamical …
A new method for disentangling action sequences improves model stability.
problem Challenges in unsupervised disentanglement learning due to incomplete theories and abstract notions.
method Introducing disentangling action sequences and a novel fractional variational autoencoder (FVAE) framework.
result FVAE improves the stability of disentanglement for action sequences.
Geometric obstructions prevent gravity in high dimensions.
problem Obstacles to realizing gravity in various geometries.
method Analyzing the tetradic Einstein-Hilbert-Palatini action in different geometric settings.
result Gravity is only meaningful in Lorentzian geometry for dimensions n≥4. Abstract: Generalizes Milnor-Schwarz lemma to inverse monoids.
problem Applying Milnor-Schwarz lemma to inverse monoids.
method Two proofs provided: elementary and using Vietoris-Rips complex.
result Generalization of Milnor-Schwarz lemma to inverse monoids.
In this paper we discuss the relationship between groups of diffeomorphisms of spheres and balls. We survey results of a topological nature and then address the relationship as abstract (discrete) groups. We prove that the identity component Diff_0(S^{2n-1}) of the group of smooth diffeomorphisms of S^{2n+1} admits no …
Sequential decision making in the presence of uncertainty and stochastic dynamics gives rise to distributions over state/action trajectories in reinforcement learning (RL) and optimal control problems. This observation has led to a variety of connections between RL and inference in probabilistic graphical models (PGMs)…
Abstracts index for ML4H workshop at NeurIPS 2019.
problem No specific problem stated; index of accepted abstracts.
method Not specified; index of accepted abstracts.
result No specific result stated.