The paper introduces MDP homomorphic networks for faster reinforcement learning.
problem Current reinforcement learning approaches do not exploit symmetries in the joint state-action space.
method Equivariant neural networks with group-structured symmetries (reflections, rotations).
result MDP homomorphic networks converge faster than unstructured baselines on various tasks.
New algorithm learns abstractions from experience to simplify complex tasks.
problem Solving complex problems in environments with continuous state spaces.
method Uses MDP homomorphisms to find abstract MDPs and guides exploration.
result Demonstrates task transfer method outperforms deep Q-networks.
This work uses action equivariance to learn structured latent spaces for reinforcement learning.
problem Learning structured latent spaces for reinforcement learning.
method Introduced a contrastive loss function to enforce action equivariance on learned representations.
result Optimal policies in the abstract MDP can be successfully lifted to the original MDP.
This paper optimizes MDP policies for efficient state aggregation.
problem Optimizing policies in aggregated Markov chains while preserving optimal performance.
method Homomorphic mappings to establish optimal policy equivalence and derive performance bounds.
result Developed HPG and EBHPG methods for efficient aggregation and policy optimization.
Characterizes a general range decreasing group homomorphism.
problem Understanding range decreasing group homomorphisms in the entire mapping group.
method Characterization of a general range decreasing group homomorphism.
result Computes a particular class of homomorphisms and identifies all range decreasing group homomorphisms on specific mapping groups.
We extend certain homomorphisms defined on the higher Torelli subgroups of the mapping class group to crossed homomorphisms defined on the entire mapping class group. In particular, for every k ≥ 2 k\geq 2 k ≥ 2 , we construct a crossed homomorphism ε k ε_k ε k which extends Morita's homomorphism τ ~ k \tilde τ_k τ ~ k to the entire mapping clas…
Adaptive reduction scheme approximates optimal policy in regularized MDPs.
problem Finding near optimal policy in regularized MDPs with biased solutions.
method Adaptive reduction of regularization parameter λ to approximate optimal policy.
result Iteration complexity reduced for obtaining ε-optimal policy.
We study Exo-MDPs to reduce sample complexity in reinforcement learning.
problem Reducing sample complexity in reinforcement learning for structured MDPs.
method Introducing Exo-MDPs and proving structural equivalence to linear mixture MDPs, establishing regret bounds.
result Proved O ( H 3 / 2 d K ) O(H^{3/2}d\sqrt{K}) O ( H 3/2 d K ) regret bound for Exo-MDPs, matching lower bounds. In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…
Two crossing homomorphisms on braid groups are shown to be equivalent.
problem Comparing two definitions of crossing homomorphisms on braid groups.
method Diagrammatic and algebraic definitions of crossing homomorphisms compared and computed for simple braids.
result Diagrammatic and algebraic crossing homomorphisms are equivalent.
The study classifies homomorphisms from mapping class groups using finite subgroups.
problem Classifying homomorphisms from mapping class groups.
method Using finite subgroups to classify homomorphisms.
result Only finitely many mapping class groups have non-trivial homomorphisms into Homeo(S^n) for any n.
Paper establishes new lower bounds for MDPs with changing transition kernels.
problem Minimizing sample complexity and regret in non-stationary MDPs.
method Developed novel lower bounds and constructed hard MDPs.
result Proved Ω ( ( H 3 S A / ε 2 ) log ( 1 / δ ) ) Ω((H^3SA/ε^2)\log(1/δ)) Ω (( H 3 S A / ε 2 ) log ( 1/ δ )) sample complexity lower bound. Study of Chebyshev-Frobenius homomorphism in 3-manifold skein modules.
problem Exploring the Chebyshev-Frobenius homomorphism in 3-manifold skein modules.
method Generalization of splitting homomorphism for stated skein modules of 3-manifolds.
result Existence and properties of Chebyshev-Frobenius homomorphism for 3-manifold skein modules.
ARL algorithm reduces adversarial MDP to bandit problems for reliable policy learning.
problem Learning reliable policies in non-stationary, adversarial MDPs.
method Adversarial Reinforcement Learning (ARL) algorithm that converts MDP to a sequence of adversarial bandit problems.
result Achieves optimal regret bound of O ( S A T H 3 ) O(\sqrt{SATH^3}) O ( S A T H 3 ) . Graph homomorphism numbers embed graphs for classification.
problem Graph classification using graph homomorphisms.
method Embed graphs into vectors using homomorphism numbers.
result Homomorphism vectors are universal for approximating graph invariants.
New conditions for weighted composition operators in group homomorphisms.
problem Conditions for weighted composition operators in group homomorphisms.
method Range decreasing group homomorphisms.
result New insights into weighted composition operators and their algebraic structure.
Deep RL solves combinatorial selection problems with large item spaces.
problem Solving MDPs with large state and action spaces, especially for combinatorial selection.
method Convert S-MDP to IS-MDP, use weight-shared Q-networks to manage state space explosion.
result Our approach effectively handles large item spaces and scales to diverse environments.
New method approximates POMDPs with PB-MDPs, providing error bounds and practical algorithms.
problem Difficulty in solving POMDPs with continuous or hybrid state and observation spaces.
method Bounding particle filtering error and adapting MDP algorithms to POMDPs.
result General theory and practical algorithms for POMDPs with no direct dependence on state and observation space sizes.
We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic programming for small to medium sized problems. However, due to the "curse of dimen…
A new method for CMDP solving without compromising safety constraints.
problem Solving CMDP problems while adhering to safety constraints.
method Decomposition into reconnaissance and planning MDPs.
result Achieves safe policies for any safety constraint set.
Study homomorphisms from groups to 3-manifold fundamental groups.
problem Understanding homomorphisms between groups and 3-manifold fundamental groups.
method Proves foundational results and answers specific questions.
result Answers questions posed by Reid-Wang-Zhou and Agol-Liu.
New homomorphisms from knot Floer homology help classify knots.
problem Classifying knots based on their concordance properties.
method Defined an infinite family of concordance homomorphisms using knot Floer complexes.
result Explicitly computable homomorphisms that are linearly independent.
Optimizes learning policies in MDPs with weakly communicating structure.
problem Learning optimal policies in weakly communicating MDPs with generative model.
method Span-based approach, reducing to discounted MDPs for analysis.
result First minimax optimal sample complexity bound for weakly communicating MDPs.
New homomorphism from Khovanov homology for knot concordance.
problem Understanding smooth concordance of knots.
method Modulo equivalence relation on Khovanov chain complex.
result Strictly stronger than Rasmussen invariants.
Stability of Lie group homomorphisms and subgroups via Moser type argument.
problem When a deformation of Lie group homomorphisms and subgroups is trivial.
method Moser type argument for compact groups.
result Stability results for compact Lie groups.
New method removes oracle and reduces memory usage for robust MDPs.
problem Applying robust MDPs in practice due to model estimation and oracle requirements.
method Transformed robust MDPs into an alternative form allowing stochastic gradient methods and model-free approach.
result Sample-efficient algorithm with lower storage requirement and no oracle.
Classifies homomorphisms from braid groups, proving their extensions to automorphisms.
problem Classifying homomorphisms from commutator subgroups of braid groups.
method Theory of totally symmetric sets.
result Each nontrivial homomorphism extends to an automorphism of the braid group.
New RL method learns to skip states in linearly q π q^π q π -realizable MDPs, simplifying to linear MDPs.
problem Online RL in episodic MDPs with linearly q π q^π q π -realizable action-values. method Derives a novel algorithm that learns to skip states and applies a linear MDP algorithm.
result First polynomial-sample-complexity online RL algorithm for linearly q π q^π q π -realizable MDPs. Classifies homomorphisms between specific braid groups.
problem Classifying homomorphisms between braid groups.
method Complete classification through recursive approach.
result Recursive classification of homomorphisms between braid groups.
DeepAveragers solves offline RL by solving derived MDPs from static data.
problem Offline reinforcement learning with limited data.
method Solves derived non-parametric MDPs (DAC-MDPs) using deep representations and costs for under-represented parts.
result The approach can lower-bound performance and scale to complex offline RL problems.
We introduce the notion of tight homomorphism into a locally compact group with nonvanishing bounded cohomology and study these homomorphisms in detail when the target is a Lie group of Hermitian type. Tight homomorphisms between Lie groups of Hermitian type give rise to tight totally geodesic maps of Hermitian symmetr…
A chord index homomorphism for knots in thickened surfaces is constructed.
problem Knot invariants in thickened surfaces.
method Constructing a chord index homomorphism from a subgroup of H 1 ( Σ , Z ) H_1(Σ, \mathbb{Z}) H 1 ( Σ , Z ) to chord indices of a knot K K K in Σ i m e s I Σ imes I Σ im es I . result Derived knot invariants from the homomorphism.
Study classifies biharmonic and harmonic homomorphisms between specific Lie groups.
problem Classifying biharmonic and harmonic homomorphisms between Riemannian three-dimensional unimodular Lie groups.
method Classification based on left invariant Riemannian metrics.
result Classification of biharmonic and harmonic homomorphisms between specific Lie groups.
Paper surveys Johnson homomorphisms and related tools.
problem Understanding generalized Johnson homomorphisms and their stable images.
method Surveying and unifying various related threads in literature using Hodge theory.
result Clarification of existing results and relationships among Johnson homomorphisms.
Study on Euler class and flux homomorphisms for non-orientable surfaces.
problem Investigate Euler class and flux homomorphisms for non-orientable surfaces.
method Analyze Euler class and flux homomorphisms for non-orientable compact surfaces with one boundary component.
result Prove the simplicity of the kernel of the flux homomorphisms, implying the non-existence of invariants analogous to the Calabi invariant.
We examine functorial and homotopy properties of the exotic characteristic homomorphism in the category of Lie algebroids which was lastly obtained by the authors in [4]. This homomorphism depends on a triple (A,B, ∇ \nabla ∇ ) where B ⊂ \subset ⊂ A are regular Lie algebroids, both over the same regular foliated manifold (M,…
Optimizes learning policies in average-reward MDPs with improved sample complexity.
problem Learning optimal policies in average-reward MDPs with limited samples.
method Reduces to discounted MDPs and uses improved bounds for variance parameters.
result Establishes minimax optimal sample complexity bound of O(SA(H/ε^2))
Optimistic algorithms achieve logarithmic regret bounds for MDPs without diameter dependence.
problem Achieving logarithmic regret bounds for episodic MDPs without relying on diameter-like quantities.
method Novel 'clipped' regret decomposition applied to optimistic algorithms.
result Smooth interpolation between gap-dependent and minimax rates of convergence.
We extend each higher Johnson homomorphism to a crossed homomorphism from the automorphism group of a finite-rank free group to a finite-rank abelian group. We also extend each Morita homomorphism to a crossed homomorphism from the mapping class group of once-bounded surface to a finite-rank abelian group. This improve…
A new homomorphism connects group actions on circles to Euler classes.
problem Understanding group actions on circles and their implications.
method Using crossed homomorphisms and Poincaré translation numbers.
result Relates the Euler class of actions to a specific homomorphism.
The paper disproves a conjecture about satellite maps not inducing homomorphisms.
problem Satellite maps and their impact on knot concordance groups.
method Casson-Gordon signatures and n n n -solvable filtration analysis. result Examples of satellite maps that act like homomorphisms but do not induce them.
We consider bundle homomorphisms between tangent distributions and vector bundles of the same rank. We study the conditions for fundamental singularities when the bundle homomorphism is induced from a Morin map. When the tangent distribution is the contact structure, we characterize singularities of the bundle homomorp…
We propose an approach to study non-Abelian Iwasawa theory, using the idea of Johnson homomorphisms in low dimensional topology. We introduce arithmetic analogues of Johnson homomorphisms/maps, called the p-Johnson homomorphisms/maps, associated to the Zassenhaus filtration of a pro-p Galois group over a Z_p-extension …
We solve POMDPs by approximating them as finite-state MDPs.
problem Computational challenges in learning optimal policies for POMDPs.
method Transform POMDP into a Superstate MDP, apply TD-learning and policy optimization.
result Finite-time bounds on TD-learning error for non-Markovian dynamics.
Efficiently plans large MDPs with weak function approximations.
problem Planning in large MDPs with limited function approximation capabilities.
method Uses linear value function approximation with weak requirements and a generative oracle.
result Produces almost-optimal actions for any state with polynomial computation time.
Satellite operations with winding number ≠ 1 are not homomorphisms.
problem Characterizing homomorphisms in satellite operations.
method Using d d d -invariants of branched covers and Torelli group properties. result Satellite operations with winding number ≠ 1 are not homomorphisms.
Study of handlebody group intersections with Torelli and Johnson kernels.
problem Intersection of handlebody group with Torelli and Johnson kernels.
method Use Birman--Craggs--Johnson (BCJ) homomorphism to study intersections and compute cup products.
result Determine images of BCJ homomorphism restricted to handlebody group intersections and compute cup products.
Reward suffices for convex MDPs, expanding RL to new problems.
problem Capturing goals as convex functions of stationary distribution.
method Reformulated as a min-max game using Fenchel duality.
result Convex MDPs require non-stationary reward functions.