arXiv research

Exploration is a difficult challenge in reinforcement learning and is of prime importance in sparse reward environments. However, many of the state of the art deep reinforcement learning algorithms, that rely on epsilon-greedy, fail on these environments. In such cases, empowerment can serve as an intrinsic reward sign…

New method recovers diverse policies from expert data using state-action pair weighting.

Cumulative entropy regularization introduces a regulatory signal to the reinforcement learning (RL) problem that encourages policies with high-entropy actions, which is equivalent to enforcing small deviations from a uniform reference marginal policy. This has been shown to improve exploration and robustness, and it ta…

We show that many algebraic actions of higher-rank abelian groups on zero-dimensional groups are mutually disjoint. The proofs exploit differences in the entropy geometry arising from subdynamics and a form of Abramov--Rokhlin formula for half-space entropies.

New framework for RL transfer learning with state-action mismatch.

This work merges 3-anchored bundles into 3-Lie algebroids.

The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents…

Recurrent networks learn beliefs from history in partially observable environments.

Mutual Information between agent Actions and environment States (MIAS) quantifies the influence of agent on its environment. Recently, it was found that the maximization of MIAS can be used as an intrinsic motivation for artificial agents. In literature, the term empowerment is used to represent the maximum of MIAS at …

New method learns diverse solutions in reinforcement learning without gradient bias.

For a particular class of backgrounds, equations of motion for string sigma models targeted in mutually dual Poisson-Lie groups are equivalent. This phenomenon is called the Poisson-Lie T-duality. On the level of the corresponding string effective actions, the situation becomes more complicated due to the presence of t…

We present a first attempt to elucidate a theoretical and empirical approach to design the reward provided by a natural language environment to some structure learning agent. To this end, we revisit the Information Theory of unsupervised induction of phrase-structure grammars to characterize the behavior of simulated a…

This paper is a rigorous study of two dual pairs of momentum maps arising in the context of fluid equations whose configuration Lie group is the group of automorphism of a trivial principal bundle, generically called here non-abelian fluids. It is shown that the actions involved are mutually completely orthogonal, whic…

The paper applies information theory to financial markets, improving risk management and asset allocation.

In a previous article, analytic 1-submanifolds had been classified w.r.t. their symmetry under a given regular and separately analytic Lie group action on an analytic manifold. It was shown that such an analytic 1-submanifold is either free or (via the exponential map) analytically diffeomorphic to the unit circle or a…

Study mutual insurance market dynamics using mean field games.

The goal of temporal alignment is to establish time correspondence between two sequences, which has many applications in a variety of areas such as speech processing, bioinformatics, computer vision, and computer graphics. In this paper, we propose a novel temporal alignment method called least-squares dynamic time war…

We prove the existence of a one-parameter family of nearly parallel $G_2$-structures on the manifold $S^3\times \mathbb R^4$, which are mutually non isomorphic and invariant under the cohomogeneity one action of the group $SU(2)^3$. This family connects the two locally homogeneous nearly parallel $G_2$-structures which…

When estimating the relevancy between a query and a document, ranking models largely neglect the mutual information among documents. A common wisdom is that if two documents are similar in terms of the same query, they are more likely to have similar relevance score. To mitigate this problem, in this paper, we propose …

Action-bisimulation learns long-horizon controllability for reinforcement learning.

Learning to cooperate with friends and compete with foes is a key component of multi-agent reinforcement learning. Typically to do so, one requires access to either a model of or interaction with the other agent(s). Here we show how to learn effective strategies for cooperation and competition in an asymmetric informat…

Proposes a new framework for resource-limited recommendation.

Analytic curves are classified w.r.t. their symmetry under a regular and separately analytic Lie group action on an analytic manifold. We show that an analytic curve is either exponential or splits into countably many analytic immersive curves, each of them discretely generated by the symmetry group (i.e., each such cu…

Reinforcement learning algorithms struggle when the reward signal is very sparse. In these cases, naive random exploration methods essentially rely on a random walk to stumble onto a rewarding state. Recent works utilize intrinsic motivation to guide the exploration via generative models, predictive forward models, or …

Improved statistical efficiency of Thompson Sampling for combinatorial semi-bandits.

This text presents some basic notions in symplectic geometry, Poisson geometry, Hamiltonian systems, Lie algebras and Lie groups actions on symplectic or Poisson manifolds, momentum maps and their use for the reduction of Hamiltonian systems. It should be accessible to readers with a general knowledge of basic notions …

New neural network approach using mutual information.

Paper describes profiles of multivariate normal distributions and novel estimators for mutual information.

Paper benchmarks mutual info estimators on diverse distributions.

We propose a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning (MARL), through rewarding agents for having causal influence over other agents' actions. Causal influence is assessed using counterfactual reasoning. At each timestep, an agent simulates alternate actions t…

Neural estimator improves mutual information estimation in high dimensions.

We propose a new method to study the internal memory used by reinforcement learning policies. We estimate the amount of relevant past information by estimating mutual information between behavior histories and the current action of an agent. We perform this estimation in the passive setting, that is, we do not interven…

Improved bounds on learning algorithms' performance using conditional mutual information.

We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance…

The paper argues that normalized mutual information is biased in clustering and community detection.

Synthetic construction of 3D complex bases.

Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confide…

Deep RL learns to construct objects from 2D images by avoiding brick overlaps.

This study analyzes mutual influence on investment strategies of financial market agents.

We find the maximum mutual information for neural networks and its key determinants.

We introduce the Mutual Information Machine (MIM), a novel formulation of representation learning, using a joint distribution over the observations and latent state in an encoder/decoder framework. Our key principles are symmetry and mutual information, where symmetry encourages the encoder and decoder to learn differe…

A new method estimates mutual information using neural classifiers.

New estimator improves mutual information estimation.

Study examines Indian equity mutual funds' investment style and risk-shifting.

Proposes a framework to maximize mutual information in VAE models for better latent code representation.

The paper proposes a method to learn structured representations from unlabeled data using mutual information maximization.

Improved method for encoding contingency tables reduces mutual information bias.

A deep reinforcement learning approach for slate re-ranking in e-commerce.