arXiv research

Compressed imitation learning uses simplicity priors for efficient expert behavior copying.

Simplifies RL training with fewer techniques, reducing bias and instability.

We introduce Implicit Policy, a general class of expressive policies that can flexibly represent complex action distributions in reinforcement learning, with efficient algorithms to compute entropy regularized policy gradients. We empirically show that, despite its simplicity in implementation, entropy regularization c…

OSIRIS reduces variance in off-policy evaluation by omitting irrelevant states.

Policy gradient is a generic and flexible reinforcement learning approach that generally enjoys simplicity in analysis, implementation, and deployment. In the last few decades, this approach has been extensively advanced for fully observable environments. In this paper, we generalize a variety of these advances to part…

Study introduces a new framework for policy learning without positivity assumption.

In many areas, practitioners seek to use observational data to learn a treatment assignment policy that satisfies application-specific constraints, such as budget, fairness, simplicity, or other functional form constraints. For example, policies may be restricted to take the form of decision trees based on a limited se…

A new reinforcement learning method reduces action complexity for robust control.

This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tr…

The paper presents a model-free method for stabilizing unknown control systems.

The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning. This process, referred to as distillation, has been used to great success, for example, by enhancing the optimisation of agents, leading to stronger performance faster, on harder domains [26, 32, 5, 8]. Despite the…

SPReD uses uncertainty to decide imitation from demonstrations.

Generically learns movement control policies from exploration data.

Tabular Q-learning outperforms advanced RL methods in monetary policy.

Enhances large language models' reasoning through simpler off-policy reinforcement learning.

Theoretical analysis confirms non-conservative algorithms can converge to optimal policies.

Paper tackles active learning for GNNs, reducing annotation costs.

There has recently been a surge in research in batch Deep Reinforcement Learning (DRL), which aims for learning a high-performing policy from a given dataset without additional interactions with the environment. We propose a new algorithm, Best-Action Imitation Learning (BAIL), which strives for both simplicity and per…

The paper develops formulas for hyperbolic simplices based on edge lengths.

We aim to develop off-policy DRL algorithms that not only exceed state-of-the-art performance but are also simple and minimalistic. For standard continuous control benchmarks, Soft Actor-Critic (SAC), which employs entropy maximization, currently provides state-of-the-art performance. We first demonstrate that the entr…

Estimates dimensions of maximal simplices for rational and irrational trees in Outer space.

The paper establishes conditions for Riemannian connections and semi-simplicity of Lie algebras using spray structures.

It is proved that the volume of spherical or hyperbolic simplices, when considered as a function of the dihedral angles, can be extended continuously to degenerated simplices.

This paper introduces a set of algorithms for Monte-Carlo Bayesian reinforcement learning. Firstly, Monte-Carlo estimation of upper bounds on the Bayes-optimal value function is employed to construct an optimistic policy. Secondly, gradient-based algorithms for approximate upper and lower bounds are introduced. Finally…

Risk management in dynamic decision problems is a primary concern in many fields, including financial investment, autonomous driving, and healthcare. The mean-variance function is one of the most widely used objective functions in risk management due to its simplicity and interpretability. Existing algorithms for mean-…

In this paper, we present an online reinforcement learning algorithm, called Renewal Monte Carlo (RMC), for infinite horizon Markov decision processes with a designated start state. RMC is a Monte Carlo algorithm and retains the advantages of Monte Carlo methods including low bias, simplicity, and ease of implementatio…

Geodesic simplices in pseudo-hyperbolic space get a cohomological treatment.

Study PL bordism theories with quantitative bounds on filling simplices.

Framework reduces simplicity bias in NNs, improving OOD generalization and robustness.

New framework shows $C^*$-simplicity for groups without certain subalgebras.

In this article, we prove a theorem comparing the dihedral angles of simplices in the hyperbolic, spherical and Euclidean geometries.

A fair policy for hiring candidates from different groups is proposed in a linear contextual bandit problem.

We study a natural intrinsic definition of geometric simplices in Riemannian manifolds of arbitrary dimension $n$, and exploit these simplices to obtain criteria for triangulating compact Riemannian manifolds. These geometric simplices are defined using Karcher means. Given a finite set of vertices in a convex set on t…

Research reveals simplicity bias in random logistic map, impacting data analysis and forecasting.

The Apollonius theorem is generalized for m-simplices, with applications in geometry and optimization.

Similar simplices can be inscribed in most smoothly embedded spheres.

We study prismatics sets analogously to simplical sets except that realization involves prisms, i.e., products of simplices rather than just simplices. Particular examples are the prismatic subdivision of a simplicial set S and the prismatic star of S. Both have the same homotopy type as S and in particular the latter …

Simplicial sets deformation retract onto transverse simplices.

We generalize the very well known boundary operator of the ordinary singular homology theory, defined in many books about algebraic topology. We describe a variant of this ordinary simplicial boundary operator where the usual boundary (n-1)-simplices of each n-simplex are replaced by combinations of internal (n-1)- sim…

The paper explores how simplicity leads to better out-of-distribution generalization in models.

Bayesian approach improves $\varepsilon$-greedy exploration in RL.

The study reveals simplicity bias in neural networks leading to better compositional mappings.

Study on simplicity of Lie skew braces, proving new results for compact cases.

Triangulations of R^n have at least tensor rank of determinant simplices.

Two-layer networks favor simple features, especially in complex datasets.

Adam avoids simplicity bias in neural networks, leading to better generalization.

We give several new criteria to judge whether a simple convex polytope in a Euclidean space is combinatorially equivalent to a product of simplices. These criteria are mixtures of combinatorial, geometrical and topological conditions that are inspired by the ideas from toric topology.

Neural nets learn simple distributions first, then more complex ones.