arXiv research

TensorPlan algorithm finds δ-optimal policies with poly$(H,d)$ queries under linearly realizable state-value function.

New method improves deep policy gradient algorithms by learning relative state values.

An Atlas model is a rank-based system of continuous semimartingales for which the steady-state values of the processes follow a power law, or Pareto distribution. For a power law, the log-log plot of these steady-state values versus rank is a straight line. Zipf's law is a power law for which the slope of this line is …

Brain uses synaptic failure to sample from posterior distributions.

Generative model handles varying data dimensions using jump diffusion processes.

Enhances financial market valuation and trading algorithms using distributional value functions.

Compressed imitation learning uses simplicity priors for efficient expert behavior copying.

Improves BC policies by generating new plausible trajectories.

We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yieldi…

RWR converges to global optimum in certain settings.

We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal di…

Improves LSTM performance by initializing states via manifold learning.

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep $\mathcal{Q}$-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the …

Algorithm finds latent structure in value functions for improved reinforcement learning.

Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorithms and theory that provide strong problem-dependent bounds could help illuminate the key features of what makes a RL problem hard and reduc…

This paper makes one step forward towards characterizing a new family of \textit{model-free} Deep Reinforcement Learning (DRL) algorithms. The aim of these algorithms is to jointly learn an approximation of the state-value function ($V$), alongside an approximation of the state-action value function ($Q$). Our analysis…

Unified q-learning for mean-field jump-diffusion models with unobservable population distribution.

Distribution and sample models are two popular model choices in model-based reinforcement learning (MBRL). However, learning these models can be intractable, particularly when the state and action spaces are large. Expectation models, on the other hand, are relatively easier to learn due to their compactness and have a…

Transfer in Reinforcement Learning (RL) refers to the idea of applying knowledge gained from previous tasks to solving related tasks. Learning a universal value function (Schaul et al., 2015), which generalizes over goals and states, has previously been shown to be useful for transfer. However, successor features are b…

In this work we present a modification in the conventional flow of information through a LSTM network, which we consider well suited for RNNs in general. The modification leads to a iterative scheme where the computations performed by the LSTM cell are repeated over a constant input and cell state values, while updatin…

This paper presents a new meta-modeling framework to employ deep reinforcement learning (DRL) to generate mechanical constitutive models for interfaces. The constitutive models are conceptualized as information flow in directed graphs. The process of writing constitutive models are simplified as a sequence of forming g…

New RL method improves on standard discounted RL for operations research.

New RL approach uses resets to learn complex MDPs efficiently.

ReF-ER algorithm improved performance in multi-agent reinforcement learning.

Algorithm identifies best policy in MDPs with adaptive sampling.

The width $w$ of a curve $γ$ in Euclidean space $R^n$ is the infimum of the distances between all pairs of parallel hyperplanes which bound $γ$, while its inradius $r$ is the supremum of the radii of all spheres which are contained in the convex hull of $γ$ and are disjoint from $γ$. We use a mixture of topological and…

A new algorithm finds optimal solutions for constrained decision processes.

Paper tackles stochastic control with mean and higher-order moments, finding Nash equilibria.

Current imitation learning techniques are too restrictive because they require the agent and expert to share the same action space. However, oftentimes agents that act differently from the expert can solve the task just as good. For example, a person lifting a box can be imitated by a ceiling mounted robot or a desktop…

Improves RL from historical data by stitching trajectories.

Deep reinforcement learning for high dimensional, hierarchical control tasks usually requires the use of complex neural networks as functional approximators, which can lead to inefficiency, instability and even divergence in the training process. Here, we introduce stacked deep Q learning (SDQL), a flexible modularized…

The complex, time-dependent statistical structures observed in the Dow Jones Industrial Average on a typical trading day are modeled with Lorentzian functions. The resonant-like structures are characterized by the values of the basic ratio: the average lifetime of the individual states associated with a given structura…

Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL. On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori. However, in complex environments where transition models need t…

We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…

Network slicing is a key technology in 5G communications system. Its purpose is to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware resource allocation is of significant importance to network slicin…

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …

New method learns state embeddings from demonstrations for improved reinforcement learning.

MoNODEs improve neural ODEs by separating dynamic states from static factors.

MuZero visualizes its internal representations to stabilize planning.

In this paper, we provide a unified analysis of temporal difference learning algorithms with linear function approximators by exploiting their connections to Markov jump linear systems (MJLS). We tailor the MJLS theory developed in the control community to characterize the exact behaviors of the first and second order …

Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.

Unified framework for solving MDPs with stochastic mirror descent.

New method finds unseen states for RL, improving performance.

Transformers learn to predict chess moves with surprising accuracy and strength.

A new framework enhances IDW models for complex industrial datasets.

Activities in reinforcement learning (RL) revolve around learning the Markov decision process (MDP) model, in particular, the following parameters: state values, V; state-action values, Q; and policy, pi. These parameters are commonly implemented as an array. Scaling up the problem means scaling up the size of the arra…

The Linear Attention Recurrent Neural Network (LARNN) is a recurrent attention module derived from the Long Short-Term Memory (LSTM) cell and ideas from the consciousness Recurrent Neural Network (RNN). Yes, it LARNNs. The LARNN uses attention on its past cell state values for a limited window size $k$. The formulas ar…

New neural network models for complex functional data analysis.