A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new method shapes reinforcement learning environments by abstracting large state spaces.
This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.
Building models, or maps, of robot environments is a highly active research area; however, most existing techniques construct unstructured maps and assume static environments. In this paper, we present an algorithm for learning object models of non-stationary objects found in office-type environments. Our algorithm exp…
Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.
Self-supervised reward prediction improves RL in sparse reward settings.
In this work we present a technique to use natural language to help reinforcement learning generalize to unseen environments. This technique uses neural machine translation, specifically the use of encoder-decoder networks, to learn associations between natural language behavior descriptions and state-action informatio…
Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the ag…
Universal spaces for finite topological spaces simplify shape descriptions.
Evolution of planar curves under a nonlocal geometric equation is investigated. It models the simultaneous contraction and growth of carbonate particles called ooids in geosciences. Using classical ODE results and a bijective mapping we demonstrate that the steady parameters associated with the physical environment det…
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
Many practical environments contain catastrophic states that an optimal agent would visit infrequently or never. Even on toy problems, Deep Reinforcement Learning (DRL) agents tend to periodically revisit these states upon forgetting their existence under a new policy. We introduce intrinsic fear (IF), a learned reward…
While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward shaping method through predictive coding to tackle sparse reward problems. By learning predictive repr…
Accurate estimation of the positions and shapes of microscale objects is crucial for automated imaging-guided manipulation using a non-contact technique such as optical tweezers. Perception methods that use traditional computer vision algorithms tend to fail when the manipulation environments are crowded. In this paper…
T-Basis represents neural network tensors with fewer parameters.
When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environments. We show that a predictive algorithm with an expressive generative model can form stable belief-…
Catastrophic forgetting continues to severely restrict the learnability of controllers suitable for multiple task environments. Efforts to combat catastrophic forgetting reported in the literature to date have focused on how control systems can be updated more rapidly, hastening their adjustment from good initial setti…
We propose a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing cooperative tasks in partially-observable environments. This targeting behavior is learnt solely from downstream task-specific reward withou…
Reinforcement learning usually uses the feedback rewards of environmental to train agents. But the rewards in the actual environment are sparse, and even some environments will not rewards. Most of the current methods are difficult to get good performance in sparse reward or non-reward environments. Although using shap…
Building agents that can explore their environments intelligently is a challenging open problem. In this paper, we make a step towards understanding how a hierarchical design of the agent's policy can affect its exploration capabilities. First, we design EscapeRoom environments, where the agent must figure out how to n…
This article extends, in a stochastic environment, the Yagil (1987) model which establishes, in a deterministic dividend discount model, a range for the exchange ratio in a stock-for-stock merger agreement. Here, we generalize Yagil's work letting both pre- and post-merger dividends grow randomly over time. If Yagil fo…
A new reward shaping method balances learning efficiency and effectiveness for robot manipulation.
Model-free reinforcement learning (RL) requires a large number of trials to learn a good policy, especially in environments with sparse rewards. We explore a method to improve the sample efficiency when we have access to demonstrations. Our approach, Backplay, uses a single demonstration to construct a curriculum for a…
Financial models shape markets through performativity, creating self-fulfilling prophecies.
Improved reinforcement learning with deep learning.
Traditional reinforcement learning agents learn from experience, past or present, gained through interaction with their environment. Our approach synthesizes experience, without requiring an agent to interact with their environment, by asking the policy directly "Are there situations X, Y, and Z, such that in these sit…
Essential to each other, growth and exploration are jointly observed in populations, be it alive such as animals and cells or inanimate such as goods and money. But their ability to move, crucial to cope with uncertainty and optimize returns, is tempered by the space/time properties of the environment. We investigate h…
WDAIL uses Wasserstein distance for more effective reward shaping in IL.
Microfluidic devices are utilized to control and direct flow behavior in a wide variety of applications, particularly in medical diagnostics. A particularly popular form of microfluidics -- called inertial microfluidic flow sculpting -- involves placing a sequence of pillars to controllably deform an initial flow field…
An agent learning through interactions should balance its action selection process between probing the environment to discover new rewards and using the information acquired in the past to adopt useful behaviour. This trade-off is usually obtained by perturbing either the agent's actions (e.g., e-greedy or Gibbs sampli…
SIBRE boosts reinforcement learning convergence by rewarding improvement over past performance.
Proposes PIC and POIC for measuring task difficulty in RL.
Study how untrained policies explore in RL environments.
Our understanding of reinforcement learning (RL) has been shaped by theoretical and empirical results that were obtained decades ago using tabular representations and linear function approximators. These results suggest that RL methods that use temporal differencing (TD) are superior to direct Monte Carlo estimation (M…
Analyzes Indian chemical industry post-Covid.
Agents learn to give rewards to others in a shared learning environment.
New RL approach infers optimal policies via variational inference.
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproduc…
Interprets how intrinsic motivation shapes behavior in RL agents.
NVMDP framework tackles non-stationary MDPs with varying discount rates.
In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement lea…
Reinforcement learning requires manual specification of a reward function to learn a task. While in principle this reward function only needs to specify the task goal, in practice reinforcement learning can be very time-consuming or even infeasible unless the reward function is shaped so as to provide a smooth gradient…
Reinforcement learning methods require careful design involving a reward function to obtain the desired action policy for a given task. In the absence of hand-crafted reward functions, prior work on the topic has proposed several methods for reward estimation by using expert state trajectories and action pairs. However…
New framework promotes reproducible, domain-agnostic reinforcement learning algorithms.
The study of dexterous manipulation has provided important insights in humans sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here we describe a method using the deformation and color distribution of the finge…
With millimeter wave wireless communications, the resulting radiation reflects on most visible objects, creating rich multipath environments, namely in urban scenarios. The radiation captured by a listening device is thus shaped by the obstacles encountered, which carry latent information regarding their relative posit…
The LLS stock market model is a model of heterogeneous quasi-rational investors operating in a complex environment about which they have incomplete information. We review the main features of this model and several of its extensions. We study the effects of investor heterogeneity and show that predation, competition, o…
The use of Reinforcement Learning (RL) is still restricted to simulation or to enhance human-operated systems through recommendations. Real-world environments (e.g. industrial robots or power grids) are generally designed with safety constraints in mind implemented in the shape of valid actions masks or contingency con…
Policy shifts between Trump and Biden impact ESG investments, creating volatility.