A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new method prioritizes and recycles experiences for better reinforcement learning.
This project combines recent advances in experience replay techniques, namely, Combined Experience Replay (CER), Prioritized Experience Replay (PER), and Hindsight Experience Replay (HER). We show the results of combinations of these techniques with DDPG and DQN methods. CER always adds the most recent experience to th…
ReaPER improves learning efficiency by prioritizing reliable experiences.
This paper describes an improvement in Deep Q-learning called Reverse Experience Replay (also RER) that solves the problem of sparse rewards and helps to deal with reward maximizing tasks by sampling transitions successively in reverse order. On tasks with enough experience for training and enough Experience Replay mem…
Unified model optimizes experiment performance and reduces duration.
An important component of many Deep Reinforcement Learning algorithms is the Experience Replay which serves as a storage mechanism or memory of made experiences. These experiences are used for training and help the agent to stably find the perfect trajectory through the problem space. The classic Experience Replay howe…
New method designs experiments robustly for nonlinear estimation, improving parameter knowledge.
Experience reuse is key to sample-efficient reinforcement learning. One of the critical issues is how the experience is represented and stored. Previously, the experience can be stored in the forms of features, individual models, and the average model, each lying at a different granularity. However, new tasks may requi…
Optimal tests developed for sequential experiments with asymptotic properties.
Bayesian optimization for long-term outcomes using fast and slow experiments.
Two methods estimate effect size for online experiments, improving accuracy and efficiency.
Experience replay enables reinforcement learning agents to memorize and reuse past experiences, just as humans replay memories for the situation at hand. Contemporary off-policy algorithms either replay past experiences uniformly or utilize a rule-based replay strategy, which may be sub-optimal. In this work, we consid…
Study finds real-world datasets contain natural experiments that can improve model performance.
We reduce variance in monetization metrics for ranking experiments.
New framework for choosing optimal proxy metrics from past experiments.
Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized experience replay (PER) which samples important transitions more frequently. In this paper, we propo…
LiDER refreshes past experiences in RL by dreaming about them.
We analyze the sample complexity of learning from multiple experiments where the experimenter has a total budget for obtaining samples. In this problem, the learner should choose a hypothesis that performs well with respect to multiple experiments, and their related data distributions. Each collected sample is associat…
The paper improves reinforcement learning stability and efficiency with a new theoretical framework.
Optimal adaptive experiment for choosing best treatment with binary outcomes.
We study the task of retrieving relevant experiments given a query experiment. By experiment, we mean a collection of measurements from a set of `covariates' and the associated `outcomes'. While similar experiments can be retrieved by comparing available `annotations', this approach ignores the valuable information ava…
Expands Bayesian experiment design framework to account for model discrepancies.
ER-GNN uses experience replay to prevent GNNs from forgetting previous tasks.
The paper addresses statistical inference issues in adaptive experiments.
Online review communities are dynamic as users join and leave, adopt new vocabulary, and adapt to evolving trends. Recent work has shown that recommender systems benefit from explicit consideration of user experience. However, prior work assumes a fixed number of discrete experience levels, whereas in reality users gai…
New insights into experience replay in RL algorithms.
Policy gradient reinforcement learning (RL) algorithms have achieved impressive performance in challenging learning tasks such as continuous control, but suffer from high sample complexity. Experience replay is a commonly used approach to improve sample efficiency, but gradient estimators using past trajectories typica…
We address the problem of retrieving relevant experiments given a query experiment, motivated by the public databases of datasets in molecular biology and other experimental sciences, and the need of scientists to relate to earlier work on the level of actual measurement data. Since experiments are inherently noisy and…
Paper uses transfer learning and Bayesian optimization to reduce DNA sequence design experiments.
The paper clusters hypergraphs to find diverse and experienced groups based on past experiences.
The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves …
The logic of uncertainty is not the logic of experience and as well as it is not the logic of chance. It is the logic of experience and chance. Experience and chance are two inseparable poles. These are two dual reflections of one essence, which is called co~event. The theory of experience and chance is the theory of c…
iDAD uses neural networks to quickly adapt experiments without likelihoods.
CEA augments reinforcement learning by generating counterfactual experiences.
IMeL turns RL into SL by interpolating improved experiences.
A framework for efficient multi-objective optimization using entropy search.
New BO methods exploit parallel experiments, reducing search time and improving solution quality.
Random forests classify Pokemon names based on evolutionary status.
We investigate the combination of actor-critic reinforcement learning algorithms with uniform large-scale experience replay and propose solutions for two challenges: (a) efficient actor-critic learning with experience replay (b) stability of off-policy learning where agents learn from other agents behaviour. We employ …
VRER selectively reuses past observations to reduce variance in policy optimization.
Paper proposes a method to use in silico experiments with foundation models to reduce sample size.
A-ICP selects experiments to learn causal effects efficiently.
Inertial confinement fusion (ICF) experiments are designed using computer simulations that are approximations of reality, and therefore must be calibrated to accurately predict experimental observations. In this work, we propose a novel nonlinear technique for calibrating from simulations to experiments, or from low fi…
Equal experience fairness in recommender systems reduces bias.
Reconstructing the causal network in a complex dynamical system plays a crucial role in many applications, from sub-cellular biology to economic systems. Here we focus on inferring gene regulation networks (GRNs) from perturbation or gene deletion experiments. Despite their scientific merit, such perturbation experimen…
Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…
A new score measures data reliability without ground truth.
Current recommender systems exploit user and item similarities by collaborative filtering. Some advanced methods also consider the temporal evolution of item ratings as a global background process. However, all prior methods disregard the individual evolution of a user's experience level and how this is expressed in th…