Combines experience replay techniques to improve reinforcement learning.
problem Improving reinforcement learning algorithms.
method Combines CER, PER, and HER with DDPG and DQN.
result Effective combinations of these techniques in various environments.
ReaPER improves learning efficiency by prioritizing reliable experiences.
problem Inefficient sampling of past experiences in reinforcement learning.
method Introducing a novel measure of reliability to prioritize experiences in PER.
result ReaPER outperforms PER in various environments, including Atari-10.
Reverse Experience Replay improves Deep Q-learning for sparse rewards.
problem Sparse rewards and reward-maximizing tasks in Deep Q-learning.
method Sampling transitions in reverse order for training.
result Significantly increased performance in tasks with limited experience and memory capacity.
A new method for prioritizing sequences in reinforcement learning improves learning efficiency.
problem Efficient learning in reinforcement learning algorithms.
method Prioritized Sequence Experience Replay (PSER) for prioritizing sequences of experience.
result PSER is guaranteed to converge faster than PER and substantially improves performance.
Algorithm creates synthetic experiences to enhance Deep Reinforcement Learning.
problem Lack of synthetic experiences in classic Experience Replay.
method Bootstraps synthetic experiences to augment the replay memory.
result Synthetic experiences can improve learning speed and performance.
A novel framework optimizes experience replay for reinforcement learning.
problem Optimizing the replay policy for better performance in off-policy reinforcement learning.
method Proposes a novel ERO framework that alternately updates the agent policy and replay policy.
result Demonstrates improved performance in various continuous control tasks.
Improved learning from multiple experiments with cost constraints.
problem Learning from multiple experiments with cost constraints.
method Rademacher complexity approach to show gap between training and generalization error.
result The gap between training and generalization error is O(C−1/2). A new method prioritizes and recycles experiences for better reinforcement learning.
problem Improving reinforcement learning efficiency by prioritizing and recycling experiences.
method Double-prioritized state-recycled (DPSR) experience replay.
result DPSR achieved state-of-the-art results in Atari games, outperforming original and prioritized methods.
Paper proposes PRR network for better experience reuse in reinforcement learning.
problem Efficient experience reuse in reinforcement learning across multiple granularities.
method Proposes PRR network trained on multi-level architecture to extract and store experience.
result PRR network leads to better experience reuse and improved performance.
Framework uses experience replay to prevent deep networks from forgetting past tasks.
problem Deep networks forget past tasks after learning new ones in sequential multitask learning.
method Generative model that couples current task with past learned tasks through a discriminative embedding space.
result Framework learns a shared abstract distribution across all tasks, preventing catastrophic forgetting.
Bayesian optimization improves machine learning system tuning with online and offline experiments.
problem Limited simultaneous experiments in complex policy spaces.
method Augment online field experiments with an offline simulator and apply multi-task Bayesian optimization.
result Substantial gains from including biased offline data in live machine learning systems.
Current evaluations of continual learning are flawed and misleading.
problem Flawed experiment designs in current evaluations.
method Examine and propose new experiment designs.
result New experiment designs reveal true performance of continual learning approaches.
DeepDIVA simplifies reproducible deep learning experiments.
problem Difficulty in reproducing deep learning research results.
method A framework for easy experimentation and reproduction.
result Facilitates sharing and easy experimentation of experiments.
Efficient actor-critic learning with shared experience replay improves data efficiency.
problem Challenges in actor-critic reinforcement learning with experience replay and off-policy learning stability.
method Combining actor-critic algorithms with shared experience replay, analyzing V-trace, proposing a trust region scheme.
result State-of-the-art data efficiency on Atari achieved with 200M environment frames.
AES improves policy gradient performance by adaptively selecting experience.
problem High variance in gradient estimators from past trajectories.
method AES learns an adaptive sampling distribution to minimise gradient variance.
result AES leads to significantly improved performance in continuous control tasks.
IMeL turns RL into SL by interpolating improved experiences.
problem Improving reinforcement learning efficiency and scalability.
method IMeL uses a reservoir of experiences and a NN regressor for interpolation.
result IMeL achieves preliminary results and proposes itself as a baseline.
ER-GNN uses experience replay to prevent GNNs from forgetting previous tasks.
problem Catastrophic forgetting in GNNs when learning multiple tasks sequentially.
method Experience Replay framework to store and replay knowledge from previous tasks.
result ER-GNN effectively mitigates catastrophic forgetting in GNNs.
Paper uses transfer learning and Bayesian optimization to reduce DNA sequence design experiments.
problem Designing many similar DNA sequences for specific applications is expensive and time-consuming.
method Combines transfer learning with Bayesian optimization to reduce experiment count.
result Total number of experiments can be significantly reduced by sharing information between tasks.
CEA augments reinforcement learning by generating counterfactual experiences.
problem Challenges in reinforcement learning, especially out-of-distribution and inefficient exploration.
method CEA uses variational autoencoders to model state transitions and introduces randomness for non-stationarity. It expands learning data through counterfactual inference.
result CEA outperforms SOTA algorithms in diverse environments.
The paper studies how memory replay affects reinforcement learning performance.
problem Understanding the effects of memory size and prioritization in experience replay.
method Formulated a dynamical systems ODE model of Q-learning with experience replay, derived analytic solutions, and proposed an adaptive memory buffer size algorithm.
result The amount of memory kept and prioritization significantly impact learning dynamics; too much or too little memory slows down learning.
The paper improves reinforcement learning stability and efficiency with a new theoretical framework.
problem Stability and efficiency in reinforcement learning, especially in data-scarce scenarios.
method Theoretical framework using resampled U- and V-statistics to model experience replay, applied to policy evaluation and kernel ridge regression. result Significant improvements in stability and efficiency, particularly in data-scarce scenarios.
WALL-E accelerates reinforcement learning by parallelizing experience collection.
problem Slow experience collection time in reinforcement learning.
method Parallel rollout samplers to speed up experience generation.
result Achieved higher average rewards with parallel processes.
New insights into experience replay in RL algorithms.
problem Understanding the impact of replay capacity and replay ratio in Q-learning.
method Systematic and extensive analysis of experience replay in Q-learning methods, focusing on replay capacity and replay ratio.
result Greater replay capacity significantly improves performance for certain algorithms, while other techniques offer limited benefit.
The paper investigates the effectiveness of reusing experience in Deep Q-Learning for FPS environments.
problem The high number of interactions required for reinforcement learning limits its practicality.
method The authors test the effectiveness of applying learning update steps multiple times per environmental step in the VizDoom environment.
result Updating learning steps less frequently than every 4th environmental step does not improve performance and can degrade performance.
Machine learning boosts physics research, especially at high energy experiments.
problem Finding new fundamental physics in high energy experiments.
method Review of machine learning methods and applications in high energy physics.
result Modern machine learning techniques have expanded the scope of physics research.
FRD protects privacy in distributed RL by sharing proxy experience memory.
problem Privacy violation in exchanging experience memory in distributed RL.
method Proposes FRD framework using proxy experience memory.
result Numerical evaluation shows FRD is effective and performance depends on proxy memory structure.
Transfer learning improves fusion simulation accuracy.
problem Calibrate fusion simulation models to experimental data.
method Hierarchical transfer learning using deep neural networks.
result Calibrated models predict Omega experiments more accurately.
New algorithm improves continual learning by balancing transfer and interference.
problem Challenges in scaling neural network learning to non-stationary data.
method Proposes a new conceptualization of continual learning as a trade-off between transfer and interference, and develops Meta-Experience Replay (MER) to optimize this.
result Demonstrates consistent improvement over recent baselines in continual learning benchmarks.
ReF-ER improves RL by selectively updating experiences based on policy similarity.
problem Deterioration of experience replay accuracy when policies diverge.
method Enforces policy similarity in replay memory by skipping unlikely experiences and regulating policy changes.
result ReF-ER consistently improves RL performance across various benchmarks.
RER improves sample complexity by updating in reverse order.
problem Theoretical analysis limits RER's convergence rate.
method Tighter analysis for larger learning rates and longer sequences.
result RER converges faster with larger learning rates and longer sequences.
Synthetic experiments are crucial for assessing causal machine learning methods.
problem Current empirical evaluations of causal machine learning methods are insufficient and unreliable.
method Propose principles for conducting rigorous empirical analyses with synthetic data.
result Rigorous synthetic experiments are essential for building trust in causal machine learning methods.
Machine learning experiments often contain errors, especially in confusion matrices and statistical tests.
problem Errors in machine learning experiments, particularly in confusion matrices and statistical tests.
method Analyzed 49 papers describing 2456 experiments, checking for errors in confusion matrices and statistical significance.
result 22 out of 49 papers contain demonstrable errors, with 7 statistical and 16 related to confusion matrix inconsistency.
Algorithm learns new tasks efficiently from past experience.
problem Lack of robustness to distributional shift in meta-reinforcement learning.
method Model Identification and Experience Relabeling (MIER) using dynamics models.
result Efficient extrapolation to out-of-distribution tasks.
Machine learning experiments often mislead due to unmet assumptions.
problem Machine learning experiments with pooled data may not meet necessary assumptions for unbiased causal effect estimation.
method Analysis of assumptions required for unbiased causal effect estimation in machine learning experiments.
result Practical applications of A/B-tests with machine learning models may not yield unbiased estimates of causal effect.
Machine learning, specifically LSTM, models quantum experiments efficiently.
problem Modeling complex quantum states with high-dimensional entanglement.
method Used a long short-term memory (LSTM) neural network to predict quantum experiment outcomes.
result LSTM neural networks can accurately predict quantum experiment outcomes without computing the states themselves.
A method using competitive experience replay enhances learning from sparse rewards.
problem Learning from sparse rewards in reinforcement learning.
method Competitive experience replay method that augments sparse rewards through an exploration competition between agents.
result The method leads to faster convergence and improved task performance.
GBS uses machine learning to design products based on consumer preferences.
problem Designing products to meet consumer preferences.
method GBS is a discrete choice experiment that uses machine learning to adaptively construct paired comparison questions.
result GBS outperforms existing methods in accuracy and sample efficiency.
VRER selectively reuses past observations to reduce variance in policy optimization.
problem Lack of effective experience replay for accelerating policy optimization in complex systems.
method Variance Reduction Experience Replay (VRER) framework that selectively reuses informative samples.
result VRER reduces gradient variance and improves policy learning over state-of-the-art algorithms.
A-ICP selects experiments to learn causal effects efficiently.
problem Learning causal effects from observational data is difficult.
method Active learning framework based on Invariant Causal Prediction.
result Proposes intervention selection policies to reveal direct causes.
Experience replay helps neural networks learn new tasks without forgetting old knowledge.
problem Catastrophic forgetting in neural networks trained on non-stationary data.
method Experience replay buffers with a mixture of on- and off-policy learning.
result Experience replay can learn new tasks quickly and reduce catastrophic forgetting.
ORIL learns a reward function from unlabeled data to improve robot learning.
problem Leveraging unlabeled data for robot learning.
method ORIL learns a reward function from demonstrator and unlabeled trajectories, annotates data, and trains an agent via offline reinforcement learning.
result ORIL consistently outperforms BC agents on various robotic tasks.
CDP improves RL performance and sample-efficiency by prioritizing rare goal states.
problem Learning from imbalanced data in RL.
method Curiosity-Driven Prioritization (CDP) framework.
result CDP improves both performance and sample-efficiency of RL agents.
ACTRCE uses natural language to improve reinforcement learning performance.
problem Sparse reward in reinforcement learning.
method Extends HER framework with natural language goal representation.
result ACTRCE can solve challenging 3D navigation tasks and generalize to unseen instructions.
Algorithm learns shared demand structure across dynamic pricing experiments.
problem Learning shared demand parameters across multiple dynamic pricing experiments.
method Meta dynamic pricing algorithm that learns prior online while solving Thompson sampling experiments.
result Algorithm achieves sublinear meta regret in experiment-rich environments.
How useful can machine learning be in a quantum laboratory? Here we raise the question of the potential of intelligent machines in the context of scientific research. A major motivation for the present work is the unknown reachability of various entanglement classes in quantum experiments. We investigate this question …
Automated PDE discovery from multiple noisy experiments.
problem Inherent variability in experiments makes single experiment inference unreliable.
method Randomised adaptive group Lasso sparsity estimator in deep learning framework.
result More generalizable PDEs found from multiple datasets.
Algorithm learns feature representations from randomized experiments to improve counterfactual inferences.
problem Measuring the impact of interventions with limited feedback.
method Feature learning algorithm from randomized experiments to identify effective and ineffective interventions.
result The algorithm leverages feature representations to derive the value of interventions for each instance, improving decision-making.
Unsupervised machine learning helps design complex experiments more efficiently.
problem Designing experiments with many factors and constraints is challenging and costly.
method Applied a beta variational autoencoder (beta-VAE) to represent trials in a low-dimensional latent space.
result Generated pragmatic designs with fewer trials while maintaining objectives.