Lab experiment reveals market imitation and win-stay lose-shift patterns in financial decision-making.
problem Understanding how people make decisions in financial markets.
method Lab-in-the-field experiment with financial information, statistical analysis, and cohort analysis.
result Market imitation and win-stay lose-shift strategies emerge as dominant behaviors in financial decision-making.
A new method prioritizes and recycles experiences for better reinforcement learning.
problem Improving reinforcement learning efficiency by prioritizing and recycling experiences.
method Double-prioritized state-recycled (DPSR) experience replay.
result DPSR achieved state-of-the-art results in Atari games, outperforming original and prioritized methods.
Reverse Experience Replay improves Deep Q-learning for sparse rewards.
problem Sparse rewards and reward-maximizing tasks in Deep Q-learning.
method Sampling transitions in reverse order for training.
result Significantly increased performance in tasks with limited experience and memory capacity.
Combines experience replay techniques to improve reinforcement learning.
problem Improving reinforcement learning algorithms.
method Combines CER, PER, and HER with DDPG and DQN.
result Effective combinations of these techniques in various environments.
ReaPER improves learning efficiency by prioritizing reliable experiences.
problem Inefficient sampling of past experiences in reinforcement learning.
method Introducing a novel measure of reliability to prioritize experiences in PER.
result ReaPER outperforms PER in various environments, including Atari-10.
Algorithm creates synthetic experiences to enhance Deep Reinforcement Learning.
problem Lack of synthetic experiences in classic Experience Replay.
method Bootstraps synthetic experiences to augment the replay memory.
result Synthetic experiences can improve learning speed and performance.
Unified model optimizes experiment performance and reduces duration.
problem Balancing reward maximization and experiment termination.
method Unified model that considers both within-experiment and post-experiment outcomes.
result Familiar algorithms can optimize a broad class of objectives with proper parameter adjustment.
Paper proposes PRR network for better experience reuse in reinforcement learning.
problem Efficient experience reuse in reinforcement learning across multiple granularities.
method Proposes PRR network trained on multi-level architecture to extract and store experience.
result PRR network leads to better experience reuse and improved performance.
New method designs experiments robustly for nonlinear estimation, improving parameter knowledge.
problem Designing robust experiments for nonlinear estimation under parametric uncertainty.
method Multi-stage robust optimization framework for sequential experiments.
result Identifies experiments better conducted early for improved parameter knowledge.
A novel framework optimizes experience replay for reinforcement learning.
problem Optimizing the replay policy for better performance in off-policy reinforcement learning.
method Proposes a novel ERO framework that alternately updates the agent policy and replay policy.
result Demonstrates improved performance in various continuous control tasks.
Optimal experiment design reduces unknown structure in fixed experiments.
problem Learning causal structure from limited experiments.
method Characterizes optimal learning strategy, designs experiments efficiently.
result Proposed algorithm is ρ-approximation for bounded degree graphs.
Improved learning from multiple experiments with cost constraints.
problem Learning from multiple experiments with cost constraints.
method Rademacher complexity approach to show gap between training and generalization error.
result The gap between training and generalization error is O(C−1/2). Optimal tests developed for sequential experiments with asymptotic properties.
problem Performing hypothesis tests after sequential experiments without prior design.
method Analyze asymptotic properties of sequential experiments; develop tests for Gaussian process observations.
result Asymptotic power function of any test can be matched by a specific test in a limit experiment.
Bayesian optimization for long-term outcomes using fast and slow experiments.
problem Optimizing long-term system effects with short-term misleading results.
method Combining fast and slow experiments for Bayesian optimization.
result Sequential optimization over large action spaces in a short time.
Two methods estimate effect size for online experiments, improving accuracy and efficiency.
problem Determining the correct effect size for online experiment duration.
method Two approaches: hierarchical models and utility theory.
result Proposed methods outperform baseline approaches in accuracy and efficiency.
Study finds real-world datasets contain natural experiments that can improve model performance.
problem Detecting natural experiments in real-world datasets for causal inference.
method Synthetic graph simulation and feature selection based on causal links.
result Real-world datasets contain natural experiments that can be exploited for improved model performance.
This paper models continuous user experience evolution for better item recommendations.
problem Dynamic user experience in online review communities.
method Combines Geometric Brownian Motion, Brownian Motion, and Latent Dirichlet Allocation to model continuous user experience and language evolution.
result The model outperforms discrete models and state-of-the-art methods in predicting item ratings.
A new method for prioritizing sequences in reinforcement learning improves learning efficiency.
problem Efficient learning in reinforcement learning algorithms.
method Prioritized Sequence Experience Replay (PSER) for prioritizing sequences of experience.
result PSER is guaranteed to converge faster than PER and substantially improves performance.
We reduce variance in monetization metrics for ranking experiments.
problem Heavy-tailed monetization metrics lead to unreliable conclusions in A/B experiments.
method Post-stratification combined with CUPED.
result Significant reduction in variance and improved decision stability.
AES improves policy gradient performance by adaptively selecting experience.
problem High variance in gradient estimators from past trajectories.
method AES learns an adaptive sampling distribution to minimise gradient variance.
result AES leads to significantly improved performance in continuous control tasks.
New framework for choosing optimal proxy metrics from past experiments.
problem Difficult to measure long-term treatment effects in experiments.
method Statistical framework to define and construct optimal proxy metrics.
result Optimal proxy metric depends on experiment's sample size.
Improves item recommendations by considering user experience evolution.
problem Current recommender systems ignore user experience evolution.
method Developed a generative HMM-LDA model to trace user evolution and interest facets.
result Significantly improved rating prediction over state-of-the-art baselines.
LiDER refreshes past experiences in RL by dreaming about them.
problem Improving data efficiency in off-policy RL algorithms.
method Refreshing past experiences in a replay buffer using the current policy.
result LiDER consistently improves performance in Atari games.
Efficient actor-critic learning with shared experience replay improves data efficiency.
problem Challenges in actor-critic reinforcement learning with experience replay and off-policy learning stability.
method Combining actor-critic algorithms with shared experience replay, analyzing V-trace, proposing a trust region scheme.
result State-of-the-art data efficiency on Atari achieved with 200M environment frames.
The paper improves reinforcement learning stability and efficiency with a new theoretical framework.
problem Stability and efficiency in reinforcement learning, especially in data-scarce scenarios.
method Theoretical framework using resampled U- and V-statistics to model experience replay, applied to policy evaluation and kernel ridge regression. result Significant improvements in stability and efficiency, particularly in data-scarce scenarios.
Optimal adaptive experiment for choosing best treatment with binary outcomes.
problem Choosing the best treatment from binary options in an adaptive experiment.
method Adaptive experiment with two phases: treatment allocation and choice. Neyman allocation method used.
result Neyman allocation is minimax and Bayes optimal, matching lower bounds for regret.
We study the task of retrieving relevant experiments given a query experiment. By experiment, we mean a collection of measurements from a set of `covariates' and the associated `outcomes'. While similar experiments can be retrieved by comparing available `annotations', this approach ignores the valuable information ava…
FRD protects privacy in distributed RL by sharing proxy experience memory.
problem Privacy violation in exchanging experience memory in distributed RL.
method Proposes FRD framework using proxy experience memory.
result Numerical evaluation shows FRD is effective and performance depends on proxy memory structure.
Expands Bayesian experiment design framework to account for model discrepancies.
problem Model misspecification in Bayesian optimal experiment design.
method Introduces Expected General Information Gain and Expected Discriminatory Information criteria.
result Demonstrates improved robustness and detection capabilities in experiment design.
Optimal testing framework for many experiments with costly observations.
problem Optimal testing in scenarios with many hypotheses and limited observations.
method Characterized and computed the optimal policy for sampling experiments, developed a heuristic.
result High-powered classical tests can be inefficient in the experiment-rich regime.
ER-GNN uses experience replay to prevent GNNs from forgetting previous tasks.
problem Catastrophic forgetting in GNNs when learning multiple tasks sequentially.
method Experience Replay framework to store and replay knowledge from previous tasks.
result ER-GNN effectively mitigates catastrophic forgetting in GNNs.
WALL-E accelerates reinforcement learning by parallelizing experience collection.
problem Slow experience collection time in reinforcement learning.
method Parallel rollout samplers to speed up experience generation.
result Achieved higher average rewards with parallel processes.
The paper addresses statistical inference issues in adaptive experiments.
problem Statistical inference problems in adaptive experiments.
method Explains and fixes statistical inference issues in adaptive experiments using various methods.
result Various methods to stabilize inferences and recover asymptotic normality.
Combines experience replay and exploration for better agent performance.
problem Improving exploration efficiency and robustness in reinforcement learning.
method Integrates Intrinsic Rewards with Prioritized Oversampled Experience Replay (POER).
result Achieves better agent performance and sample efficiency compared to PPO/RND.
Bayesian optimization improves machine learning system tuning with online and offline experiments.
problem Limited simultaneous experiments in complex policy spaces.
method Augment online field experiments with an offline simulator and apply multi-task Bayesian optimization.
result Substantial gains from including biased offline data in live machine learning systems.
ReF-ER improves RL by selectively updating experiences based on policy similarity.
problem Deterioration of experience replay accuracy when policies diverge.
method Enforces policy similarity in replay memory by skipping unlikely experiences and regulating policy changes.
result ReF-ER consistently improves RL performance across various benchmarks.
RIDS reconstructs sparse gene regulatory networks from few perturbation experiments.
problem Inference of gene regulatory networks from costly perturbation experiments.
method Robust IDentification of Sparse networks (RIDS) method using sparse optimization.
result RIDS can reconstruct GRNs from a small number of experiments, achieving high performance.
New insights into experience replay in RL algorithms.
problem Understanding the impact of replay capacity and replay ratio in Q-learning.
method Systematic and extensive analysis of experience replay in Q-learning methods, focusing on replay capacity and replay ratio.
result Greater replay capacity significantly improves performance for certain algorithms, while other techniques offer limited benefit.
DeepDIVA simplifies reproducible deep learning experiments.
problem Difficulty in reproducing deep learning research results.
method A framework for easy experimentation and reproduction.
result Facilitates sharing and easy experimentation of experiments.
Optimizes staggered treatment rollouts to minimize cost and error.
problem Efficiently scheduling treatment initiation times for staggered rollouts.
method Non-adaptive and adaptive experimental designs, including a near-optimal solution for non-adaptive cases and a new algorithm for adaptive cases.
result Reduces experiment cost by over 50% compared to static design benchmarks.
We address the problem of retrieving relevant experiments given a query experiment, motivated by the public databases of datasets in molecular biology and other experimental sciences, and the need of scientists to relate to earlier work on the level of actual measurement data. Since experiments are inherently noisy and…
Paper uses transfer learning and Bayesian optimization to reduce DNA sequence design experiments.
problem Designing many similar DNA sequences for specific applications is expensive and time-consuming.
method Combines transfer learning with Bayesian optimization to reduce experiment count.
result Total number of experiments can be significantly reduced by sharing information between tasks.
The paper clusters hypergraphs to find diverse and experienced groups based on past experiences.
problem Finding diverse and experienced groups with respect to past experiences.
method Regularized edge-based hypergraph clustering objective with a 2-approximation algorithm.
result Demonstrates an efficient 2-approximation algorithm for clustering hypergraphs.
New method uses many experiments to learn causal effects, reducing bias with regularization.
problem Learning causal relationships from large collections of small-effect randomized experiments.
method Regularized instrumental variables (l0 regularization) and modified cross-validation (IVCV).
result Reduces bias in interventional predictions, improving accuracy.
The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves …
Estimates effect sizes and power from a pilot experiment.
problem Estimating the distribution of effect sizes in multiple testing settings.
method Uses an inexpensive pilot experiment to estimate effect sizes and power.
result Simple and computationally efficient estimator guarantees the number of discoveries.
Machine learning improves quantum experiment design.
problem Designing complex quantum experiments to produce high-dimensional entangled states.
method Projective simulation model, machine learning to create new experiments.
result Machine learning system autonomously discovers experimental techniques.
iDAD uses neural networks to quickly adapt experiments without likelihoods.
problem Performing adaptive experiments in real-time with implicit models.
method iDAD learns a design policy network upfront to make quick design decisions.
result iDAD can make design decisions in milliseconds, unlike traditional BOED approaches.