Agent-based model compares different COVID-19 testing policies and their effectiveness.
problem Understanding how different testing policies reveal the true number of infected cases.
method Developed an agent-based simulation framework in Python to model various testing policies and interventions.
result Contact Tracing consistently captures more positive cases than Random Symptomatic Testing, and LBT performs similarly.
Test-time data augmentation−averaging the predictions of a machine learning model across multiple augmented samples of data−is a widely used technique that improves the predictive performance. While many advanced learnable data augmentation techniques have emerged in recent years, they are focused on the training p…
Study on policy testing in MDPs with lower bounds and new algorithm.
problem Deciding if policy value exceeds a threshold with limited samples.
method Derived lower bound, proposed new algorithm, reformulated problem, used policy optimization in reversed MDP.
result New algorithm outperforms existing methods in policy testing.
Automated testing framework finds weaknesses in deep control policies.
problem Safety of deep neural network control policies is difficult to validate.
method Adversarial reinforcement learning to test and find weaknesses.
result Framework finds weaknesses not evident during manual testing.
CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.
problem Estimating counterfactual policy outcomes for decision-making.
method Counterfactual Policy Mean Embedding (CPME) framework in RKHS, plug-in and doubly robust estimators, kernel test statistic.
result Doubly robust estimator improves convergence rates and asymptotic normality.
Improves policies with high certainty, even in small samples.
problem Ensuring new policies are better than the baseline with high probability.
method Leverages powerful safety tests and multiple testing for threshold policies.
result Controls the rate of adopting a worse policy to pre-specified error level.
Develops CLTs for Markov chain transition probabilities and policies.
problem Estimating transition probabilities and policies in controlled Markov chains.
method Non-parametric estimator for transition matrices; CLTs for value, Q-, and advantage functions; goodness-of-fit tests.
result Asymptotic normality of estimators under specific logging policies.
The paper develops a method to learn cost-optimal sequential testing policies from retrospective data.
problem Learning cost-optimal sequential decision policies from retrospective data with missing test results.
method Doubly robust Q-learning framework with path-specific inverse probability weights.
result The method reduces testing cost without compromising predictive accuracy.
Optimizes COVID-19 testing policy using a Multi-Armed Bandit approach.
problem Balancing discovery of positive cases with population surveillance.
method Risk scoring and random sampling based on Multi-Armed Bandit theory.
result Effective prioritization captures 65-92% of positive cases with varying testing capacity.
Paper introduces effect-invariance for better policy generalization.
problem Adapting policies to unseen environments efficiently.
method Introduces effect-invariance, a relaxation of full invariance, and develops testing procedures to test e-invariance directly from data.
result Effect-invariance enables zero-shot and few-shot policy generalization without assuming a causal graph.
Examines how central bank policies affect stock markets and asset prices.
problem Understanding the impact of monetary policy on stock markets and asset prices.
method Used Taylor rule equations to analyze data from 1990 to 2020 for US and UK, testing with various econometric methods.
result Monetary policy can explain asset price volatility and output gap better than just inflation rate.
As all physical adaptive quantum-enhanced metrology schemes operate under noisy conditions with only partially understood noise characteristics, so a practical control policy must be robust even for unknown noise. We aim to devise a test to evaluate the robustness of AQEM policies and assess the resource used by the po…
Step-DAD improves BED by periodically updating a design policy during experiments.
problem Improving flexibility and robustness in Bayesian experimental design.
method Semi-amortized, policy-based approach that updates a design policy during data collection.
result Consistently superior decision-making and robustness compared to current BED methods.
Paper proposes a sequential statistical test for comparing imitation learning policies with near-optimal stopping.
problem Challenges in rigorously comparing imitation learning policies due to small sample sizes and potential p-hacking.
method Sequential statistical test that adapts the number of trials based on intermediate results, achieving near-optimal stopping.
result Reduces the number of evaluation trials by up to 32% compared to state-of-the-art baselines, saving significant time and effort.
In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an external signal. In the case of continual reinforcement learning a third challenge arises: learning tasks…
The Australian Government uses the means-test as a way of managing the pension budget. Changes in Age Pension policy impose difficulties in retirement modelling due to policy risk, but any major changes tend to be `grandfathered' meaning that current retirees are exempt from the new changes. In 2015, two important chan…
Randomized trials, also known as A/B tests, are used to select between two policies: a control and a treatment. Given a corresponding set of features, we can ideally learn an optimized policy P that maps the A/B test data features to action space and optimizes reward. However, although A/B testing provides an unbiased …
Improved A/B testing by leveraging system similarities.
problem Traditional A/B testing ignores potential system similarities.
method Off-policy estimation to exploit system propensities.
result Improved A/B testing estimators achieve better accuracy.
New method optimizes treatment policies to avoid winner's curse.
problem Winner's curse in treatment policy optimization.
method Inference-aware policy optimization.
result Optimizes for both estimated performance and downstream evaluation.
When recruiting job candidates, employers rarely observe their underlying skill level directly. Instead, they must administer a series of interviews and/or collate other noisy signals in order to estimate the worker's skill. Traditional economics papers address screening models where employers access worker skill via a…
Computer simulation provides an automatic and safe way for training robotic control policies to achieve complex tasks such as locomotion. However, a policy trained in simulation usually does not transfer directly to the real hardware due to the differences between the two environments. Transfer learning using domain ra…
Study evaluates reinforcement learning for trading diverse stocks, finds Q-learning outperforms.
problem Evaluating reinforcement learning for trading diverse stocks.
method Implemented Value Iteration (VI), State-action-reward-state-action (SARSA), and Q-Learning on a diverse stock portfolio dataset.
result Q-learning performs better than VI and SARSA during testing, but performance varies based on market conditions.
The ability to perform effective off-policy learning would revolutionize the process of building better interactive systems, such as search engines and recommendation systems for e-commerce, computational advertising and news. Recent approaches for off-policy evaluation and learning in these settings appear promising. …
We study a policy gradient method with L2 regularization for MAB problems.
problem Improving policy gradient methods for MAB problems with regularization.
method Investigate convergence of a policy gradient algorithm with L2 regularization for MAB.
result Prove convergence under appropriate technical hypotheses and show practical improvements.
Current clinical practice to monitor patients' health follows either regular or heuristic-based lab test (e.g. blood test) scheduling. Such practice not only gives rise to redundant measurements accruing cost, but may even lead to unnecessary patient discomfort. From the computational perspective, heuristic-based test …
This study uses OPE methods to quickly assess auction policies.
problem Rapid decision-making in dynamic auction environments.
method Off-Policy Evaluation and counterfactual methods.
result Improved policy selection and optimization.
A major challenge in reinforcement learning (RL) is the design of agents that are able to generalize across tasks that share common dynamics. A viable solution is meta-reinforcement learning, which identifies common structures among past tasks to be then generalized to new tasks (meta-test). In meta-training, the RL ag…
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …
A method to learn robust policies for environments with model mismatches.
problem Training agents in high-stakes scenarios with mismatched training and real environments.
method Formalizes the perturbation as a zero-sum game to find Nash Equilibrium, which corresponds to the robust policy.
result Our algorithm can find a near-optimal robust policy with high probability using polynomial samples.
The outcome of Jacobian singular values regularization was studied for supervised learning problems. It also was shown that Jacobian conditioning regularization can help to avoid the ``mode-collapse'' problem in Generative Adversarial Networks. In this paper, we try to answer the following question: Can information abo…
Proposes DGCN with trajectory sampling for data-efficient policy search in MBRL.
problem Improving data efficiency in model-based reinforcement learning.
method Combines trajectory sampling and DGCN for uncertainty propagation in probabilistic world models.
result Improves sample-efficiency over other uncertainty propagation methods and probabilistic models.
Improves A/B testing power using a two-armed bandit framework.
problem Comparing outcomes under a new policy to a control.
method Doubly robust estimation, two-armed bandit framework, permutation-based method.
result Superior performance in A/B testing compared to existing methods.
SNPL learns safe policies for multi-objective interventions with high confidence.
problem Designing effective digital interventions balancing multiple objectives with noisy data.
method Leverages algorithmic stability to learn policies with high-confidence guarantees.
result Offers dramatic improvements in safety and policy gains with smaller sample sizes.
We consider a problem of learning the reward and policy from expert examples under unknown dynamics. Our proposed method builds on the framework of generative adversarial networks and introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies. Empowe…
Researchers discuss connections and distinctions between DRITR and Kallus' work, focusing on policy evaluation and efficiency.
problem Policy evaluation and efficiency under distributionally robust individualized treatment rules.
method Efficient policy evaluation using combined data and DRITR under varying sample sizes.
result Efficient value function estimates can deliver competitive performance under certain conditions but not always.
The problem of multi-hypothesis testing with controlled sensing of observations is considered. The distribution of observations collected under each control is assumed to follow a single-parameter exponential family distribution. The goal is to design a policy to find the true hypothesis with minimum expected delay whi…
Successful deployment of machine learning algorithms in healthcare requires careful assessments of their performance and safety. To date, the FDA approves locked algorithms prior to marketing and requires future updates to undergo separate premarket reviews. However, this negates a key feature of machine learning--the …
Off-policy learning in dynamic decision problems is essential for providing strong evidence that a new policy is better than the one in use. But how can we prove superiority without testing the new policy? To answer this question, we introduce the G-SCOPE algorithm that evaluates a new policy based on data generated by…
New method learns adaptive exploration strategies for dynamic tasks.
problem Learning effective exploration strategies in changing environments.
method Informed policy regularization to reduce sample complexity of RNN-based policies.
result Method learns efficient exploration strategies balancing information gathering and reward maximization.
This paper develops q-learning methods for mean-field control problems.
problem Continuous-time mean-field control problems with interaction between agents.
method Introduces two q-functions and devises model-free learning algorithms.
result Developed algorithms can learn optimal value functions and q-functions.
Learning reward functions can lead to poor policy performance despite low error.
problem Low error in learned reward functions does not guarantee low regret in policy performance.
method Mathematical analysis of reward learning and policy optimization.
result A low expected test error of the reward model guarantees low worst-case regret, but error-regret mismatch can occur with certain data distributions.
Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being dep…
The ability to perform offline A/B-testing and off-policy learning using logged contextual bandit feedback is highly desirable in a broad range of applications, including recommender systems, search engines, ad placement, and personalized health care. Both offline A/B-testing and off-policy learning require a counterfa…
Motivated by the widespread adoption of large-scale A/B testing in industry, we propose a new experimentation framework for the setting where potential experiments are abundant (i.e., many hypotheses are available to test), and observations are costly; we refer to this as the experiment-rich regime. Such scenarios requ…
Study optimizes portfolio allocation policies using off-policy data and constraints.
problem Optimizing portfolio allocation policies under constraints using off-policy data.
method Solves a minimax objective with off-policy estimators and online learning to control constraint violations.
result Constructs near-optimal allocation policies for various regimes of operation and constraints.
Novel categories are commonly defined as those unobserved during training but present during testing. However, partially labelled training datasets can contain unlabelled training samples that belong to novel categories, meaning these can be present in training and testing. This research is the first to generalise betw…
Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e.g., ExIt from [2], AlphaGo-Zero from [27]). This new family of algorithms maintains, and alternately optimizes, two policies: a fast, reactive policy (e.g., a deep neural network) deployed at t…
Adversarial attacks can manipulate deep trading policies, compromising their performance.
problem Adversarial attacks can compromise deep reinforcement learning trading policies.
method Developed a threat model and proposed two attack techniques.
result Demonstrated the effectiveness of adversarial attacks against DQN trading agents.