Neural Index Policy for multi-action bandits with heterogeneous budgets.
problem Real-world settings often involve multiple interventions with heterogeneous costs and constraints, breaking classical assumptions.
method Introduces a Neural Index Policy (NIP) that learns to assign budget-aware indices to arm-action pairs using a neural network and differentiable knapsack layer.
result Empirically achieves near-optimal performance while strictly enforcing heterogeneous budgets and scaling to hundreds of arms.
Paper tackles optimal policy learning with observational data in multi-action scenarios.
problem Optimal policy learning in multi-action settings with observational data.
method Review of estimation approaches, analysis of risk preference, discussion of potential failures.
result Average regret of a policy with multi-valued treatment is contingent on the decision-maker's attitude towards risk.
Lower bounds for PI on multi-action MDPs are established, showing complexity grows with action count.
problem Establishing the minimum number of iterations for PI to converge on MDPs with multiple actions.
method Developed lower bounds for a specific PI variant on multi-action MDPs, scaling with action count.
result A particular PI variant can take Ω(kn/2) iterations to terminate, scaling with action count. In many settings, a decision-maker wishes to learn a rule, or policy, that maps from observable characteristics of an individual to an action. Examples include selecting offers, prices, advertisements, or emails to send to consumers, as well as the problem of determining which medication to prescribe to a patient. Whil…
Optimal policy for multi-armed multi-action bandits with unknown parameters.
problem Optimal sequential action selection for multi-armed multi-action bandits with unknown parameters.
method Occupancy-Measured-Reward Index Policy (OMRIP) and R(MA)^2B-UCB algorithm.
result Asymptotically optimal policy with sub-linear regret and low computational complexity.
Study proves duality in exotic option pricing under uncertain model and delayed information.
problem Pricing and hedging of multi-action exotic options under nondominated model uncertainty and delayed information.
method Reformulated superhedging problem as a European option problem, proving duality results.
result Superhedging price equals model-based price with future look-up power.
In this work we describe a novel deep reinforcement learning architecture that allows multiple actions to be selected at every time-step in an efficient manner. Multi-action policies allow complex behaviours to be learnt that would otherwise be hard to achieve when using single action selection techniques. We use both …
Policy learning can be used to extract individualized treatment regimes from observational data in healthcare, civics, e-commerce, and beyond. One big hurdle to policy learning is a commonplace lack of overlap in the data for different actions, which can lead to unwieldy policy evaluation and poorly performing learned …
ARL and Hawkes processes improve market-making strategies with variable volatility.
problem Enhancing market-making strategies to adapt to varying volatility levels and self-exciting behaviors.
method Integrates ARL, Hawkes processes, and variable volatility levels; shifts from Poisson to Hawkes process.
result 4-action MM trained in low-volatility environment adapts to high-volatility conditions, providing stable performance.
Paper derives policy rules from observational data for hepatitis C treatment.
problem Improving treatment guidelines for HIV/HCV co-infected patients.
method Weighted K-means algorithm for estimating CATEs, decision tree implementation.
result Identifies a subgroup with high spontaneous HCV clearance rate.