arXiv research

This paper shows how diverse tasks can make inefficient exploration in MTRL efficient.

This paper optimizes sampling policies for Bayesian optimization to improve exploration and exploitation.

We maximize the expected utility from terminal wealth for an HARA investor when the market price of risk is an unobservable random variable. We compute the optimal portfolio explicitly and explore the effects of learning by comparing it with the corresponding myopic policy. In particular, we show that, for a market pri…

Proposes a model to incentivize exploration in web platforms with payments.

New $k$-step policy gradient method avoids local optima in restricted policy classes.

Global optimization in Bayesian inference yields little additional benefit.

Myopic procedures are shown to be asymptotically optimal in ranking and selection problems.

Paper confirms Feldman's conjecture on two-armed bandit problem.

Finite-horizon sequential experimental design (SED) arises naturally in many contexts, including hyperparameter tuning in machine learning among more traditional settings. Computing the optimal policy for such problems requires solving Bellman equations, which are generally intractable. Most existing work resorts to se…

Myopic investors make suboptimal choices that benefit others, leading to market inefficiencies.

Myopic optimization outperforms reinforcement learning in portfolio management, leading to lower returns and higher risks.

Efficiently optimizes constrained problems with two-step lookahead BO.

New method optimizes costly functions with unknown costs and budget constraints.

The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic program (DP) that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-sp…

New RL algorithms find SNE in Markov games with myopic followers.

Efficiently recovers network community structure from clients' small subgraphs.

We consider the problem of learning the functions computing children from parents in a Structural Causal Model once the underlying causal graph has been identified. This is in some sense the second step after causal discovery. Taking a probabilistic approach to estimating these functions, we derive a natural myopic act…

A new Bayesian method optimizes time-dependent expensive functions with lookahead.

We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal. Our approach, Myopic Posterior Sampling (MPS), is inspired by the classical posterior (Thompson) sampling algorithm for multi-armed…

We consider the problem faced by a service platform that needs to match limited supply with demand but also to learn the attributes of new users in order to match them better in the future. We introduce a benchmark model with heterogeneous "workers" (demand) and a limited supply of "jobs" that arrive over time. Job typ…

New methods improve global optimisation for expensive functions using lookahead strategies.

New approach incentivizes strategic agents to explore, making exploration almost free.

Algorithm learns optimal coordination for strategic agents in uncertain settings.

This paper derives a portfolio decomposition formula when the agent maximizes utility of her wealth at some finite planning horizon. The financial market is complete and consists of multiple risky assets (stocks) plus a risk free asset. The stocks are modelled as exponential Brownian motions with drift and volatility b…

Portfolio turnpikes state that, as the investment horizon increases, optimal portfolios for generic utilities converge to those of isoelastic utilities. This paper proves three kinds of turnpikes. In a general semimartingale setting, the abstract turnpike states that optimal final payoffs and portfolios converge under …

Bayesian methods improve drug discovery experiment design.

Proposes qPO, a new acquisition strategy for batched Bayesian optimization that maximizes the probability of including the optimum.

The paper calculates how fast optimal investment strategies approach CRRA strategies in stochastic factor models.

NM-PPG optimizes adaptive feature acquisition in POMDPs for better predictions.

We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…

Empirical study shows carriers ignore past shippers' behavior, focusing only on current actions.

Study shows bifurcating price dynamics in ASME with traders.

Efficiently reduces computational burden of rollout acquisition functions in Bayesian optimization.

We consider two active binary-classification problems with atypical objectives. In the first, active search, our goal is to actively uncover as many members of a given class as possible. In the second, active surveying, our goal is to actively query points to ultimately predict the proportion of a given class. Numerous…

ePF improves PF for ITS by balancing exploration and exploitation, outperforming baselines.

Study on predictable forward processes in trading without frequent evaluations.

We provide a new characterization of mean-variance hedging strategies in a general semimartingale market. The key point is the introduction of a new probability measure $P^{\star}$ which turns the dynamic asset allocation problem into a myopic one. The minimal martingale measure relative to $P^{\star}$ coincides with t…

Study improves policy search in continuous control by using heavy-tailed distributions.

PFNs4BO uses neural processes for flexible Bayesian Optimization.

This paper studies the properties of discrete time stochastic optimal control problems associated with portfolio selection. We investigate if optimal continuous time strategies can be used effectively for a discrete time market after a straightforward discretization. We found that Merton's strategy approximates the per…

Incentive-aware recommender system for online platforms.

Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can be incremental and slow. Recent work demonstrated that using a memory buffer of previous successful trajectories can result in more effective…

The design of multiple experiments is commonly undertaken via suboptimal strategies, such as batch (open-loop) design that omits feedback or greedy (myopic) design that does not account for future effects. This paper introduces new strategies for the optimal design of sequential experiments. First, we rigorously formul…

New algorithms optimize time series classification speed and accuracy.

Study optimal portfolio strategy with sporadic bankruptcy for isoelastic utility.

Many recommendation algorithms rely on user data to generate recommendations. However, these recommendations also affect the data obtained from future users. This work aims to understand the effects of this dynamic interaction. We propose a simple model where users with heterogeneous preferences arrive over time. Based…

A method to combine saliency metrics for better CNN pruning decisions.