arXiv research

Algorithm learns optimal coordination for strategic agents in uncertain settings.

This paper derives a portfolio decomposition formula when the agent maximizes utility of her wealth at some finite planning horizon. The financial market is complete and consists of multiple risky assets (stocks) plus a risk free asset. The stocks are modelled as exponential Brownian motions with drift and volatility b…

This paper shows how diverse tasks can make inefficient exploration in MTRL efficient.

A minimal model of a market of myopic non-cooperative agents who trade bilaterally with random bids reproduces qualitative features of short-term electric power markets, such as those in California and New England. Each agent knows its own budget and preferences but not those of any other agent. The near-equilibrium pr…

New algorithm learns optimal policies in strategic MDPs with private types.

Modeling the purposeful behavior of imperfect agents from a small number of observations is a challenging task. When restricted to the single-agent decision-theoretic setting, inverse optimal control techniques assume that observed behavior is an approximately optimal solution to an unknown decision problem. These tech…

New $k$-step policy gradient method avoids local optima in restricted policy classes.

Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent receives a reward. However, finding an equilibrium (if exists) in this game is often …

Study optimal treatment assignment policies under strategic agent responses.

Global optimization in Bayesian inference yields little additional benefit.

Many recommendation algorithms rely on user data to generate recommendations. However, these recommendations also affect the data obtained from future users. This work aims to understand the effects of this dynamic interaction. We propose a simple model where users with heterogeneous preferences arrive over time. Based…

Myopic procedures are shown to be asymptotically optimal in ranking and selection problems.

Paper confirms Feldman's conjecture on two-armed bandit problem.

Myopic investors make suboptimal choices that benefit others, leading to market inefficiencies.

New approach incentivizes strategic agents to explore, making exploration almost free.

Study on predictable forward processes in trading without frequent evaluations.

The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…

Myopic optimization outperforms reinforcement learning in portfolio management, leading to lower returns and higher risks.

Efficiently optimizes constrained problems with two-step lookahead BO.

This paper combines LLMs with RL for better trading strategies.

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL a…

Study shows how adaptive market agents can lead to persistent overpricing in financial markets.

Incentive-aware recommender system for online platforms.

Action-bisimulation learns long-horizon controllability for reinforcement learning.

This paper optimizes sampling policies for Bayesian optimization to improve exploration and exploitation.

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic program (DP) that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-sp…

Improves data efficiency in multi-agent control tasks using model-based reinforcement learning.

We study online learning settings in which experts act strategically to maximize their influence on the learning algorithm's predictions by potentially misreporting their beliefs about a sequence of binary events. Our goal is twofold. First, we want the learning algorithm to be no-regret with respect to the best fixed …

We introduce a microscopic model of interacting financial agents, where each agent is characterized by two portfolios; money invested in bonds and money invested in stocks. Furthermore, each agent is faced with an optimization problem in order to determine the optimal asset allocation. The stock price evolution is driv…

Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can be incremental and slow. Recent work demonstrated that using a memory buffer of previous successful trajectories can result in more effective…

We propose a contextual bandit based model to capture the learning and social welfare goals of a web platform in the presence of myopic users. By using payments to incentivize these agents to explore different items/recommendations, we show how the platform can learn the inherent attributes of items and achieve a subli…

New RL algorithms find SNE in Markov games with myopic followers.

Efficiently recovers network community structure from clients' small subgraphs.

We maximize the expected utility from terminal wealth for an HARA investor when the market price of risk is an unobservable random variable. We compute the optimal portfolio explicitly and explore the effects of learning by comparing it with the corresponding myopic policy. In particular, we show that, for a market pri…

We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal. Our approach, Myopic Posterior Sampling (MPS), is inspired by the classical posterior (Thompson) sampling algorithm for multi-armed…

We present a simple agent-based model to study the development of a bubble and the consequential crash and investigate how their proximate triggering factor might relate to their fundamental mechanism, and vice versa. Our agents invest according to their opinion on future price movements, which is based on three source…

IDS improves reinforcement learning with contextual information.

Portfolio turnpikes state that, as the investment horizon increases, optimal portfolios for generic utilities converge to those of isoelastic utilities. This paper proves three kinds of turnpikes. In a general semimartingale setting, the abstract turnpike states that optimal final payoffs and portfolios converge under …

Optimizes long-term social welfare in recommender systems by matching users to providers.

The paper calculates how fast optimal investment strategies approach CRRA strategies in stochastic factor models.

NM-PPG optimizes adaptive feature acquisition in POMDPs for better predictions.

We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…

Study shows bifurcating price dynamics in ASME with traders.

Empirical study shows carriers ignore past shippers' behavior, focusing only on current actions.

New method optimizes costly functions with unknown costs and budget constraints.

Finite-horizon sequential experimental design (SED) arises naturally in many contexts, including hyperparameter tuning in machine learning among more traditional settings. Computing the optimal policy for such problems requires solving Bellman equations, which are generally intractable. Most existing work resorts to se…

A meta-learning approach for efficient algorithm selection in budget-limited scenarios.

Efficiently reduces computational burden of rollout acquisition functions in Bayesian optimization.