arXiv research

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…

Paper tackles constrained bandit problems with a new learning framework.

Portfolio allocation is crucial for investment companies. However, getting the best strategy in a complex and dynamic stock market is challenging. In this paper, we propose a novel Adaptive Deep Deterministic Reinforcement Learning scheme (Adaptive DDPG) for the portfolio allocation task, which incorporates optimistic …

New algorithm for reinforcement learning in uncertain environments with unknown thresholds.

New method minimizes regret in AMDP with high probability.

An algorithm for maximizing rewards under linear cost constraints.

Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art performance. However, wide-scale adoption of these methods in real-world domains is made difficult by their poor sample efficiency. We address th…

In the present paper a model of a market consisting of real and financial interacting sectors is studied. Agents populating the stock market are assumed to be not able to observe the true underlying fundamental, and their beliefs are biased by either optimism or pessimism. Depending on the relevance they give to belief…

Robust Optimization has traditionally taken a pessimistic, or worst-case viewpoint of uncertainty which is motivated by a desire to find sets of optimal policies that maintain feasibility under a variety of operating conditions. In this paper, we explore an optimistic, or best-case view of uncertainty and show that it …

This study tackles adversarial corruption in model-based reinforcement learning.

The regularization path of the Lasso can be shown to be piecewise linear, making it possible to "follow" and explicitly compute the entire path. We analyze in this paper this popular strategy, and prove that its worst case complexity is exponential in the number of variables. We then oppose this pessimistic result to a…

We consider a Bayesian financial market with one bond and one stock where the aim is to maximize the expected power utility from terminal wealth. The solution of this problem is known, however there are some conjectures in the literature about the long-term behavior of the optimal strategy. In this paper we prove now t…

Paper optimizes GAIL for online and offline learning with linear approximations.

Safe linear bandits over unknown polytopes avoid safety violations and suboptimal actions.

Investor expectations shifted pessimistically during the 2020 stock market crash and recovery.

Computer science scans LLMs to understand and manipulate their economic forecasts.

LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.

We reformulate the Cont-Bouchaud model of financial markets in terms of classical "super-spins" where the spin value is a measure of the number of individual traders represented by a portfolio manager of an investment agency. We then extend this simplified model by switching on interactions among the super-spins to mod…

New algorithm reduces high-probability regret for time-varying feedback graphs.

Adaptive data analysis is frequently criticized for its pessimistic generalization guarantees. The source of these pessimistic bounds is a model that permits arbitrary, possibly adversarial analysts that optimally use information to bias results. While being a central issue in the field, still lacking are notions of na…

I study the behavior and the performance of the long-term forecasts issued by financial analysts with respect to the Extrapolation Hypothesis. That hypothesis states that investors, extrapolating from the firms' recent performances, are too optimistic about growth and large firms and too pessimistic about value and sma…

We consider the problem of asynchronous online testing, aimed at providing control of the false discovery rate (FDR) during a continual stream of data collection and testing, where each test may be a sequential test that can start and stop at arbitrary times. This setting increasingly characterizes real-world applicati…

Study investor sentiment and disagreement on StockTwits during COVID-19.

This work proposes a way to align statistical modeling with decision making. We provide a method that propagates the uncertainty in predictive modeling to the uncertainty in operational cost, where operational cost is the amount spent by the practitioner in solving the problem. The method allows us to explore the range…

New algorithm identifies near-optimal policies in adversarial distributed RL settings.

New algorithm CROP achieves asymptotic optimality with bounded regret.

Study proposes a new risk measure for optimal portfolio allocation.

Pessimistic estimator improves multi-objective policy optimization.

This work improves RLHF performance guarantees using greedy sampling.

Pessimistic Q-learning improves sample efficiency in offline reinforcement learning.

Algorithm minimizes regret while adhering to unknown safety constraints.

Bayesian classifier improves robustness with optimistic score ratio.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

We introduce novel variants of momentum by incorporating the variance of the stochastic loss function. The variance characterizes the confidence or uncertainty of the local features of the averaged loss surface across the i.i.d. subsets of the training data defined by the mini-batches. We show two applications of the g…

Optimistic estimate predicts best fitting performance of nonlinear models.

New algorithm reduces regret in sequential decision-making problems.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

Characterizes concept classes for optimistic online learning.

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

Algorithm learns optimal coordination for strategic agents in uncertain settings.

Semi-pessimistic RL tackles distributional shift and data scarcity in offline RL.

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

PESCAL uses mediators to learn from confounded offline data.

Pessimistic Minimax Value Iteration finds efficient NE policies from offline data.

Omega method mitigates noise in stochastic game optimization.