arXiv research

Fictitious play is a simple and widely studied adaptive heuristic for playing repeated games. It is well known that fictitious play fails to be Hannan consistent. Several variants of fictitious play including regret matching, generalized regret matching and smooth fictitious play, are known to be Hannan consistent. In …

It is now well known that decentralised optimisation can be formulated as a potential game, and game-theoretical learning algorithms can be used to find an optimum. One of the most common learning techniques in game theory is fictitious play. However fictitious play is founded on an implicit assumption that opponents' …

In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric $N$-player non-zero-sum stochastic differential games, for which we refer as \emph{deep fictitious play}, a multi-stage learning pro…

Deep fictitious play converges to Nash equilibrium in stochastic differential games.

Decentralised optimisation tasks are important components of multi-agent systems. These tasks can be interpreted as n-player potential games: therefore game-theoretic learning algorithms can be used to solve decentralised optimisation tasks. Fictitious play is the canonical example of these algorithms. Nevertheless fic…

Fictitious play with reinforcement learning is a general and effective framework for zero-sum games. However, using the current deep neural network models, the implementation of fictitious play faces crucial challenges. Neural network model training employs gradient descent approaches to update all connection weights, …

Generative adversarial networks (GANs) are powerful tools for learning generative models. In practice, the training may suffer from lack of convergence. GANs are commonly viewed as a two-player zero-sum game between two neural networks. Here, we leverage this game theoretic view to study the convergence behavior of the…

New algorithm reduces MFGs with common noise complexity.

Despite the notable successes in video games such as Atari 2600, current AI is yet to defeat human champions in the domain of real-time strategy (RTS) games. One of the reasons is that an RTS game is a multi-agent game, in which single-agent reinforcement learning methods cannot simply be applied because the environmen…

We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret. From a more traditional, discrete-time viewpoint, this continuous-time approach allows us to derive the no-regret properties of a large class of discrete-time algorithms i…

Deep RL improves Diplomacy performance, outperforming previous methods.

We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individua…

Novel analysis of EFP for finite-sum problems in neural networks.

Two new RL methods enable deep learning of MFG equilibria.

The game-theoretic risk management framework put forth in the precursor work "Towards a Theory of Games with Payoffs that are Probability-Distributions" (arXiv:1506.07368 [q-fin.EC]) is herein extended by algorithmic details on how to compute equilibria in games where the payoffs are probability distributions. Our appr…

We study the problem of learning classifiers robust to universal adversarial perturbations. While prior work approaches this problem via robust optimization, adversarial training, or input transformation, we instead phrase it as a two-player zero-sum game. In this new formulation, both players simultaneously play the s…

New method labels GAN-generated faces without stereotyping.

Learning by experience in Multi-Agent Systems (MAS) is a difficult and exciting task, due to the lack of stationarity of the environment, whose dynamics evolves as the population learns. In order to design scalable algorithms for systems with a large population of interacting agents (e.g. swarms), this paper focuses on…

BOFiP optimizes high-dimensional functions by distributing them into sub-spaces and using game theory.

To improve efficiency and reduce failures in autonomous vehicles, research has focused on developing robust and safe learning methods that take into account disturbances in the environment. Existing literature in robust reinforcement learning poses the learning problem as a two player game between the autonomous system…

The paper simplifies complex jump-diffusion markets to complete models.

The paper proposes a new approach to model risk measurement based on the Wasserstein distance between two probability measures. It formulates the theoretical motivation resulting from the interpretation of fictitious adversary of robust risk management. The proposed approach accounts for equivalent and non-equivalent p…

Proposes a deep learning method for solving complex financial games with delays.

We present a simple model of a non-equilibrium self-organizing market where asset prices are partially driven by investment decisions of a bounded-rational agent. The agent acts in a stochastic market environment driven by various exogenous "alpha" signals, agent's own actions (via market impact), and noise. Unlike tra…

New method identifies cause-effect relations in multivariate time series data.

ESCHER avoids importance sampling to estimate regret in large games.

In this paper, we develop a multi-agent reinforcement learning (MARL) framework to obtain online power control policies for a large energy harvesting (EH) multiple access channel, when only causal information about the EH process and wireless channel is available. In the proposed framework, we model the online power co…

Cross-correlations in fluctuations of the daily exchange rates within the basket of the 100 highest-capitalization cryptocurrencies over the period October 1, 2015, through March 31, 2019, are studied. The corresponding dynamics predominantly involve one leading eigenvalue of the correlation matrix, while the others la…

Algorithm finds ε-equilibrium policies for multi-agent Markov games with hidden low-rank structure.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

Paper addresses theoretical risks in neural MCCFR, proposing Robust Deep MCCFR for improved performance.

Ghost points affect stability in finite difference schemes for diffusion equations.

Almost all of the work in graphical models for game theory has mirrored previous work in probabilistic graphical models. Our work considers the opposite direction: Taking advantage of recent advances in equilibrium computation for probabilistic inference. We present formulations of inference problems in Markov random f…

A large set of daily FOREX time series is analyzed. The corresponding correlation matrices (CM) are constructed for USD, EUR and PLZ used as the base currencies. The triangle rule is interpreted as constraints reducing the number of independent returns. The CM spectrum is computed and compared with the cases of shuffle…

Bayesian framework improves trading robustness against market shifts.

Proposes a new jackknife method for time series hyperparameter selection.

In this article we propose a model for stochastic delay differential equation with jumps (SDDEJ) in a differentiable manifold $M$ endowed with a connection $\nabla$. In our model, the continuous part is driven by vector fields with a fixed delay and the jumps are assumed to come from a distinct source of (càdlàg) noise…

Quantum annealing is a generic solver of the optimization problem that uses fictitious quantum fluctuation. Its simulation in classical computing is often performed using the quantum Monte Carlo simulation via the Suzuki--Trotter decomposition. However, the negative sign problem sometimes emerges in the simulation of q…

In this paper, we consider the sigmoid Gaussian Hawkes process model: the baseline intensity and triggering kernel of Hawkes process are both modeled as the sigmoid transformation of random trajectories drawn from Gaussian processes (GP). By introducing auxiliary latent random variables (branching structure, Pólya-Gamm…

In frictionless markets, utility maximization problems are typically solved either by stochastic control or by martingale methods. Beginning with the seminal paper of Davis and Norman [Math. Oper. Res. 15 (1990) 676--713], stochastic control theory has also been used to solve various problems of this type in the presen…

An algorithm is presented for momentum gradient descent optimization based on the first-order differential equation of the Newtonian dynamics. The fictitious mass is introduced to the dynamics of momentum for regularizing the adaptive stepsize of each individual parameter. The dynamic relaxation is adapted for stochast…

We reformulate the Cont-Bouchaud model of financial markets in terms of classical "super-spins" where the spin value is a measure of the number of individual traders represented by a portfolio manager of an investment agency. We then extend this simplified model by switching on interactions among the super-spins to mod…

Develops a new theory for neural systems stability and width effects.

We study the mean escape time in a market model with stochastic volatility. The process followed by the volatility is the Cox Ingersoll and Ross process which is widely used to model stock price fluctuations. The market model can be considered as a generalization of the Heston model, where the geometric Brownian motion…

Self-play is an unsupervised training procedure which enables the reinforcement learning agents to explore the environment without requiring any external rewards. We augment the self-play setting by providing an external memory where the agent can store experience from the previous tasks. This enables the agent to come…

We seek to infer the parameters of an ergodic Markov process from samples taken independently from the steady state. Our focus is on non-equilibrium processes, where the steady state is not described by the Boltzmann measure, but is generally unknown and hard to compute, which prevents the application of established eq…

New algorithms prove self-play can be effective in competitive RL.

Paper solves tracking control for $(x,u)$-flat systems using classical states.