Agents learn to play a first-person multiplayer game at human level performance.
problem Training AI agents for complex, multi-agent, real-time environments.
method Population-based deep reinforcement learning with concurrent training of multiple agents.
result Achieved human-level performance in a first-person multiplayer game.
ViZDoom competition tests AI in Doom using only visual input.
problem Creating AI bots that play Doom effectively from raw visual data.
method Multi-player deathmatch in Doom; bots make decisions based on screen buffer.
result Reinforcement learning can produce capable bots but not competitive against humans.
We consider the problem of learning to play first-person shooter (FPS) video games using raw screen images as observations and keyboard inputs as actions. The high-dimensionality of the observations in this type of applications leads to prohibitive needs of training data for model-free methods, such as the deep Q-netwo…
Neural MMO simulates MMOs to study multiagent intelligence.
problem Limited research environments for multiagent intelligence.
method Developed a new game environment inspired by MMOs.
result Standard methods can learn interesting behaviors in MMOs.
Deep RL agent performs well in Doom, a complex FPS game.
problem Addressing complex environments with sparse rewards and large state spaces.
method Divide and conquer approach using unsupervised auxiliary tasks.
result Our agent performs better in unknown environments than state-of-the-art algorithms.
AI agent plays CSGO deathmatch with human-like style.
problem Lack of API for CSGO limits data for reinforcement learning.
method Behavioural cloning on large noisy and expert datasets.
result Matches human difficulty level in deathmatch mode.
We applied Generative Adversarial Networks (GANs) to learn a model of DOOM levels from human-designed content. Initially, we analysed the levels and extracted several topological features. Then, for each level, we extracted a set of images identifying the occupied area, the height map, the walls, and the position of ga…
Motivated by recent advance of machine learning using Deep Reinforcement Learning this paper proposes a modified architecture that produces more robust agents and speeds up the training process. Our architecture is based on Asynchronous Advantage Actor-Critic (A3C) algorithm where the total input dimensionality is halv…
A number of recent approaches to policy learning in 2D game domains have been successful going directly from raw input images to actions. However when employed in complex 3D environments, they typically suffer from challenges related to partial observability, combinatorial exploration spaces, path planning, and a scarc…
AppStreamer reduces mobile game storage by predicting needed files.
problem Expanding storage needs of mobile games and apps.
method Predictive streaming of app files from cloud or edge servers.
result Reduces storage by 87% for Dead Effect 2 and 86% for Fire Emblem Heroes.
High-throughput 3D control training system achieves 100,000 FPS.
problem Lack of efficient, single-machine reinforcement learning systems.
method Sample Factory combines asynchronous sampling and off-policy correction.
result Achieves 100,000 FPS on 3D control problems without sacrificing sample efficiency.
WILD-SCAV benchmarks AI in complex 3D FPS environments.
problem Lack of complexity and diversity in RL environments.
method Developed a 3D open-world FPS game environment.
result Demonstrates effectiveness in benchmarking RL algorithms.
Flatland is a simple 2D environment for reinforcement learning.
problem Testing and prototyping reinforcement learning agents.
method Simple, lightweight, customizable environment with physical properties.
result Baseline reinforcement learning agents can solve a navigation task in Flatland.
Study how actions affect perception in embodied agents using group theory.
problem Understanding how actions influence perception in autonomous agents.
method Mathematical formalism of group theory applied to sensory commutativity of action sequences.
result Introduced Sensory Commutativity Probability (SCP) to measure action effects on perception.
We explore the perspective of a bug living on the two-dimensional surface of a polyhedron. Images of various kinds of effects like lensing and cloaking are shown via color pictures of three viewpoints: the first person perspective of the bug, a map of the bug's viewpoint, and a look at the bug on the embedded polyhedro…
Deep reinforcement learning has been successfully applied to several visual-input tasks using model-free methods. In this paper, we propose a model-based approach that combines learning a DNN-based transition model with Monte Carlo tree search to solve a block-placing task in Minecraft. Our learned transition model pre…
The ability to use a 2D map to navigate a complex 3D environment is quite remarkable, and even difficult for many humans. Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community. In this paper we teach a reinforce…
The paper investigates the effectiveness of reusing experience in Deep Q-Learning for FPS environments.
problem The high number of interactions required for reinforcement learning limits its practicality.
method The authors test the effectiveness of applying learning update steps multiple times per environmental step in the VizDoom environment.
result Updating learning steps less frequently than every 4th environmental step does not improve performance and can degrade performance.
Visual analogies help transfer knowledge between Atari games.
problem Can visual analogies transfer knowledge between Atari games?
method Created visual analogies between pairs of Atari games and used them to train policies for one game using data from another.
result Visual analogies can be used to transfer knowledge between Atari games.
Paper presents content-based models for game recommendation in cold start scenarios.
problem Cold start problem in game recommendation where new games and players have no historical data.
method Uses survey data to develop content-based interaction models that generalize to new games, players, and both.
result Content models outperform collaborative filtering in predicting new interactions.
Potential games, originally introduced in the early 1990's by Lloyd Shapley, the 2012 Nobel Laureate in Economics, and his colleague Dov Monderer, are a very important class of models in game theory. They have special properties such as the existence of Nash equilibria in pure strategies. This note introduces graphical…
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture. Despite their effectiveness, they suffer from several disadvantages including training instability, random …
A new framework for playing and learning board games.
problem Tackling the tedious and repetitive aspects of coding for board game AI.
method Developed a generic TD(λ)-n-tuple agent for arbitrary board games. result TD(λ)-n-tuple outperforms other generic agents on various games. IGGP learns game rules from varying quality game play, finding no overall trend.
problem Learn game rules from varying quality game play.
method Used Sancho's intelligent game traces and ILP systems (Metagol, Aleph, ILASP) to induce game rules from traces of varying quality and volume.
result No overall trend in accuracy of learned game rules from varying quality and volume of training data.
New game introduces linking-unlinking strategy for two-component links.
problem Tackling the linking and unlinking of two-component links.
method Introducing and analyzing the Linking-Unlinking Game on various link shadows.
result Winning strategies for specific link shadows are presented.
A game on diagrams switches crossing directions to achieve connectedness.
problem Achieving connectedness in diagrams through crossing switches.
method Players switch crossing directions on regions of a diagram to achieve connectedness.
result Connectedness can be achieved through strategic crossing switches.
Introduces SM-games to analyze machine learning interactions.
problem Lack of understanding and control in n-player games.
method Introduces SM-games with pairwise zero-sum interactions.
result SM-games are amenable to first-order optimization methods.
Just as war is sometimes fallaciously represented as a zero sum game -- when in fact war is a negative sum game - stock market trading, a positive sum game over time, is often erroneously represented as a zero sum game. This is called the "zero sum fallacy" -- the erroneous belief that one trader in a stock market exch…
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independe…
TextWorld is a Python library for RL agents in text-based games.
problem Training RL agents on text-based games with varying challenges and sparse rewards.
method Developed a Python library with backend functions for state tracking and reward assignment. Enables users to create new games with precise control over difficulty and scope.
result Demonstrated the effectiveness of TextWorld in training RL agents on a curated list of games and generated sets of games.
Game theory helps analyze ESOs/EBIs in production and service sectors.
problem Economic incentives affect traditional production/service functions and create intangible capital.
method Uses game theory to analyze interactions in ESO/EBI transactions.
result No perfect Nash Equilibria for two-stage games involving many participants.
We start briefly surveying research on optimal stopping games since their introduction by E.B.Dynkin more than 40 years ago. Recent renewed interest to dynkin's games is due, in particular, to the study of Israeli (game) options introduced in 2000. We discuss the work on these options and related derivative securities …
The paper explores how regularization can lead to convergence in imperfect information games.
problem Finding equilibrium in imperfect information games with imperfect information.
method Investigates Follow the Regularized Leader dynamics and how adding a regularization term can lead to strong convergence guarantees.
result The approach leads to algorithms that converge exactly to the Nash equilibrium in imperfect information games.
Educational game on crypto investment helps students grasp macroeconomics.
problem Weak connections between microeconomic decision-making and macroeconomic concepts in classroom games.
method Design and study of an educational game on cryptocurrency investment.
result Engages students in understanding macroeconomics through incentivized individual investment decisions.
Simplified NFT games discussed with methods for extracting value.
problem Issues influencing NFT games' structure and stability.
method Three methods for extracting value from NFT games.
result Various design constraints and mutual beneficial games.
The paper proposes a method to learn continuous-action graphical games from perturbed equilibria.
problem Learning the exact structure of continuous-action graphical games from limited data.
method A ℓ12− block regularized method to recover the graphical game structure. result The method recovers the exact structure of the graphical game under certain conditions.
Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.
problem Understanding dynamics of zero-sum games with hidden structure.
method Gradient Descent Ascent applied to hidden zero-sum games with specific convex-concave structure.
result Gradient Descent Ascent converges to von-Neumann solution in strictly convex-concave hidden games.
Paper tackles hidden game problem in AI alignment and language games.
problem Hidden game problem in AI alignment and language games.
method Developed a composition of regret minimization techniques to discover and exploit hidden structures.
result Achieved optimal external and swap regret bounds for rapid convergence to correlated equilibria.
AEC Games model represents software MARL environments better than POSGs.
problem POSGs are conceptually unsuitable for software MARL environments.
method Introduced AEC Games model as an equivalent to POSGs.
result AEC Games model is more representative of software MARL environments.
We introduce CSE for MLSF games and devise online learning algorithms for achieving no-external Stackelberg-regret.
problem Learning equilibrium in leader-follower games with noisy bandit feedback.
method Proposed Correlated Stackelberg Equilibrium (CSE) and online learning algorithms balancing exploration and exploitation.
result Achieves no-external Stackelberg-regret, converging to approximate CSE.
Generalizes region select game to k-colored knot diagrams.
problem Play game on knot diagrams with multiple colors.
method Generalize region select game to k-colored knot diagrams. result Generalization of the region select game to k-colored knot diagrams. Deep Reinforcement Learning automates match-3 game testing.
problem Reducing human effort in testing match-3 video games.
method Dueling Deep Q-Network paradigm applied to Jelly Juice game.
result The network outperforms random player and adapts to game difficulty.
Unified framework for Bayesian and Frequentist statistics.
problem Embedding Bayesian statistics within a broader decision-making framework.
method Game theory and statistical analysis.
result Statistical games unify Bayesian and Frequentist statistics.
New games model strategic interactions in incomplete information settings.
problem Modeling strategic interactions in incomplete information settings.
method Introduced new games that map input to private player types, aggregate strategies, and converge to near-Nash equilibria.
result Games can recover meaningful strategic interactions from real data.
New algorithm improves performance in nontransitive games.
problem Nontransitive games lack a clear winner.
method Geometric framework for agent objectives, PSRO_rN algorithm.
result PSRO_rN consistently outperforms alternatives in nontransitive games.
New game approximates mean curvature flow evolution.
problem Approximating geometric mean curvature flow evolution.
method Two-player zero-sum game with probabilistic elements.
result Value function approximates mean curvature flow.
Study on mean field games with singular controls and their applications.
problem Optimal productivity expansion in dynamic oligopolies.
method Existence and uniqueness of mean field equilibria through nonlinear equations, Abelian limit for discounted and ergodic games.
result Valid connection between discounted and ergodic games, approximation of Nash equilibria.
Gradient methods converge exponentially in concave network games.
problem Finding Nash equilibria in concave network zero-sum games.
method Gradient Ascent and Optimistic Gradient Ascent analyses.
result Exponential convergence rates in various game settings.