Google Research and Tel Aviv University have just announced a very interesting project, GameNGen. This is a technology using a neural network, machine learning and a modified implementation of Stable Diffusion to create a playable version of 1993’s DOOM.
This project sets out to answer the question Can a neural model running in real-time simulate a complex game at high quality?
In this work we demonstrate that the answer is yes. Specifically, we show that a complex video game, the iconic game DOOM, can be run on a neural network (an augmented version of the open Stable Diffusion v1.4 (Rombach et al., 2022)), in real-time, while achieving a visual quality comparable to
that of the original game. While not an exact simulation, the neural model is able to perform complex game state updates, such as tallying health and ammo, attacking enemies, damaging objects, opening doors, and persist the game state over long trajectories.GameNGen answers one of the important questions on the road towards a new paradigm for game engines, one where games are automatically generated, similarly to how images and videos are
generated by neural models in recent years. Key questions remain, such as how these neural game engines would be trained and how games would be effectively created in the first place, including how to best leverage human inputs. We are nevertheless extremely excited for the possibilities of
this new paradigm.
Details of how this was accomplished and videos of the end results are available in the links below.
Key Links
Venture Beat (Over Hyped Coverage)
The Register (Realistic Coverage)
You can learn more about Google GameNGen and the use of neural networks, stable diffusion and machine learning to reproduce a playable version of DOOM in the video below.