A team of Nvidia researchers has created “Voyager”, a Minecraft bot based on GPT-4, the language model behind ChatGPT. It is he who gives instructions to the bot, capable of learning to play Minecraft on its own.
Several artificial intelligence models exist for Minecraft: they aim to create real programs that play the game on their own, which requires advanced technical knowledge and a lot of development resources. Among them, a new one is appearing: Voyager, powered in part by GPT-4, the great language model behind ChatGPT.
A Minecraft bot that learns on its own thanks to artificial intelligence
As MineDojo team member Jim Fan wrote in a Twitter thread, “ Voyager is continually improving by writing, refining, validating, and fetching code from a skill library. “Minecraft represents for him and his team a very important playground: as a video game says” Sandbox », the number of actions to be carried out and decisions to be made is almost infinite.
What if we set GPT-4 free in Minecraft? ⛏️
I’m excited to announce Voyager, the first lifelong learning agent that plays Minecraft purely in-context. Voyager continuously improves itself by writing, refining, committing, and retrieving *code* from a skill library.
GPT-4 unlocks… pic.twitter.com/hjTxk6Qb1x
—Jim Fan (@DrJimFan) May 26, 2023
Reading the description of the program, we learn that it is divided into three main elements:
- ” An iterative incentive mechanism that incorporates game feedback, runtime errors, and self-checking to fine-tune programs » ;
- ” A library of code skills to store and retrieve complex behaviors » ;
- ” An automatic program to maximize exploration “.
In practice, Voyager first tries to achieve a specific goal, which it does not happen on the first try (or very rarely). At this, a program returns to Voyager for it to learn. Each additional knowledge of the game is added to a database. The most complex skills have been simplified into a simpler division of tasks: we think in particular of the creation of objects which requires the creation of other intermediate objects.
From a technical point of view, the bot does not ” sees » not, it receives information directly from the game via an API: position of blocks, animals, etc. In the future, the teams having worked on Voyager would like to integrate a real vision AI model.
What GPT-4 brings to this program for Minecraft
It’s not GPT-4 that directly decides Voyager’s movements or tells you to right-click. In reality, it is this third part of Voyager which offers exploration tasks ” appropriate based on the agent’s current skill level and the state of the world “.
It is the language model developed by OpenAI that resonates from the data given to it as input. This can for example be objects in the fake player’s inventory, which allow him to suggest to make an object from these first ones.
All this allows Voyager to lead a game and even accomplish some achievements: craft a diamond sword, build a house or even a portal to another dimension. If he is still very similar to a beginner player, his progress is already great.
Want to join a community of enthusiasts? Our Discord welcomes you, it’s a place of mutual aid and passion around tech.