At war with OpenAI and its business models, Elon Musk promised in early March that he would make the language model behind Grok, his chatbot, open source. Promise kept.
GPT-4, Gemini Pro, Llama 2 and Mistral Large now know the name of their rival: Grok-1.
Since March 18, Elon Musk’s teams have been offering a gigantic 318.24 GB file for download, accessible in the form of a torrent file (in bio on his Twitter account). It contains a language model with 314 billion parameters, developed by the company xAI. It is Grok-1 which powers the Grok chatbot, available on X since the end of 2023. Mistral carried out the same torrent technique in December 2023.
Can Grok become a serious alternative to other language models?
Touted by many as a satirical version of ChatGPT (it can insult its users and make vulgar jokes), the Grok chatbot has never been considered a serious contender in the generative artificial intelligence war. With its transformation into an open source model, and therefore usable by anyone who wishes, the situation may change.
Grok-1 is no longer a language model exclusive to Elon Musk’s service, but can be tested locally by people who own powerful enough machines. Its developers also published it on Github, to encourage the AI community to test Grok-1. Elon Musk’s LLM is becoming one of the references in open source, alongside the French company Mistral and the giant Meta (Google and OpenAI have more closed approaches).
In a blog post, xAI explains that the model published in torrent dates from October 2023 and that it is based on an architecture called Mixture-of-Experts (MoE). This is a model restricted in its capabilities, since it is not designed to directly respond to specific needs (for developers to adapt). Each token can use up to 25% of Grok-1’s total capacity, making it more efficient.
At the beginning of March, Elon Musk announced that he was filing a complaint against OpenAI, which he accuses of having lied about its intentions (Elon Musk helped create the company, before walking away from the project). OpenAI explains that Elon Musk dreamed of merging the company with Tesla, which caused a dispute. With xAI, Elon Musk hopes to create a counter-model to that of OpenAI, in the hope of harming the company.
Subscribe for free to Artificielles, our newsletter on AI, designed by AIs, verified by Numerama!