Meta is launching the generative AI war with LLaMA, its language model for artificial intelligences.
On Friday, February 24, 2023, Meta, the parent company of Facebook, affirmed its desire not to miss the AI war. Faced with ChatGPT and Bard, the social network giant’s response will therefore be called LLaMA, a ” new language model […] designed to help researchers in their work “, According to Mark Zuckerberg in a message posted on a Facebook account.
Recently, generative artificial intelligences have shown their ability to “ generate text, hold conversations, summarize texts, and even perform more complex tasks such as solving mathematical theorems “, notes the founder of the social network – and Meta does not intend to leave the field open to its rivals in an increasingly strategic sector.
LLaMA, for Large Language Model Meta AI, is, as its name suggests, a language model. This is the basis for using artificial intelligences like ChatGPT and other conversational agents – chatbots, in English. The arrival of Meta in this area, which has attracted everyone’s attention since the release of ChatGPT, is not insignificant — and LLaMA is not a model like the others.
What is a language model?
First of all, it should be clarified what a language model is. This is a statistical model that predicts which word will follow, based on the terms already entered. It is thanks to this that the large language models know that the word cat will more likely be followed by the adjective ” black » than by the term « paella “. LLaMA is therefore the equivalent of GPT-3, the model that allows ChatGPT to work, and of LaMDA, the language model developed by Google to power Bard.
The latest generations of LLMs contain billions of parameters, which have made it possible to refine the responses of ChatGPT and to ensure that these responses appear as natural as possible, as if the Internet user was exchanging with another human. For example, GPT-3 operates on 175 billion parameters, and Google’s model, the largest to date, includes 540 billion.
What features for Facebook’s LLaMA?
LLaMA has the particularity of only working that over 65 billion parameters. This is less than its rivals, but according to Meta, this smaller dimensioning would nevertheless be an advantage: LLaMA is available in several versions (65 billion parameters, and others on 33 billion, 13 billion and 7 billion).
“ Smaller models, like LLaMA, are interesting in the very large model world, because they require much less computing power and resources. notes Meta. A reduced size would democratize access to language models — and save processing power.
But according to Meta, this relative smallness in terms of parameters does not prevent it from being “ more efficient » than other larger models. The research paper published to accompany the launch of the model thus states that “ LLaMA-13B outperforms GPT-3 on most criteria, and LLaMA-65B is competitive with the top models, Chinchilla 70B and PaLM-540B — these are models developed respectively by DeepMind, the company behind AlphaGo, and by Google.
Difficult to give a definitive answer on the capacities of LLaMA today: it is impossible to test the model. For the moment, unlike Chat-GPT, LLaMA is not accessible to everyone: only a few researchers who request it can have access to the model, the faculties of which will remain to be tested.
Meta believes this is a huge advancement in AI, because ” LLaMA is designed to be versatile and used in many scenarios unlike models trained for a specific task “. The research paper also argues that LLaMA can run on a computer equipped with a single graphics processing unit (GPU, found in graphics cards), a unique level of accessibility for LLMs.
Finally, ” unlike Chinchilla, PaLM and GPT-3, we only used publicly available data », which would make the model open-source.
Meta joins the war of artificial intelligence
The arrival of LLaMA will not have an immediate impact on the level of the general public, as it could have been for ChatGPT. Still very technical and only reserved for research work, LLaMA should not be integrated into a search engine either, as will be the case for Bard. Meta, nevertheless, positions itself on the media plan, and reminds that it is also the coup.
Indeed, this presentation proves that Meta does not intend to leave the front of the stage to its two other opponents in the field of AI, OpenAI (and, therefore, Microsoft), and Google. In January, Yann Le Cun, the head of artificial intelligence at Meta, had also criticized the prowess of ChatGPT, believing that there was ” nothing revolutionary and hinting that the company would soon unveil something big.
LLaMA is not Meta’s first project. The two previous ones were rather poorly received, which may explain the relative discretion of the company on the subject, and the fact that LLaMA is not accessible to the public. Released in August 2022, Meta’s first attempt, Blenderbot, was supposed to be a chatbot that could learn from its mistakes, but quickly made anti-Semitic and conspiratorial rhetoric.
A few months later, in November 2022, it was Galactica’s turn to go public. The AI was this time specialized in writing scientific articles and could solve math problems, write Wikipedia articles and write code. But after just 3 days, Galactica had to be unplugged after posting wrong and racist responses to users, Next Inpact explains. Could LLaMA be the solution that will allow Meta to stand out for good in the AI war? That remains to be seen.
Subscribe to Numerama on Google News to not miss any news!