Spring Update: OpenAI unveils its GPT-4o model, a desktop application and good news for the free version of ChatGPT

Little by little, OpenAI is making its mark and making its appointments. This May 13, CTO Mira Murati took the microphone to announce big news for ChatGPT and the GPT-4 API. Curious timing on the eve of the Google I/O…

From its very beginnings, the problem of the man-machine interface has arisen in computer science. How can we facilitate our use of IT tools? The screen, the keyboard, the mouse, the touchscreen have all had a significant impact on our uses. Voice and sight are the logical next steps according to OpenAI. Moving in this direction, the company revealed its new GPT-4o multimodal model, as well as a native application for macOS.

Still no GPT-5, but a GPT-4o for omni

Far from the rumors of an in-house search engine or the overpowering GPT-5, OpenAI started its spring with new features no less ambitious and just as impressive in many respects. Speaking of rumors regarding OpenAI, however, one of them was confirmed this evening, namely that the gpt2 model that surfaced in recent days was indeed GPT-4o. Like GPT-4 Turbo, the new model does not improve reasoning capabilities but expands the possibilities and modes of exchange while costing less than its predecessor, at least in terms of the API .

OpenAI also shakes up the economic model of ChatGPT since GPT-4o as well as many features exclusive to Plus, Team and Enterprise subscriptions such as GPTs or image upload are now free although limited in the number of requests that can be submitted on a period of 3 hours. Paying subscribers will benefit from 5 times more requests as well as priority access during peak periods.

Finally, OpenAI announces that it has improved understanding and tokenization capabilities (i.e. ChatGPT’s ability to “compress” data to maximize its memory in a conversation) in languages ​​other than English. If French only benefits a little from these improvements, Asian languages ​​such as Arabic, Hindi or Urdu will benefit from compression up to 4 times more effective!

ChatGPT more conversational than ever

The bulk of this Spring Update concerned a live demo of the new model, first on iPhone and then on Mac. From the outset, GPT-4o is much more responsive than the currently available voice assistant. It responds in real time and can be cut off at any time if you want to bounce back on what the AI ​​just said. Female voice, reaction to emotions and modulation of voice timbre according to requests, clearly this demo will recall the film “Her” by Spike Jonze. The first iteration of ChatGPT’s text-to-speech model was already pretty compelling, but what we have here is of a different caliber.

That’s not all since GPT-4o also pushes image recognition further and justifies its status ” omni “. While conversing with the AI, it will be possible to open the camera of your smartphone and interact with what it captures in real time. Not only can GPT-4o respond to what is displayed at the moment T, but it can also return to things that it “saw” shortly before. Among the different illustrations of his abilities, we can cite in spades: translation (in real time of course), help with homework, recognizing emotions on a face, counting quickly (yes, yes), singing. And, although it was not discussed during the conference, the possibilities in terms of accessibility for the visually and hearing impaired are definitely there! We can easily imagine the model translating sign language into voice to facilitate exchange between 2 people like real-time translation.

Despite Microsoft’s investments… a native application exclusive to the Mac

To punctuate its demonstration, the OpenAI team showed its native desktop application for macOS in a little more detail. And yes, despite Microsoft’s millions of dollars, OpenAI still seems to have its slight preference. It should be noted that third-party applications integrating ChatGPT into macOS are legion but that these were often limited, notably requiring the use of the API rather than a subscription to ChatGPT. OpenAI also offers here the possibility of conversing with GPT-4o by voice in addition to text. To go further, the application offers to capture your screen to assist you and see what you see in real time. The example given during the live focused on a piece of Python code displayed in an IDE.

Less original, the demonstration on a graph still shows the convenience of having ChatGPT to carry click (voice?) while avoiding copying and pasting text or taking a screenshot to start the interaction. In all this, textual interaction is not left out since it will be possible to open ChatGPT using a simple shortcut (option + space). Adding to this convenience, this shortcut shows the ambition for OpenAI to make ChatGPT a second Spotlight on Mac. More evasive on availability, OpenAI specifies that the application, like the rest of the new features announced, will be deployed gradually in the coming weeks for subscribers of the service.

This is reassuring about the future of the company after a turbulent end to 2023. With rumors of a deeper integration of ChatGPT in iOS 18 (as an alternative to Siri?) these announcements bode well for WWDC expected in early June.