A clone of ChatGPT which harnesses the power of GeForce to process data locally on your computer? This is NVIDIA’s idea.
The best-known conversational agent prototype, ChatGPT of course works thanks to artificial intelligence, but it only works online, with an Internet connection to provide information to its interlocutors.
With Chat with RTX, NVIDIA embraces a more original segment of these conversational agents. The tool made available by NVIDIA works in isolation, locally on your machine… equipped with a GeForce of course.
Local artificial intelligence
Chat with RTX is not yet available in final version and NVIDIA is talking about providing a “simple” demo as if to show what it is possible to do on our small machines.
Well, not that small since, to work, Chat with RTX uses TensorRT-LLM acceleration which requires a GeForce RTX 30 series (Ampere) or 40 series (Ada Lovelace) graphics card. Strong cards whose objective is to animate a robot expert in data research, but a robot operating exclusively locally: there is then no risk of any leak or collection of data.
As presented by NVIDIA, the idea is simple: once installed – we will come back to this – Chat with RTX receives “source” documents on which to base its research. All you have to do is ask questions to summarize a subject or explore certain points in more depth without you having to read everything or understand everything.
A good half hour to install it
For now, NVIDIA is only talking about a demo of Chat with RTX, a sort of preliminary version intended to show part of its capabilities, but which will logically evolve.
The installation procedure is not complex… but long © Nerces for Clubic
To test this, you must have a computer running Windows 10/11 with a GeForce RTX series 30 or 40 graphics card. NVIDIA mentions the need for 8 GB of video memory and GeForce drivers version 535.11 or higher. There is no mention of a minimum processor anywhere, but 16 GB of RAM is required.
You must also download a 35 GB “package” which contains the Chat with RTX installer. Please note, once the installation has started, you still need to be patient: a preparatory phase with other data to download is necessary. In total, we are talking about 50 to 100 GB of data downloaded for a process of between 30 and 60 minutes.
Once the installation is complete, it is possible to launch Chat with RTX and, in the window that appears, to specify where the resources on which it will work are located. For the moment, NVIDIA mentions the limitation to TXT, PDF, DOC files as well as YouTube videos.
In use, what does it look like?
If we have not yet sought to really explore the possibilities of Chat with RTX or push interactions with the robot, we were still keen to see, quickly, what it is capable of.
To do this, we first asked him a few questions not directly related to the data he had. No miracle, the answers often made little sense and just asking him if he was capable of understanding French went well beyond his “cognitive” abilities.
So we played the game and injected it with various technical documents written by NVIDIA to talk about its new generation of GeForce RTX 40 SUPER series graphics cards and DLSS. First success, the explanations are certainly academic, but DLSS is then very detailed.
The answers from Chat with RTX are sometimes surprising © Nerces for Clubic
On the other hand, if Chat with RTX was able to extract and compile the technical data of the cards, it was more difficult for him to understand them: he thus gets confused when asked which of the RTX 4070 SUPER or the RTX 4080 SUPER is the most powerful.
Chat with RTX skillfully extracts information from various texts © Nerces for Clubic
Thirdly, we wanted to go out Chat with RTX of the IT field. We injected him with documents relating to the profession of journalist, to his law. Nice surprise to see him dissect documents in French, but also extract the key elements with brilliance. Let us recognize that the said documents were perfectly organized.
Finally, since NVIDIA is talking about YouTube, we have given some video links to Chat with RTX. On a sequence related to the functioning of generative AI, he was able to give us a summary of the situation… but don’t believe that Chat with RTX can interpret the host’s words.
To “understand” a video, it relies on the text transcription associated by YouTube and treats it like any text document. No transcription and Chat with RTX will be silent as a carp. Still, the result can be interesting.
There would still be a lot of testing to do on Chat with RTX and it would also be interesting to check its progress and developments. The potential of such software is quite remarkable and we are of course thinking of the analysis work that it would be possible to do, particularly at school, to extract information from multiple sources (Wikipedia?).
Local analysis is interesting to avoid any interference, especially in a school environment. That said, its demo status, its technical limitations (50 to 100 GB monopolized, 3 GB of RAM occupied) and bugs will undoubtedly prevent Chat with RTX from reaching a wide audience, but the promise is there. Enough to exist alongside other conversational agents like ChatGPT?
In 2023, the landscape of chatbots in French has expanded considerably, boosted by the rise of artificial intelligence. In the past, these assistants were rationed to predefined questions and answers. But now, thanks to advances like ChatGPT, it’s possible to ask any question and get relevant answers generated in real time.
Read more
Source : NVIDIA
4