Facebook first showed us the work done on his famous Avatar codecs in 2019, then last month with their evolution, the Avatar Codec 2.0 (see article). From now on, the firm announces that it can do the same with a simple iPhone. The capture device used so far for generating the avatars required a specialized capture facility called MUGSY with 171 high definition cameras. But the latest research from Meta make it possible to overcome this requirement, by generating an avatar from the scan of a smartphone equipped with a depth of field sensorlike everything iPhone equipped with Face ID (compatible withiPhone 10 and higher models).
How is it possible that what once required over 100 cameras now only requires a phone? The trick lies in the use of a “hyper-network”. The researchers formed what they call UPM by scanning the faces of 255 different people usingan advanced capture device, similar to MUGSYbut equipped with 90 cameras. You’ll first need to pan the phone around your neutral face, then pan it again by copying a series of 65 facial expressions to get a convincing rendering.
You may have guessed it, but it’s not your iPhone who will take care of the generation of the avatar, but of the scan of your face. If face scanning is around 3 minutes, you will have to wait 6 hours, 4 GPUs high-end in support, to bring your avatar to life ! If the application on iPhone went out, there is no doubt that this step would be carried out in the cloudsa bit like what is commonly done with the competition. Meta claims that the quality of its result is state-of-the-art, far beyond the competition. Let’s keep in mind that Meta is working in parallel to equip our future virtual reality headsets with a chip to facilitate the display of photorealistic avatars (see article). Thus, all forces converge to make this dream a reality. Nevertheless, a few flaws answer the call: this technology still can’t handle glasses or long hair, or even the body.
What do you think ? Will it be possible in the future, by starting a possible Meta Quest 3, that we can be invited from the beginning to integrate ourselves in a photorealistic way in our applications, does this seem possible, scary, unnecessary to you? Do not hesitate to send us your reactions in the comments.
If you haven’t taken the plunge yet, you can treat yourself to a Oculus Quest 2 at the house of Bakerthe Fnac, Darty or Amazon for €349.99. You can also find good PCs for gamers there.