Netflix has come up with a technique to simplify the addition of green screen special effects. This new, faster way of doing things is partly based on artificial intelligence. But, can it be the new benchmark?
You knew the green background, which is used to add special effects: now discover the purple characters. At the end of June 2023, scientists working for Netflix shared a research article (which has not yet been published in a scientific journal) on a very specific topic: special effects. The experts explain that they have developed a new technique which, thanks to magenta light and artificial intelligence, can give impressive results.
New Scientist, which spotted the researchers’ paper and devoted an article to it on July 7, also made a video explaining the Netflix discovery. The new method, radically different from what we know, is supposed to be faster and more precise than current techniques. But will she be able to win?
Magenta light for better special effects
The technique developed by the scientists is simple. Green screens are used during filming to be able to embed computer-generated images in post-production. The green color was not chosen at random: as Adobe reminds us, which notably created the Photoshop software suite, ” green is the furthest color from human skin tones, making it easier to cut out “.
The technique is reliable, used for years, but it still has some flaws. Transparent objects are not always well taken into account, as is hair. If someone has the bad idea to put on a green t-shirt, the result is catastrophic.
With its research paper, Netflix has just developed a new technique, named MGS, for “Magenta Green Screen”. Actors and objects appearing on set are filmed against a green screen. They are themselves illuminated by a mixture of red and blue light, which gives them a magenta complexion. It is this particular lighting that will change everything, and allow to obtain sharper and faster results for special effects.
” In a camera as in a human eye, there are three types of photoreceptors: green, blue, red “, explains Thomas Ancelle, director of Numerama videos and professional colorist. When characters are lit in magenta, ” this means that the one and only green thing the camera will pick up is the green screen. For example, if there is a tiny bit of green in the bottle, that means the bottle is transparent. If there is no green at all, it means the element is opaque. »
With this information, software will automatically create a “ alpha mask with transparency data. “ The black is where we should apply the background “, explains Thomas Ancelle. ” The white is the opaque areas, where nothing is applied, where people/objects are left. Everything else is the shades of gray, which will therefore be more or less transparent. »
It is this mask that will allow more precise processing of the image. “ It works very well for making a fine selection, or with transparent objects, but also with fine details like hair or motion blur (which creates a bit of transparency, like on the moving bottle) “. However, there is a flaw: people very well detached from the background, but in magenta “.
This is where the AI comes in. During the shots, a panel filled with colored squares also appears on the screen: this is ” a chart “, explains the videographer of Numerama. ” It’s a panel on which there are lots of colors which are a bit ‘official’, standardized, and which makes it possible to ensure that the colors of a scene are realistically represented “.
This charter serves as a reference for artificial intelligence. Since the square at the bottom left is supposed to be white, the AI knows that all areas of that shade should actually be white. ” So the AI corrects all those colors until the image that was lit in magenta looks natural again. »
This new technology has the potential to make the addition of special effects, and more generally all post-production of films and TV series, faster. However, will it be globally adopted? For the home productions of Netflix, one can suspect that this will be the case. It remains to be seen if this will soon be a standardization adopted on all sets.
If you liked this article, you will like the following ones: do not miss them by subscribing to Numerama on Google News.