In an unusual exercise, Meta (the parent company of Facebook, Instagram and WhatsApp) recently opened the doors to its R&D labs. During a video conference personally organized by Mark Zuckerberg with several of the group’s engineers, the social networking giant unveiled a series of virtual reality helmet prototypes – each more futuristic than the last.
The exercise comes as investors worry about the fortunes gushed by the Menlo Park group to develop the technologies of the Metaverse, a hyper-realistic digital universe that Meta is banking on for its future growth. The company’s Reality Labs division lost $10 billion last year. And in the first quarter, the group blew up another 3 billion for just under 700 million in sales there… Precisely at the moment when the group’s growth is sharply slowing down, Meta needs to calm the credibility of its bet.
“Virtual Turing Test”
He is extremely ambitious. The goal of the R&D teams is to create display systems on the same level as human vision. Internally, engineers speak of the “virtual Turing test” – based on the famous test that was developed in 1950 to assess the suitability of a computer program for a human being. Likewise, Meta doesn’t want its headset users to be able to tell if they’re seeing a screen or the reality of the outside world.
“It’s much more complex than displaying a realistic image on a computer or television screen,” says Mark Zuckerberg. Human vision includes many parameters that need to be reproduced if we want to capture the user’s eye and brain. The meta teams have therefore developed several prototypes – each trying to solve one of the many challenges of the “virtual Turing test”.
resolution and varifocals
The first device Mark Zuckerberg pulls out of his pocket is codenamed “Butterscotch”. It aims to drastically increase the number of pixels in the display. Ideally, there should be 8K resolution across the VR headset’s field of view – much wider than a TV screen.
The meta teams have managed to more than double the resolution of the Quest 2 (the currently commercially available headset) to get closer to the human retina. “We can align the 20/20 line of an optometric chart [ceux utilisés par les ophtalmologues, NDLR] ‘ says Mark Zuckerberg happily. However, the engineers had to sacrifice half the field of view for this, and the device remains heavy and bulky.
The second prototype is called “Half Dome” and deals with the subject of depth of field. This is built into the helmets currently on the market. But “in the real world, the lens of our eyes acts like a lens, constantly changing shape to focus on what we’re seeing,” says Mark Zuckerberg. After experimenting with mechanical vari-focal screen systems in 2017, Meta switched to an electronic version that is simpler and more robust. This allows the helmet to follow your gaze and adjust the depth of field depending on where you are wearing it – a bit like the autofocus of a camera.
A lens hologram
For its part, a third model, Starburst, aims to offer brightness and contrasts as good as natural light – which means multiplying them by a hundred times compared to current helmets. The prototype is so heavy it has handles.
To solve major bulk and weight concerns, Meta has one final toy: Holocake, a helmet that looks more like a basic ski mask, with the light passing through a lens’ hologram rather than through a lens. . The only problem with this holographic system is that it would need a reliable, and most importantly cheap, laser to power it.
Ultimately, all of these solutions must be integrated into a single device. Meta also modeled this helmet of the future named “Mirror Lake”. “There’s nothing in physics stopping us from getting there,” boasts Mark Zuckerberg — though he’s cautious about suggesting a date. Not to mention that, in addition to the display, there are many other problems to be solved at the level of sensors, on-board computer performance, battery…
The first devices to use Meta-Labs’ innovations will likely be too expensive for the general public and reserved for professionals, according to Mark Zuckerberg. This will already be the case with a helmet expected later this year called “Project Cambria”, which is aimed at businesses and will be the first to embed a gaze and face tracking system so that avatars can make eye contact and communicate with facial expressions. One small step for the Metaverse, one giant leap for Facebook engineers.