How Meta's Hyperscape Transforms Phone Cameras into Gateways to the Metaverse: Explained

Mark Zuckerberg is intensifying Meta's focus on metaverse initiatives, including hiring experts in generative AI. Recently, Meta showcased a demo of its latest technology, Hyperscape, which turns a smartphone camera into a gateway to photorealistic metaverse environments. According to Zuckerberg, this tool allows smartphones to scan 2D spaces and transform them into hyper-realistic metaverse backgrounds, enabling users to explore and interact within these digital worlds.

What is Hyperscape

During a recent demo of Hyperscape, Zuckerberg was heard describing it as“pretty wild”. Explaining the concept, the tech mogul said, “We are working to bring photo-realistic avatars into the metaverse. We are also bringing photo-realistic spaces to the metaverse. We call this Hyperscape. You can use your phone, scan a room, and recreate it.”

Meta employs a 3D volume rendering technique called ‘Gaussian Splatting', which leverages cloud rendering and streaming. This technology transforms spaces scanned by smartphone cameras into immersive, virtually viewable environments.

Jonathon Luiten, a Meta official and research scientist at the company's metaverse division, Reality Labs, described Hyperscape as a technology that allows users to create "high-fidelity replicas" of their physical environments and embody them in Virtual Reality (VR).

In the demo released for Hyperscape, multiple photos captured by a smartphone camera are seamlessly integrated into a high-resolution video format. Luiten shared a video on his X profile demonstrating how Hyperscape stitches these images together to create a lifelike setting.

Presently, the app for Hyperscape has been made available only on Meta's Quest 3 headset, that is priced at $429.99 (roughly Rs. 36,098) for the 128GB variant. It runs the specially designed Meta Horizon OS. The technology was first showcased at the Meta Connect 2024 event on September 25.

Meta's Tryst with Metaverse

In September 2021, Zuckerberg rebranded Facebook as Meta, signaling his commitment to metaverse technology.

Supported by blockchain networks, metaverse ecosystems are photorealistic virtual representations of real-world objects. According to Meta, these ecosystems have the potential to transform digital interactions. Platforms like The Sandbox and Decentraland allow users to participate as avatars, enabling them to play games, collect NFTs, and hold virtual meetings and social gatherings in hyper-realistic environments.

Since being established in 2022, Meta's Reality Labs unit has logged losses repeatedly. The unit lost $13.7 billion (roughly Rs. 1,12,200 crore) in 2022, with a subsequent loss of $46.5 billion while generating nearly $11 billion (roughly Rs. 91,744 crore) in revenue in the fourth quarter of 2023.

Despite these financial setbacks, Zuckerberg has stayed loyal to his belief that the metaverse technology will become popular in the coming times. Zuckerberg's bet on the metaverse initially looked like a colossal mistake, erasing over $100 billion (roughly Rs. 8,39,113 crore) from his net worth. As per recent figures on the Bloomberg Billionaire's Index, however, Zuckerberg's net worth has climbed almost sixfold in less than two years to $201 billion (roughly Rs. 16,86,617 crore), the first time he's exceeded the $200 billion mark (roughly Rs. 16,78,226 crore) -- making him the fourth richest individual in the world.