What Is Magic Leap's Mixed Reality, How Does It Work, and Does It Matter?

By now we've all probably heard the words virtual reality and augmented reality, and probably know the differences between the two as well. And then there's Magic Leap, a secretive company that has been massively funded by Google and others, raising ever increasing amounts thanks to Alibaba and more. Magic Leap talks about yet another kind of reality - something it calls 'mixed reality'.

The company has been quite stealthy about its technology in the past, giving only glimpses and a short demo in October. But now, a large profile by Wired, along with a newly released YouTube video, tell us a little bit more about Magic Leap.

As you can see in the video above, Magic Leap seems to promise Minority Report style floating controls and virtual displays, which is very cool. It's also somewhat reminiscent of early demo videos of Microsoft's HoloLens. Microsoft also touted a future where screens vanish, though early impressions of the device seem to focus more on its potential, than what it delivers.

Virtual, augmented, and... mixed?
In case you're not already familiar with the first two terms, let's go over them quickly. In virtual reality, you're stepping into the virtual world - you strap the headset to your face, and you lose sight of the real world. You could interact with other people, but the entire experience happens inside the virtual space. Great if you want to explore the view from the top of Mt. Everest, but a little impractical when you're just trying to get through some spreadsheets.

With augmented reality, you're bringing the virtual world into the real world instead. Google Glass was an early example, and HoloLens is a more advanced one. The idea here is to bring notifications, pop-ups, and information about real world objects, so you could see a line on the road telling you where to go, instead of having to look at the map on your phone's screen. Or, if you're looking at a product in a store, you could peek at its barcode, and see a window pop-up on the side showing you its price on your favourite e-commerce site. AR won't "transport" you the same way VR does, so it's not ideal for gaming, or watching a 360-degree movie, but it can enhance everyday life.

Magic Leap looks a lot like AR, as you can see in the video above, but in the Wired profile, its creators insist that it's different, and prefer the term 'mixed reality'. In Wired, the following distinction is made (paraphrasing a little for clarity):

In augmented reality, the visible world is overlaid with a layer of digital content. In mixed reality, virtual objects are integrated into the natural world. So a virtual ball under your desk would be blocked from view unless you bend down to look for it. In theory, MR could become VR.

How does it work?
The problem is that the lengthy profile is actually very short on real details, so we don't know much about Magic Leap's mixed reality technology. Things like its field of view, refresh rate, display resolution, even the display technology being used (!) are not really talked about. All we know is that like HoloLens, it uses a projector to display images on a semi-transparent material, reflected into your eyes by beam-splitting technology.

We don't know much else about it - whether you're talking about apps, compatibility, hardware requirements, or even what it would be used for. If you go back and look at the video, you'll see someone looking at a list of chats, opening a message, and looking at a spreadsheet, all on non-existent monitors. Which probably means that you're going to have a hard time just adjusting the angle at which you're looking at a virtual "screen", and although no control interfaces were shown, it gave the impression of being a "look to click" system, which is incredibly cumbersome and not useful at all.

There is a lot of talk about "photonics" and "digital light fields" but these are so few and far between that there is no really clear answer on what Magic Leap really is, or how it works. As of now, it looks cool, and sounds cool, and could well be a revolutionary product. We just don't know much about it, other than the fact that it's raised over $1.4 billion (over Rs. 9,000 crores) from investors.

Does it matter?
Putting a flat window on your side to browse shoes is not empowering. It's not even an improvement over looking at the same shoes on your phone or laptop, where you can easily share the view with others, and have total tactile control over your interaction. The advantage with Magic Leap is that you can pull out a realistic, 3D image and look at it from all sides. That part is great, but everything else in the video (apart from the jellyfish at the end) looks like something that's better done with a computer or phone. And while it is cool that you can bring virtual objects like the jellyfish into the real world,it seems more useful for a tech demo than a product.

There's a part in the video where someone has sent a report on Mt. Everest, which looks cool, until you realise that there is absolutely no value in seeing a mountain peek out of a map. If you think PowerPoint decks are bad, wait until everyone is expected to have an augmented, sorry, mixed-reality kit.

Also, it's very unlikely that Magic Leap can take a regular presentation and turn it into a 3D diorama as shown in the video, which means that the presentation had to be created using a similar headset. This raises obvious questions about compatibility - does it mean that your fancy expensive headset will only be useful for seeing augmented, sorry, mixed-reality if other people also own one?

All nitpicking aside, if the Magic Leap can deliver even a simple AR experience without making you look like an idiot (hi Google Glass) or being huge and bulky (hello HoloLens), it will be an amazing achievement. But at this stage, although Wired used the headline Hyper Vision, it might have been more accurate to drop the 'r'.