The Future Will Be Augmented

Facebook had already begun adding camera effects to its apps, letting users overlay objects, animations and filters on their images—an unabashed knockoff of Snapchat’s popular AR-powered Lenses. With a new open platform where developers can create their own effects, art and 3D games, Facebook is betting that it can become the go-to destination for AR experiences, a WeChat-like repository of third-party apps- within-its-apps. 

After years of dormancy, the hype around AR is ratcheting back up. Beyond Facebook’s augmented ambitions (which include, down the road, a wearable device), there’s Google’s four-year-old Glass, Microsoft’s HoloLens and the mysterious, well-funded Magic Leap—along with a rumoured device from Apple. According to market research rm CB Insights, 49 AR companies have secured equity financing deals since last autumn—a 75% increase from the year before. 

They’re all vying to dominate a future where the separation between the physical and the digital is wafer-thin, and you won’t need a keyboard or a touchscreen to navigate it. “Augmented reality is the next mobile computer, the next OS, the next social platform,” says Ori Inbar, founder of Super Ventures, a VC rm specialising in AR. “The smartphone is dead; it just doesn’t know it yet.”

What pieces of this hyperbole may actually prove out? Here’s our three-part guide to how AR will actually unfold. 

Our phones will be the gateway—for now

For the better part of a decade, developers have been promising to use smartphone cameras to enhance our perspective on the world. In 2009, Yelp introduced its Monocle feature, annotating users’ camera lenses with ratings for local businesses; a spate of similar apps followed, but none gained any significant traction. It was rainbow-vomiting millennials who pushed the technology into the mainstream, with the introduction of Snapchat’s addictive, selfie- enhancing Lenses in September 2015. By the following August, Pokémon Go had 100 million downloads, as people peered through their smartphones to hunt for Charmanders and Squirtles in habitats across the globe.

The ability to replace your nose with a dog snout or capture a Jigglypuff lurking outside your favourite café may appear frivolous, but it’s actually profound—a clever way of easing consumers into AR, without calling it that. Snapchat now has a promising advertising platform with its sponsored lenses, which can be tied to specific locations. Niantic, the developer behind Pokémon Go, created a hit that has generated an estimated R12 billion-plus in revenue. In the current rush to create the next big AR app—from multiplayer games to more practical applications like interactive travel guides and shopping assistants—success may ultimately depend on the formula these two companies laid out: social interaction grounded by superior location intelligence. 

At the same time, smartphones are growing more sophisticated. Since 2014, Google has been developing its Tango platform that gives mobile devices spatial awareness. Late last year, Lenovo released the $500 (R6 400) Phab 2 Pro, the first Tango-enabled smartphone. 

Using multiple cameras and advanced, motion- tracking sensors, the Phab 2 creates 3D maps from two-dimensional images. Train the phone’s lens on your living room, and Tango will know the lamp is 200cm to the left of the couch. You can then use a Tango-optimised app from e-commerce giant Wayfair to see how a (virtual) coffee table looks between them. Lowe’s Home Improvement has a similar app, along with one that allows customers to use the phone’s camera to navigate its stores. 

Such efforts are early, and their executions fairly crude. Lenovo’s embrace of Tango is more of a proof of concept than a groundbreaking device. But that may change quickly. The second Tango-enabled phone, the Asus ZenFone AR, landed in July in the US. And according to rumours, the upcoming iPhone 8 will also sport a depth-sensing camera to enable AR apps. Notably, Apple typically waits until a technology is mature—and consumers are ready—before incorporating it into a flagship product. “Once the iPhone has that [camera],” Inbar contends, “it will become a de facto standard.”

Wearables will be refined in the workplace

Despite the success of Snapchat and Pokémon Go, no one believes the future of AR consists of staring into smartphones, chasing mythical creatures. 

That’s because the phone is a less-than-ideal interface. “Let’s say you walk into a supermarket that’s enabled with augmented reality,” says Tuong Nguyen, principal analyst for research rm Gartner. “How many times during your shopping trip are you willing to take out your phone? How long are you willing to hold it up?” The biggest hurdle for AR, Nguyen says: “It needs to be built into the glasses I’m already wearing.” 

Today, there are some 50 AR headsets in production—from basic eyeglasses that can display 3D images to $20 000 (R230 000) industrial-strength helmets from maker Daqri. But none are small, cheap or elegant enough for mass appeal. So for the next few years, AR devices will be found primarily in work environments, where their cost and appearance don’t matter as much. ABI Research projects that the AR market will grow to $96 billion (R1.2 trillion) by 2021, with 60% of that going to industrial and commercial uses. 

Google Glass, for example, has found a home on the factory floor after failing to take off among consumers. Boeing uses Glass to display technical diagrams to workers assembling electric wire harnesses for aircraft, leaving their hands free to perform tasks. (When you’re cutting assembly time by 25%, no one calls you a Glasshole.) And it’s not just Google Glass: Med-tech startup Scopis has made a HoloLens app to guide surgeons through spinal surgery. 

At Minneapolis’s Mortenson Construction, contractors can don a Daqri Smart Helmet, walk through a 3D model of a hospital under construction, and see where the plumbing will be routed before it’s actually in place. 

Widespread industrial use won’t just change the way we work; it will inform future consumer-facing products. Just as industrial workers use AR to summon remote assistance during complicated manoeuvres, homeowners who want to retile their bathrooms may one day turn to a pair of glasses for virtual walk-throughs and diagrams.

AR will surround us

In the meantime, AR is continuing to pop up in everyday devices. If your car’s rear-facing cam shows you a squiggly red line as you’re about to back into a tree, you’re using AR. Smart mirrors are being rolled out at Sephora stores to enable virtual makeup testing, and at Neiman Marcus to let shoppers change the colour of their out ts or try on prescription glasses. In the same way that ‘adaptive cruise control’ and ‘lane-change assist’ are leading us toward fully autonomous cars, AR will be insinuated into our lives one feature at a time.

Augmented features are also likely to seep further into inexpensive wearables, as seen in Snapchat’s video-capture Spectacles and wireless earbuds like Apple’s AirPods. Doppler Labs has already released its Here One smart earbuds, which let you amplify certain frequencies and filter out others—augmenting your aural reality. CEO Noah Kraft sees a future where AR exists naturally in your ear. “Say you’re walking down the street and all of a sudden Siri pops into your head and says, ‘Hey, your next meeting is running 15 minutes behind,’ ” he says. “In our world, that doesn’t distract from what’s going on around you.” 

Still, bringing the sophistication and reliability of industrial applications to a device that fits seamlessly into our daily lives is a daunting task. The technical challenges are steep, and it’s unclear whether the public will embrace yet another wearable (and if the content will be good enough to convince them to). Nonetheless, Apple is reportedly plunging ahead, as are Google, Facebook, Microsoft and many others. 

For tech rms, getting skin in the AR game may simply be a matter of survival. Just as the Internet and mobile radically changed the tech landscape, AR has the potential to create new giants while humbling old ones, says Piers Harding-Rolls, director of games research for IHS Technology, a London-based research firm. 

The future will be augmented. But by the time it happens, we may not even notice.