Behind the Scenes of Face Swap Live, the 'Creepy' App That Launched a Thousand Memes

When I smile, Hilary Clinton smiles back. When I raise an eyebrow, hers lifts in unison, like a bizarre game of Simon Says. When I grimace, the wrinkles on her forehead deepen, her lips crinkling and pursing to one side.

Thanks to the wonders of computer vision and a goofy new app called Face Swap Live, I am controlling Hilary's face - with nothing more than the expression on mine.

If you haven't yet experienced the viral, nightmarish joys of Face Swap Live, it's well worth the 99 cents it's currently selling for. Download the app and point your phone's camera at a friend, and it will convincingly map their face, in real time, onto someone else's: yours, a baby's, Beyonce's, Richard Nixon's.

Since appearing in the app store about a month ago, the app hasn't strayed far outside of Apple's most-downloaded offerings, peaking in the United States. at No. 14. And while the technology isn't perfect - the app's first truly ubiquitous meme was a disastrous face-swap between a dad and his baby - the results are lifelike enough, enough of the time, that the "Today" show dubbed it "Kafkaesque" and the Daily Dot called it "creepy."

"It's a lot of fun," said Jason Laan, one of the app's two creators. "But behind the fun, there's some really amazing, hardcore technology."

Laan, a chemical engineer by training, has a long history of turning serious tech to more frivolous purposes: In the eight years since he founded his app development firm, Laan Labs, he estimates that they've launched around 50 products, from Tap DJ ("mix and add FX to your iPod music!") to Dog Vision HD ("see the world how your dog sees it!").

But for Laan, computer vision - the science of training computers to extract and understand information from pictures, the same way humans do - has always possessed a special intrigue. Researchers at places like Google and IBM, with their extraordinary 3-D cameras and lightning-fast processing speeds, had enabled computers to catalog objects, recognize faces and even interpret feelings. Laan and his partner, Will Perkins, began wondering if the iPhone's improving camera and processing capabilities would allow them to try out similar projects, albeit less seriously.

So late last winter, Face Swap Live was conceived. The app that has since launched a thousand YouTube videos, Imgur posts and nightmare memes.

Face-swapping makes a pretty ideal consumer application for the new computer vision tools, incidentally. While the technology is novel, the art form is not: Know Your Meme traces the first instances back to the early aughts, when the visages of an eccentric Vietnamese singer and and a 16-year-old Chinese kid began showing up on other bodies and in other places.

In 2004, the Something Awful forums fatefully began switching the faces of babies and their grandparents. It was an onerous Photoshop process, a labor of lolz, if you will: isolating the faces manually; copying, moving and rotating them; blending and feathering the mismatched edges until the heads and bodies fit. Even all that work made for some pretty unholy collages: The babies' heads pixelated, over-large; the grandparents' shrunken and neckless.

"Wasn't that (expletive) creepy?" exclaimed an SA writer in 2004. "Now I have to go to bed ... Oh, the dreams I'm going to have."

But the appeal of the face-swap has always been its weirdness - the degree to which it inverts and diverges from reality. The best face-swaps are also the most surreal: Tom Cruise as Jack Nicholson, Barack Obama as George Bush, Nicholas Cage as literally everybody.

"There's something about absurdity that gives Internet memes a lot of traction," said Britney Summit-Gil, a doctoral student at Rensselaer Polytechnic Institute who studies Reddit culture. "Absurdity has a long, storied history of entertaining humans." And it's not so different from awe, she says - one of our more viral emotions.

Oddly enough, however, we're moving closer to a world where face swaps are both less "awesome," in the sense of inspiring wonder, and less obviously absurd. Thanks to innovations like the ones that spurred Face Swap Live, face-swapping no longer requires any time at all, to say nothing of expensive editing software and human effort.

Just look at how fast Face Swap Live is, an accomplishment Laan and Perkins credit to a basket of cutting-edge algorithms. When they look at your face, they simply look for reference points - the corners of your eyes where the color changes, the curve of your chin - and then line them up with those points on another face, auto-smoothing and blending them in.

With better cameras, Laan and Perkins say (3D cameras, particularly, like the ones Intel just unveiled at CES), our smartphones could do far more than copy-paste a face. Already, Disney is working on a technology that can map your face down to its individual wrinkles. At Stanford University and Germany's Max Planck Institute, researchers have developed a technique that photorealistically transfers one person's facial expressions to another - not face-swapping, in the traditional sense, but face-hijacking via algorithm.

These researchers believe we're moving closer to a world where remote workers can Skype into meetings half-clothed, their faces mapped onto a body in a business suit. They suspect we'll be able to tweak actors' bad takes and zap unsuspecting bystanders from live TV news.

Far outside the realm of face-swapping, real-time computer vision - particularly of human bodies and faces - will enable a million other technologies: self-driving cars, diagnostic computers, robots that understand emotions and react accordingly. We won't even delve into the more dystopian applications, like live video-manipulation or mass surveillance.

I ask Perkins and Laan about that, because it's seems odd: a silly app that advances a promising, and ominous, technology. Do they contemplate the juxtaposition at all, I wonder?

"We just want to have fun," Laan says. Then they both laugh nervously.

© 2016 The Washington Post