Disney Unveils Real-Time High-Fidelity Performance Capture Technique

The method can realistically incorporate details such as deepening forehead wrinkles and crow's feet around the eyes and can potentially take character animation to an altogether new level.

"This could open up a variety of new applications, from casual home users who wish to give their own likeness to a game character to commercial applications where, for example, a customer can apply and view virtual makeup," said Derek Bradley, research scientist at Disney.

"In film production, our method would allow a director to assess an actor's performance transferred to a digital creature even as he is filming," he added.

The method, which requires only a single video camera such as a webcam, is the first to both operate in real-time and to capture facial features in high resolution, including such details as wrinkles.

Capturing facial details has proven critical for conveying emotions or the state of mind of a character and audiences are sensitive to subtle characteristics of the face.

Animators thus devote great care to tracking facial expressions of actors so that they can create realistic virtual characters.

Until now, these facial performance capture methods have been designed to either capture these details in high-resolution offline, or to record less detail but at high speed.

The new method represents a substantial step towards the goal of attaining both high quality and high speed.

"By learning the relationship between local image appearance and local wrinkle formation, we can reconstruct plausible face wrinkles in real-time from a single camera," Beeler noted.

The researchers will present their facial performance capture method at ACM SIGGRAPH 2015, the International Conference on Computer Graphics and Interactive Techniques, in Los Angeles from August 9 to 13.