Twitter Moves to Curb Manipulated Content Including 'Deepfakes'

Twitter unveiled a plan Tuesday to curb the spread of manipulated content including "deepfake" videos as part of a move to fight misinformation which could result in violence or other harm.

The policy was announced after Twitter asked for comments last year on ways to reduce "synthetic and manipulated media" on the online platform that could deceive people during election campaigns or provoke violence or physical harm.

Twitter, which along with other social platforms has been struggling to respond to concerns over misinformation, said its new policy calls for a mix of warning "labels" for tweets that include manipulated images or video, and removal of the tweets.

The move comes amid growing concerns over "deepfake" videos altered using artificial intelligence, along with other kinds of manipulation to deceive social media users.

Twitter vice president of trust and safety Del Harvey said the new policy addresses not only deepfakes but other kinds of manipulation, sometimes described as "shallow fakes" or "cheapfakes."

"This isn't a deepfake rule," she said.

"We want to address any incident where media has been altered or fabricated."

The decision on whether to include a label or remove the content will depend on "the likelihood and severity of harm that could result," Harvey said during a call with journalists.

Twitter said in a blog post it would begin enforcing the new policy on March 5, alerting users to manipulated pictures or video, and sometimes offsetting them with links to more information about the subjects.

"You may not deceptively share synthetic or manipulated media that are likely to cause harm," group product manager Ashita Achuthan and head of site integrity Yoel Roth said in the blog post.

"In addition, we may label tweets containing synthetic and manipulated media to help people understand the media's authenticity and to provide additional context."

Politicians included
The new rule will apply to politicians and their campaigns, according to Harvey.

"If the media is altered or fabricated, regardless of who the individual is, this policy will still apply," she said.

Under the policy, for example, doctored videos of former US vice president Joe Biden and of Democratic speaker of the House of Representatives Nancy Pelosi that caused recent controversies on social media would violate the new rule, according to Twitter executives.

"Whether you are using advance machine-learning tools or using a 99-cent app to slow down videos on your phone, our rules apply to the content not how the result was achieved," Roth said.

"Does the end tweet result in confusion or misunderstanding or result in a deliberate attempt to mislead people? That is the question."

The announcement comes a day after YouTube said it will remove election-related videos that are "manipulated or doctored" to mislead voters, amid heightened concerns on efforts to spread misinformation that could influence elections around the world.

Facebook, which has its own policies on misinformation, in January said it would ban deepfake videos while allowing heavily edited clips so long as they are parody or satire.

Filtering for fakes
Factors Twitter will weigh when deciding what content is deceptively manipulated or outright fabricated will include how heavily it is edited; whether audio has been dubbed over or removed, and whether those pictured are simulations.

Also on the banned content list were threats to privacy or free expression in the form of "stalking or unwanted and obsessive attention; targeted content that includes tropes, epithets, or material that aims to silence someone; voter suppression or intimidation."

Roth said Twitter will seek to be "proactive" in the policy "to reduce the burden of people to report to us."

Along with showing warning labels on doctored imagery before it is "liked" or shared, Twitter will prevent it from being included in recommended content and will typically provide additional information intended to counter misperception.

Twitter said the new rule was based on a survey of US users as well as feedback from people around the world who chimed in using tweets.

Globally, more than 70 percent of Twitter users who voiced opinions said that taking no action on abusively doctored tweets would be unacceptable, according to Achuthan and Roth.