Facebook Patents Tech That Records Ambient Audio From Client Devices to Measure Ad Impressions

It is not surprising when Facebook users think that the social media giant is spying on their conversations. While a large number of people believe that Facebook secretly uses the microphone on their devices to better target ads on its platform, CEO Mark Zuckerberg had refuted this belief a couple of months back. Now, a Facebook patent application has been published by the USPTO that describes a technology which remotely activate microphones on a client device like a phone, using inaudible signals broadcast via TV. Of course, as is the case with all patents, they only represent an area of research that a company is exploring, rather than a definite plan for a product that will be launched commercially.

Earlier this month, Facebook said that it is not secretly using your smartphone's microphone to listen in on your conversations for the purpose of ad targeting. The patent application applied by Facebook in December 2016, and published by the USPTO on June 14. Spotted first by UK-based newspaper Metro, the patent is titled 'Broadcast Content View Analysis Based on Ambient Audio Recording', and details a technology that is meant to determine whether a broadcast advertisement impressions has been served. It involves recording ambient audio from a client device if one or more broadcasting signal is detected, and then sending a fingerprint of that audio apart from an identifier of the user of client device, as well as time information, all to determine whether the impression was made.

Here's the abstract: "An online system analyses broadcast content viewed by individuals in a household. Each individual in the household is associated with a client device on which a software application module is executed. When the software application module detects one or more broadcasting signals of a content item broadcasted to the household, the software application module records the ambient audio, including audio from the broadcasting device. The software application module sends an identifier of the individual associated with the client device, an ambient audio fingerprint derived from the recorded ambient audio, and time information for the recorded ambient audio to the online system. The online system, based on the ambient audio data, identifies the corresponding individual and content item and logs an impression for the content item upon determination that there was an impression of the identified content item by the identified individual."

Photo Credit: USPTO

Client devices are defined by Facebook in its patent to mean a device with computer functionality that is also connected to the same network as the broadcast device (like a set-top box). It includes "personal digital assistant (PDA), a mobile telephone, a smartphone, a smartwatch, or another suitable device." Diagrams provided in the patent also show how the technology would know which adult or child within a home was watching a particular broadcast. This means that the tech can potentially be used to build profiles of individual members of a house for better targeted advertising.

As you could imagine, this has caused some apprehension amongst readers, in the light of the various privacy debacles that Facebook has got itself involved in. While the patent doesn't indicate Facebook its actual intent to record audio from users' phones for the sake of advertising, it does indicate the company has explored that avenue of research. Patents can be filed defensively as well, with the aim to prevent other companies from developing similar tech, suing the patentor for having developed related tech, or having an edge in the market if the patentor ever rolls out related tech. It's not certain if Facebook will ever roll out such functionality, whether its recording will be covert (without user permission), whether the recorded ambient audio is meant to listen to the user, whether it will allow humans to listen to the recording, or whether the "audio fingerprint" can contain any potentially sensitive information.

Facebook VP and Deputy General Counsel Allen in a statement to Engadget responded to the fresh controversy surrounding the patent, saying it had been filed "to prevent aggression from other companies," adding that "patents tend to focus on future-looking technology that is often speculative in nature and could be commercialized by other companies." He added that the tech discussed in the patent has not been included in any of Facebook's products, "and never will be."