Facebook CEO Mark Zuckerberg Builds 'Jarvis' Software Butler for His Home

Mark Zuckerberg's artificial intelligence-imbued software "butler" - named Jarvis - is now in service, and even plays with his family, the Facebook chief said Monday.

Zuckerberg took on the personal project this year, devoting about 100 hours to making a system inspired by the "Iron Man" film character Jarvis as a virtual assistant to help manage his household.

"In some ways, this challenge was easier than I expected," Zuckerberg said in a post on his page at the leading social network.

"In fact, my running challenge (I also set out to run 365 miles in 2016) took more total time."

Jarvis is not a physical robot, but an application Zuckerberg can access through his phone or computer to control lights, temperature, music, security, appliances and more.

The software learns his tastes and patterns, as well as new words or concepts, and can even entertain his one-year-old daughter Max, according to Zuckerberg.

Diagram of the systems connected to build Jarvis.

Knowing faces
Natural language processing and facial recognition capabilities were built into Jarvis, enabling it to understand spoken or texted commands and recognize who is issuing them, Zuckerberg noted.

The software can determine when a guest at the door is expected and let them into the home, while notifying the family that someone has arrived, according to the post.

"One aspect that was much more complicated than I expected was simply connecting and communicating with all of the different systems in my home," Zuckerberg said.

"Most appliances aren't even connected to the internet yet."

Assistants such as Jarvis would not only need devices in homes to be linked to the Internet, they would have to run on common standards, according to the Facebook co-founder and chief executive, who returned to his software-writing roots for the project.

His foray into AI also impressed upon him the importance of getting software to understand context, such as who is speaking and where they are.

"When I tell it to turn the AC (air conditioning) up in 'my office,' that means something completely different from when Priscilla tells it the exact same thing," Zuckerberg said, referring to his wife.

"That one caused some issues!"

Similar problems could arise when asking for music to be played without the AI being told which room, especially if a napping baby is a factor.

The more context AI has, the better it can handle open-ended requests, he noted.

"At this point, I mostly just ask Jarvis to 'play me some music' and by looking at my past listening patterns, it mostly nails something I'd want to hear," he said.

Zuckerberg said he tends to text his Jarvis using a 'bot' he built for the Facebook Messenger service instead of speaking to it, mostly not to disturb people around him.

AI personality
Speaking to the AI had the psychological effect of making him think of it more as a real person, he noted.

"Ever since I built voice into Jarvis, I've also wanted to build in more humor," Zuckerberg said.

"Part of this is that now it can interact with Max and I want those interactions to be entertaining for her, but part of it is that it now feels like it's present with us."

He told of teaching the AI a game in which they ask it who they should tickle, and Jarvis randomly picks someone in the family, including the dog, to target.

Zuckerberg planned to continue improving Jarvis, and said he was more convinced than ever that AI technology would improve greatly in the coming five to 10 years.

"I'd love to have Jarvis control my Big Green Egg and help me cook, but that will take even more serious hacking than rigging up the T-shirt cannon," he said, referring to a popular brand of ceramic cooker.

"In the longer term, I'd like to explore teaching Jarvis how to learn new skills itself rather than me having to teach it how to perform specific tasks."

AI is getting a foothold in people's homes, starting with devices such as Amazon Echo and Google Home speakers which links to personal assistants to answer questions and control connected devices.