Mark Zuckerberg's Jarvis AI: Everything You Need to Know

Earlier this week, Mark Zuckerberg announced that he has completed his personal challenge for 2016 – building a home Artificial Intelligence (AI) assistant called Jarvis. In a series of videos and Facebook posts, Zuckerberg has explained what Jarvis is, and what are the tasks it can be programmed to do at home. The videos posted by the sixth-richest person in the world show the Jarvis home AI doing daily household tasks for the Zuckerberg family, such as cooking some toast, and taking care of Max, the Facebook CEO’s infant daughter. Scroll down to find everything you want to know about the Jarvis home AI.

What is Jarvis?

Jarvis is an AI system Zuckerberg has built to control his home and perform basic tasks, such as turning the lights off or on, control a particular room’s temperature, playing music, opening doors, and so on. The home AI uses natural language processing and speech recognition to understand the user’s voice, and the context of the command, in order to perform the tasks it is asked to do. It also has facial recognition capabilities and, therefore, can open doors of the home for known guests, or tell Zuckerberg where each member of the family is, whether Max is asleep or playing around, and so on.

Why did Mark Zuckerberg build Jarvis?

Building Jarvis, and running 365 miles, were Zuckerberg’s personal challenge for 2016; his annual challenges in the past have been to learn Mandarin, read two books a month, and to meet a new person every day. The name is inspired by the Jarvis AI assistant used by Tony Stark/ Iron Man in the Marvel comic books and movies.

What can Jarvis do?

Apart from what we mentioned above, Jarvis can tell you the schedule for the day, teach languages, inform you what others in the home are doing, prepare toasts, set up video conferences, and play movies. Jarvis can also make remind you to, for example, go to the gym. It can even make suggestions, such as offering to make a toast when you are looking to whip up some breakfast, and crack a few jokes.

In the latest video Zuckerberg posted on his profile, Jarvis show's a personality of his own, joking that his talents are being wasted in mundane household tasks and ending the video with “that’s it, I’m joining The Avengers."

Who is the voice of Jarvis?

Jarvis is voice by Hollywood veteran and Oscar-winning actor Morgan Freeman. In an interview with Fast Company, Zuckerberg said that more than 50,000 people suggested using Freeman’s voice when the Facebook asked the public who should voice the home AI. After meeting the Hollywood star at an event, Zuckerberg sent him a request to voice the AI system, and Freeman agreed. Robert Downey Jr. – who plays Iron Man in the movies – had offered to do the voice too.

However, Freeman has recorded only a “limited set of commands” that Jarvis will say, according to a Fast Company report.

How do Zuckerberg and family operate Jarvis?

The Zuckerberg family can either speak directly to the Jarvis AI, or use a Facebook Messenger bot, or a dedicated app. In the post announcing Jarvis, Mark Zuckerberg said that the dedicated Jarvis app (only the iPhone is ready at present, while he is building the Android app) is always listening for instructions, similar to what Google Now and the Google Assistant do.

How does Jarvis work?

Jarvis, like other AI, learns the user’s behaviour and habits to perform the tasks. For example, Jarvis can understand synonyms (family room and living room, for example), and context (tracking key phrases said by the user to perform the necessary action). Open-ended requests, such as “play me some music,” need context and user’s past behaviour to execute, which Jarvis learns over time as it is used. If it gets something wrong, Zuckerberg corrects it and tells it how it needs to be done, so that the home AI learns and does better next time.

What is next for Jarvis?

Zuckerberg plans to build more capabilities into Jarvis, and has asked the public for more ideas. He also said that he would like to explore teaching Jarvis how to learn skills itself rather than having someone teach it. An Android app to control Jarvis is in the works, as is making the AI available publicly.

Zuckerberg also said in a post, “For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other.”