Google Made a Small Mistake, Causing a Widespread Internet Outage in Japan

Google has been linked to a wide Internet outage that occurred in Japan on Friday in what has been a big reminder of how inter connected services are and how any disruption in the chain could affect services that are seemingly unrelated.

Several people in Japan reported an Internet disruption on Friday around 12.20pm local time. Even as the outage lasted for about 40 minutes, it was reportedly enough that a significant population couldn't access train reservation services and their bank accounts, according to local media reports. The outage affected the networks of NTT Communications, which runs the largest service provider in the country OCN, and KDDI.

Google was the cause of the whole mess, according to reports, by botching up a border gateway protocol (BGP), the resulted of a large number of peer prefixes being sent by Google to Verizon. The Register noted that the downtime occurred because Google "leaked" a big route table to Verizon, something described as BGP route hijack. BGP is a protocol for distributing routing information between networks. This, according to the report, caused the traffic that was supposed to go to NTT Communications and KDDI to be sent to Google. Because Google didn't know what to do with the additional traffic, things got stuck.

BGPmon, an industry recognised expert in network monitoring and routing security, said over the weekend that more than 135,000 prefixes on the Google-Verizon route had been accidentally announced, causing the disruption. "Google accidentally became a transit provider for Jastel by announcing peer prefixes to Verizon. Since Verizon would select this path to Jastel it would have sent traffic for this network towards Google. Not only did this happen for Jastel, but thousands of other networks as well," the company wrote.

The report added, "It's easy to make configuration mistakes that can lead to incidents like this... In this case it appears a configuration error or software problem in Google's network led to inadvertently announcing thousands of prefixes to Verizon, in turn propagated the leak to many of its peers."

The situation became so severe that the country's Internal Affairs and Communications directed the Internet service providers to report why the Internet outage occurred, Japan Times reported.

Google later admitted its mistake in a statement to The Asahi Shimbun, with a spokesperson saying, "We set wrong information for the network and, as a result, problems occurred. We modified the information to the correct one within eight minutes. We apologize for causing inconvenience and anxieties."