Is the Internet Broken Today? Major Outages Spur Chatter

Anthony Freed
By | October 26, 2012

Posted in: Network Security Trends

Widespread packet loss and downtime for some big players has been the cause of quite a bit of chatter today regarding the possible causes for the widespread disruptions on the Internet. Google App, Tumblr, and Dropbox have all been the subject of reported service interruptions. So far, no one seems to know the cause or if the events are related.

Stats provided by the Internet Traffic Report show that some regions around the country were experiencing 100% packet loss figures and measured an overall rate of packet loss at 32%. The stats correspond to a report of performance issues by the Google App Team and other service providers.

"At approximately 7:30am Pacific time this morning, Google began experiencing slow performance and dropped connections from one of the components of App Engine. The symptoms that service users would experience include slow response and an inability to connect to services. We currently show that a majority of App Engine users and services are affected. Google engineering teams are investigating a number of options for restoring service as quickly as possible, and we will provide another update as information changes," Google's Max Ross reported.

Christina Ilvento of the Google App Team followed up on the thread by reporting that most services had been restored by 11:00am PST, though some were still experiencing higher than normal error rates and slow response times.

The latest update from the Google App Team states that "at this point, we have stabilized service to App Engine applications. App Engine is now successfully serving at our normal daily traffic level, and we are closely monitoring the situation and working to prevent recurrence of this incident... As the system recovered, individual jobs became overloaded with backed-up traffic, resulting in cascading failures. Affected applications experienced increased latencies and error rates. Once we confirmed this cycle, we temporarily shut down all traffic and then slowly ramped it back up to avoid overloading the load balancing infrastructure as it recovered. This restored normal serving behavior for all applications. We’ll be posting a more detailed analysis of this incident once we have fully investigated and analyzed the root cause."

Tumblr also reported service interruptions during the same period in a tweet this morning, which stated that "Tumblr is experiencing network problems following an issue with one of our uplink providers. We will return to full service shortly." The message was followed several hours later with confirmation that the issues seemed to have been resolved: "Tumblr is back online - a full postmortem regarding today's service interruption will follow."

There were also reports of disruptions to cloud storage provider Dropbox, though few details are available at this time.

IT Harvest’s Chief Research Analyst Richard Stiennon told Security Bistro the he was "having trouble verifying that anything is unusual at all" regarding the scattered reports. "Could a Tumblr outage explain a 6% drop in Internet traffic? Maybe. Getting reports of high (5%+) packet loss on major carriers. Might have more later," Stiennon said.

You May Also Be Interested In: