Google explains Google Docs outage

Not the best week for cloud services.

Users of Google Inc.'s popular Google Docs suite of web-based applications were unable to use the service for some time on Wednesday. Alan Warren, Engineering Director at Google posted an explanation on the official Google Docs blog.

The outage is the result of an update that should have made the service better. The change was designed to improve real-time collaboration within document lists, but a lurking memory management bug showed its ugly head when the servers came under heavy usage.

"Every time a Google Doc is modified, a machine looks up the servers that need to be updated. Due to the memory management bug, the lookup machines didn't recycle their memory properly after each lookup, causing them to eventually run out of memory and restart," Warrern writes.

"While they restarted, their load was picked up by the remaining lookup machines - making them run out of memory even faster. This meant that eventually the servers couldn�t properly process a large fraction of the requests to access document lists, documents, drawings, and scripts which led to the outage you saw on Wednesday."

Google's automated monitoring alerted the team to a spike in failure rate of attempts to access Google Docs. The engineering team atarted rolling it back 23 minutes after the first automated alert. They then doubled the capacity of the lookup service to mitigate the impact of the memory management bug, and the rollback completed 24 minutes later. Within five minutes the service was restored and everything got back to normal.

Read more: http://googledocs.blogspot.com/...

Written by: James Delahunty @ 10 Sep 2011 3:26

FreddyF

Ah, the joys of Cloud Services. Something to look forward to if you use it. More users, less reliability. Companies are very slow to make capital investments to improve capavity.

10.9.2011 04:51 #1

KillerBug

So, in short...

With cloud computing you can't test bug fixes on non-critical systems...you just have to toss the bug fix onto the server that everyone is using and hope for the best. Oh, and it isn't easy or quick to uninstall such updates if they are bad.

The idea of the cloud computing projects like this is that it should allow a cheap desktop to do things it cannot do with internal hardware alone. Thus, a $300 laptop can play the latest games and run high-end CAD software smoothly. That was how cloud was sold, that and remote backup/file sharing while the PC is turned off.

I can see the value of the cloud for remote backup...I wouldn't trust it by itself and I want AES encryption, but the chances that my house and the server room holding my data would both be destroyed is very slim. I can certainly see that cheap laptops running high end software with minimal battery drain is a good thing too. The cloud has a lot of great possibilities, and I am optimistic about the future of the cloud.

...BUT...

There is no reason to have your office suite in the cloud. It doesn't use much power to edit a spreadsheet, if you can't do it smoothly and quickly with a $300 laptop, then there is something seriously wrong with the application or the laptop, and if that is the case, then the cloud would be practically useless.

There is virtually nothing to be gained by going to the cloud in this case. There are some unique features of google docs, but there is absolutely no reason why these online features couldn't be built into PC-side apps, providing enhanced functionality when the cloud is accessible, and providing basic functionality the rest of the time so business never comes to a screeching stop.

If Google Docs and Open Office were the same price, Open Office would be the better choice...but open Office is free and Google Docs is actually rather expensive...it just makes no sense that people are using it.

10.9.2011 07:41 #2

KillerBug

LoL...now Microsoft Office 365 is down too...not a great week for overpriced cloud-based office suites.

11.9.2011 06:31 #3

AfterDawn: News