It was a dark and stormy night. The air was quiet. Too quiet. Yet stormy. Suddenly, a beep rang out from a bedside pager. The engineer woke up, grabbing a soda to sharpen his senses. Blogger was down. He needed to bring it back up.

When I get the chance to write my pulp story of a gritty Blogger engineer struggling to keep the site alive, I may look back on this past week as a prime source of choice dramatic fodder. Until then, I, like many of you, will look upon this past week with irritation, disappointment, and maybe even a bit of anger.

You need to look no further than our status blog or perhaps your own experiences to know that Blogger had a significant number of unplanned outages this last week (forgive me my euphemisms?) and a handful of planned ones to clean up from the unplanned ones. It’s been a Murphyesque cavalcade of power failures, fileserver trouble, and wonky network hardware, and I hope you’ll believe me when I say that the Blogger staff is even more sick of it than you are.

First up, our apologies. We really regret these outages, which were a nuisance (or worse) to you. The past week’s performance was not representative of the kind of service we want to provide for you.

More importantly, though, what are we doing to prevent this in the future? Some good news:

• In the short term, we’re replacing quirky hardware and increasing our monitoring to stop problems before they start (forgive me my clichés?). This afternoon’s planned outage did just such a thing.
• In the long term, we’re developing a new version of Blogger with some great new features that is built on technology and hardware that has proven, Google-quality reliability. The current Blogger infrastructure is — albeit in a very Lincoln’s axe way — the same that Google acquired four years ago. Sure, we’ve built on it and expanded it significantly since then, but the truth is we’ve more than out-grown it. The new version is ground-up more scalable and less error-prone.
  

The news gets better: We foresaw the need for the long-term solution, well, a long time ago. Long enough ago that it’s almost done, and you can use it as the new version of Blogger in beta. If you can switch to it (see requirements) you really should. The new version of Blogger is better in almost* every way, including reliability. (It’s worth pointing out that none of this past week’s trouble affected the new version of Blogger or its blogs.)

It’s been a bad week for Blogger, and, as I hope you can tell, we’re not denying it. Instead, we have taken and will continue to take specific steps that make Blogger a more reliable, overall better service for you to use.

Oh, and as a final dogfoodish note, I’m pleased to point out that our status blog is now powered by the new version of Blogger. This means that we will be free of the Catch-22 of problems with the current version of Blogger preventing us from reporting about the problems with the current version of Blogger. (And we’ll fix that bug that makes it look like all the posts came from me. We’re on it.)

* The new version of Blogger is available only in English, which we will remedy very shortly. Also FTP publishing isn’t there yet, but that’s coming soon, too. Once these are in place, the new version will be better than the current version in every way.

There's a saying in the software industry -- it's always good to eat your own dog food. Wikipedia has a great article about this:

"... the company has not merely considered the value of the product for consumers (that is, whether the dog will eat the dog food), but actually is a consumer of the product. When properly executed, this can add a new level of sincerity to advertising and customer relations, as well as helping to shape the product."

Blogger is quintessentially Google dogfood. We use it to publish the main Google Blog, ad-related blogs like Inside AdWords and AdSense, product-specific blogs like those for Reader and Book Search, and even language-specific blogs like Google 한국 블로그 and Google Россия. At last count there were 37 Google blogs powered by Blogger, and even more are in the works.

So when I see posts like these from Michael, Phillip or Juan Carlos, I get somewhat disheartened. They may be missing an important point: because we are eating our own dogfood, Blogger becomes a better, more secure product.

The issues they raise (like the fake Google Blog post, which involved an API bug) typically get fixed and pushed live within hours of their discovery. The Google Blog deletion that Michael mentions was the result of an automated anti-blogspam process and a double-corner case involving a feature of Blogger we discontinued many years ago -- no actual users were affected. Andrea's accidental post on Blogger Buzz was simple human error. Those of us with multiple blogs have probably all made the same mistake at one point or another (Danny even mentioned doing it not too long ago):

"I can completely sympathize with this. About two weeks ago, I posted something to the Search Engine Watch Blog that I meant for my personal blog Daggle. Both use Movable Type, on completely different systems. But I had browser windows open to both of them and just picked the wrong one."

In conclusion, dogfood = yum, even when it has bugs! Blogger is powerful, secure, sophisticated -- and yet a tremendously easy to use tool that enables millions of people around the world to share their lives with each other on the web. And it's only getting better with age.

Want to add labels to all your old posts? There's a new labels management feature to make this easy. On the Edit Posts page, you can select batches of posts and add or remove labels to all the posts at once.


Let loose your internal librarian! Happy labeling.

We've made it really easy to put AdSense ads on your blog using the new Blogger in Beta. If you're using the new layouts-enabled templates, try out the new AdSense Page Element.

You can sign up for an AdSense account from within Blogger, if you don't already have one. We offer a variety of recommended color schemes for the ads, or you can customize the colors. To get started, go to the Template tab of your blog, click Add Page Element, and select AdSense. Learn more.

If you're not using the new layouts-enabled templates, you can still add AdSense to your blog by clicking on the AdSense subtab of the Template tab of your blog.

Check out the Google Earthlings blog! 5 Google folks (plus their dog Sparky) are participating in 24 Hours of Moab the weekend of Oct14-15, and are using their beta blog to document the whole thing. They've even got nifty integration going with Earth and Blogger Mobile:

One of the last features missing from the new version of Blogger in beta — posting photos directly from Google’s Picasa photo organizer — has been released today. Picasa’s BlogThis! button is the easiest, most convenient way to post photos from your Windows or Linux desktop to your blog, and now you can use this great (and free!) program with the new version of Blogger.


There are a few known issues that we are working to quickly resolve, but we know that Picasa users have been eagerly awaiting this feature and didn’t want to make them wait any longer. Happy photoblogging!

(Gerbil photo courtesy Cute Overload.)