July 17, 2009

PubSubHubbub: Real-Time Feeds and Real-Time Feedback Too?

Getting data from one place to another can never be too fast. Just a year or two ago, the speed of RSS was considered excellent, but now, with latency of RSS feeds getting to destinations such as Google Reader or other networks ranging from minutes to hours, combined with issues related to repeated polling at source sites, teams of talented engineers have taken on these problems, with programs such as FriendFeed's Simple Update Protocol (SUP) and the new PubSubHubbub taking the most-visible route.

PubSubHubbub, an protocol designed as an extension to Atom, is designed to reduce the time for updates to flow from one network to the next to almost zero, and practically eliminates the need for downstream services to constantly ask for the data.

Recently, we learned that a few major players in the feed distribution system had signed on to the PubSubHubbub train. First, last Friday, FeedBurner announced initial support for the service, making it built in for all publishers who had feeds enabled with the PingShot option (myself included). Second, Google Reader integration was demoed at the TechCrunch Realtime Crunchup, if only in prototype form. And third, FriendFeed announced support almost immediately, meaning that if I publish a post here, it should hit both FriendFeed and Google Reader practically in an instant, without any manual updates on my part.

An avid user of both services, the announcements were excellent, but in the last week, I hadn't really seen any difference. Posts made on the blog didn't hit FriendFeed right away and I didn't see updates in Google Reader any more quickly. So I did a test post Wednesday night at 11:11 p.m., titled "PubSubHubbub Test: 11:11 p.m." and nothing happened. Minutes later, I deleted it. But it didn't go unnoticed. After 1 a.m. I got an e-mail from Brett Slatkin, one of the engineers behind the protocol, asking if it had worked. That's right. After 1 a.m., one Google engineer behind this very cool tool cared enough to send me a personal e-mail and make sure all was well.

I said that I hadn't seen it work, and asked if it was, as usual, user error. But I was assured that "Nope you did everything right!" and that as things matured, they would "get the kinks worked out."

This kind of friendly, helpful outreach is the type of very real and personal service that one doesn't expect from big companies like Google, yet it happened, and it was very much appreciated. As I've stated time and again for years here on this blog, anything that can be done to help distribute data further and more quickly I am a big fan of. That I am a big fan of RSS, Google Reader and FriendFeed is no surprise to any of you, and PubSubHubbub looks like it is going to be a major boost to the entire ecosystem.

Just take a look at the test I did again tonight. At 1:08 a.m., a post from a dev server took less than 4 seconds to get from the blog to FriendFeed. (see here) In fact, it was probably faster, but it had already landed before I changed browser windows. Brett may not be "4 seconds" fast, but he's doing a fabulous job engaging with the growing PubSubHubbub community, as you can see from his comments on FriendFeed. This is the way tech and Silicon Valley should be.