January 11, 2010

In Case of Real-time Clogs, Check Pipes for Stoppages

At the conclusion of 2009, I named Pubsubhubbub as the top new Web service to debut in the year, as it best exemplified our rapid adoption of real time technologies to push data from one site to another, and reduce or eliminate the need for downstream aggregation services to repeatedly request updates from source sites. I have configured my blog to be enabled by Pubsubhubbub, share my Google Reader items downstream using a Pubsubhubbub-enabled tool, and recognize the benefits of the protocol by way of fast updates on social networks including Facebook and FriendFeed. So, having grown accustomed to near-instantaneous flow, hiccups in the system rapidly gain my attention.

For the nearly 400 people who follow my @lgshareditems Twitter account, seeing my own personal filter of technology news, it would appear I have taken a hiatus from sharing the best of Google Reader, despite what some might consider a full-time obsession. But this assumption would not be true, as it is simply a symptom of a break in the flow of data.

How Pubsubhubbub Works, More or Less

Similarly, I noticed that my Google Reader shares were not populating FriendFeed in seconds, or even several minutes to hours, after they were shared. To force the updates to make their way into my stream, I needed to force FriendFeed to poll Google Reader and pull down the latest, something I haven't had to do ever since they supported Pubsubhubbub shortly after its introduction. So something was clearly broken.

In parallel, other Pubsubhubbub-enabled items continued to act as they always had. Blog posts flowed as smoothly as ever, and after asking some techies to look into it, they reported the hub itself was fine, but instead, the "Reader/Ping" bridge that updates the Pubsubhubbub hub had found a glitch.

But There Is Currently a Break In One Pipe

While my natural inclination was to wonder if maybe Pubsub had grown to big for its britches, and stretched under load, it was just one of the upstream services that has hit a bump. It'd be worth overlooking, out of deference to the good folks at Google, but given the near-outage of about 48 hours, albeit on a weekend, it's gained awareness, and I have had others ask me what had caused the lags.

The Internet continues to be "a series of tubes", all interconnected to get us our data from one place to another, and even the best, latest, cool toys are interdependent on other sources, and no product, so far, is failsafe. If you've noticed a lag in your shares to FriendFeed and other networks, or you use Reader2Twitter on Twitter, and saw your updates disappear, yes, there is a disturbance in the force right now, and it should be solved soon, from what I understand.

For more on Pubsubhubbub, check out their new video posted last week: