Scoble Joins The Real-Time Web Conversation

Well, maybe "joins" isn't the proper word, but this showed up on my FriendFeed account today: RSS Shows Its Age. As I've posted about already, I think polling for updates is pretty lame, and asynchronous messaging is the right approach. But yet again, XMPP isn't it.

Enter the next contender: SUP. Took me a while to figure out what precisely it is, but it's essentially a JSON-encoded consolidated feed URL for a batch of URLs. So if you are a service that has a million individual RSS URLs, you provide one SUP URL, which allows clients to make a request and see all URLs that have been updated "recently". The magic sauce is:

• Instead of individual RSS URLs, they get encoded into a nonce (an SUP-ID) to save on transfer costs;
  
• JSON rather than XML;
• Definition of "recently" is up to the publisher so that it can be statically served and cached and changed depending on publisher server load.
  

The end results are you can poll the SUP feed for a whole site, and then find out what you have to hit the individual RSS for. So this makes it faster to poll the million RSS URLs, particularly since at any given time a tiny fraction are actively updating. And it makes FriendFeed feel better about polling Flickr every second to reduce latency.

But it's still polling! It still ultimately doesn't scale. You can make polling suck less, but it will always suck.

Again, asynchronous message oriented middleware systems are the ultimate solution to this, and anything you do in the interim is only postponing the inevitable.