Microsoft, Longhorn, and RSS: I'm having IE4 flashbacks!

Steve Rubel's recent post ("What default feeds will IE7 and Longhorn carry?") made me think about my experience back in 1997 at CNN.com and CNNSI.com when IE4 was released with the Active Desktop driven by CDF, a syndication format now in the dustbin of history (along with ICE). CDF wasn't RSS, but it also wasn't that different conceptually. It's a bad analogy from a technical purity standpoint, but if you don't remember Active Desktop, think of it as Pointcast built into the OS and driven by RSS-like feeds in the background. Both CNN.com and CNNSI.com (that's CNN and Sports Illustrated) were default "channels" -- which is why Steve's post got me thinking about the technical implications of Microsoft's RSS announcement. Working on the tech side for a major site providing default channels (which were really just text feeds), I was on the front lines back in '97 on Microsoft's IE4/Active Desktop rollout and it was UGLY. I'll get to the details in a moment, but for some reason, this is feeling a little like deja vu (although it is certainly different).

So, if you're caught up in RSS fever (and who isn't at this point?), you might be thinking, "Where did this talk of Active Desktop come from? What does that have to do with anything?" I'll point you to this Microsoft Knowledge Base article, "How to Configure the Active Desktop," particularly this bit:

You can use the Active Desktop to make your desktop act like your own personal Web page. You can display pieces of your favorite Web sites directly on your desktop and keep them up-to-date automatically. You can easily customize the Active Desktop at any time.

Sound familiar? My preferred aggregator (Bloglines) says this when I load up my home page: Create a personal Bloglines page loaded with the freshest news about the things you love. Hmmm. . . automatic updating, easy customization, displaying the latest news on your desktop, big Microsoft initiatives. . .

Now to the technical part, which I voiced in March 2004 in the context of my Active Desktop experience back in '97 (except I was writing about RSS scaling issues):

Another interesting technical aside -- as our RSS requests have grown quickly, we have noticed increased server loads at the top of the hour as aggregators "wake up" to pull feeds. Not a huge problem for us right now, but the surge has roughly the same characteristics as a distributed DoS attack and could eventually present trouble for really huge web sites unless aggregators become a bit smarter. I was working at CNN.com when IE4 and its Active Desktop with various CDF "channels" was released, and boy was it active. CNN.com and CNNSI.com were default channels in the new browser. All the newly-downloaded IE4 clients absolutely pounded our servers with requests for CDF files. It was a big pain, and I wish I could remember how we dealt with it.

Actually, I do remember how we dealt with it -- we assumed the fetal position and prayed for the barrage to stop. OK, I'm kidding (sort of). Our operations staff at CNN was beyond incredible and included people like Sam Gassel, who probably knows as much about scaling as anyone in the world. I recall a visiting vendor comparing the CNN ops team to Israeli tank commandos -- these guys were the best, and I still feel honored to this day for having worked with them. When we had problems, those problems were very real and very big -- and this Active Desktop problem was a doozy.

We were being absolutely slammed as more people downloaded the IE4 browser (I believe Active Desktop was enabled by default), so we set up a test machine with IE4 installed and monitored its outbound HTTP requests. Although no user was actually sitting at the machine, we discovered that it was making thousands of requests to fetch content that it had (supposedly) already cached, and it was pre-fetching entire web pages to feed the hungry
Active Desktop (hmmmm, Google Web Accelerator, anyone?) As I recall, Microsoft was very slow to respond to our pleas for help to the point of being almost disdainful when we finally had a conference call with a team over there to air our concerns. I can't remember exactly how the situation was resolved, but I didn't leave with a good taste for Microsoft.

That was eight years ago, of course, so I'm glad to see that Longhorn Team RSS has a blog with open comments. Even if there were problems like we experienced back then, I think it would be impossible for Microsoft to hide from them anyway (not in a Scoble world anyway). Also, scaling to handle RSS traffic seems to be better understood now -- look here to see how we dealt with some minor scaling issues here at InfoWorld last year (as described by our IT manager Kevin Railsback).

I'm guessing that the Longhorn RSS reader will be a lot gentler than the old Active Desktop driven by CDF -- but if it ends up roughing up a few sites along the way, don't think that something like that hasn't happened before.