Lazycoder

5Jan/052

Sriram Krishnan had the same idea I had

Solving the blog bandwidth problem using Bittorrent

It looks like Sriram Krishnan and a few of his friends had the same idea I had a few days ago. Using Bittorrent to solve some of the bandwidth problems that RSS creates. One thing that came up in his discussion though is the idea of hosting the seed file and the torrent on a central server, a “smoke” server. My idea is centered around the aggregators rather than the blog engine itself. Since the aggregators are causing most of the bandwidth problems caused by polling for changed RSS files, let them host the files (controlled by the user of course).

Maybe a blog engine could support a lightweight “hasRssChanged” method? Maybe the blog writer could “push” a new RSS file out to a small group of “seed readers” only when it’s changed? That would require a few dedicated aggregators though.

The key here could be to either create a new Bittorrent like API for RSS or to modify the existing flock of web servers to allow for sharing of their content, or maybe just specified content, in a distributed, Bittorrent like manner. It’d be almost impossible to modify even just the big two web servers (Apache and IIS) to “host” their content in a distributed manner though.

Still looking for the Reese’s moment.

  • http://www.dotnetjunkies.com/weblog/sriram Sriram

    There are certain problems with the aggregators hosting the torrent. Let’s say aggregator A downloads it first. But how in hell would aggregator B know how to find A? The best solution would be for the blog engine to either keep track of who’s hosting it – or host it themselves. But since that’s not going to happen anytime soon,this idea of a central server might work. Also, letting the aggregators host it means problems with NAT

  • http://www.lazycoder.com Scott

    You build the Bittorrent tracker into the blog engine I suppose. Make it possible for the blog engine to be a tracker. My big question is, how would this system handle updates to the file? Do we push out a new rss file and create a new tracker? Do we set up a system where the rss file is just continuously streamed out to the aggregators? (I don’t think so). Do we restart the tracker and tell the “rssTorrent” clients that they need to re-download the file? If so, does that really solve our problem?