Bittorrent’s DHT And Its DRM Problem

torrentsOver this past month, I’ve been researching a few topics based around BitTorrent, and I’ve come up on one point that I remember well, but which was never adequately explained for people. That was the controversy in 2005 around BitComet and the private flag, especially with version 0.60. It may seem an odd time, almost 10 years later, but it’s actually a key part of the history of Bittorrent, and one that’s always been badly reported (if at all).

To understand the controversy better, it’s necessary to know what the Private Flag is in the first place. Put most simply, it’s a flag that tells clients not to use certain protocols. (If you want to know why the term ‘flag’ is used, it’s just a name for a binary process. A simple example is the flagpole at Buckingham palace – if the royal standard [flag] is flying, the Queen is in residence there. If she’s not, there’s no flag. Simple)

The Private Flag, Bittorrent’s DRM

In fact the flag is a form of access control (more commonly known as Digital Rights Management), designed specifically to control access to peers in bittorrent clients. When the flag is not set, a client can use whatever methods it has available (DHT, Peer Exchange, Local peer discovery, other p2p protocols the client might support) to it to try and find peers. When the flag is set though, it’s not allowed to use anything but the listed trackers to find peers.

Now, some take umbridge with the term DRM being applied to the Private Flag (because those using it are normally against DRM) but it’s an accurate one. It’s an outside imposition (it’s embedded in the torrent file) and digitally restricts a client’s ability to use all its functionality. That is what DRM is.

Some would argue that it’s not DRM, because it’s only on the transport stream, and once it’s downloaded you can do anything with it. that also misses the point. Over the last year or so there’s been a bit of a ‘brouhaha’ (or should that be ‘brew-haha’) over Keurig and their 2.0 system. This was a ‘feature’ in their new coffee machines where they would only work if they had official Keurig K-cups inserted in them. You couldn’t source your coffee wherever you wanted, you could only get it from approved suppliers. This was quite rightly lambasted as DRM, even though, once you had brewed the coffee, you could do anything with the result. In the end, Keurig relented, and it was hailed as a victory.

DRM doesn’t stop being DRM just because the restriction results in something that can then be used without restriction. The restriction is still there and limits the production of that end item. The Keurig design was supposed to limit what coffee could be bought, to drive up sales of Keurig branded coffee pods – they don’t care what you do with the coffee once made, because that’s not their selling point. Or the extortionate prices on printer ink, enforced through DRM on ink cartridges – they dont’ care what you print, or do with that print. Likewise, the private flag is about restricting swarms to certain people, because their business is not the end files, it’s in the access to them. Also, the justification used for both DRM systems (the Keurig 2.0 and the Private flag), the same ‘concern over the quality of the end item’.

There are more examples, such as the Broadcast flag for TVs, which requires certain technologies be used for the transit portion, while not caring what you do with the end product (because that kind of DRM is not concerned about it). A movie with the Broadcast flag on is concerned only that you use an approved piece of equipment to get it to the endpoint (the TV screen) What you do with that movie once it’s on the screen (watch it alone or in a group, make fun of it, sleep through it, masturbate to it) is immaterial. However, the means to get it there is only allowed through approved equipment.

So the private flag is quite clearly DRM, and DRM is all about control (because its only function is to restrict the ability to do something you could otherwise do), and people are not that fond of DRM, so they often seek to bypass it. The Keurig DRM has been bypassed, needing nothing more than a piece of tape, or a clip given away freely by competitors. Often there are really good reasons for bypassing DRM, which is why every 3 years the US Copyright office has a hearing on granting exemptions to the law prohibiting the circumvention of DRM. Sometimes, there’s just a good reason to bypass DRM.

For a bittorrent client, one of the best reasons would be in order to keep the swarm going. A bittorrent swarm is just a name for all the peers on a particular torrent, which is labeled with the SHA-1 hash. The SHA-1 hash is actually how the torrents are identified in clients (not by name, or anything else) and as I’ve explained earlier, they’re a checksum method used to identify and distinguish. One of the key points of the Private Flag was that it was embeded inside the part of the torrent that was used to calculate the hash, so enabling it changes the hash. So, even if you disable the flag through your client, you still need someone else to disable it to connect to them. Otherwise you have the standard problem you get from just adding random trackers.

That’s the basics of the Private flag. There is a downside though. Thanks to the flag, the only way to get peers is via the tracker. So, if the tracker goes offline, there’s no way for the torrent to continue, let alone survive. This was one of the major reasons behind DHT.

Bitcomet 0.60

The developers behind BitComet saw the potential in DHT to enable torrents to survive despite the loss of a tracker, so they came up with an implementation that tried to make it work. Under ‘standard’ private tracker rules, the only way to get peer information is via the tracker, but if that tracker is down, for a sustained period, the torrent dies. So, after a half-hour of getting errors when attempting to contact the tracker, the client would temporarily enable DHT for that torrent, enabling the peers to continue. When the tracker came back up, DHT would be re-enabled again.

The horror and outrage was something to behold. The idea of potentially losing control, that people would be free to distribute and use torrents that ‘belonged’ to those sites (the irony of this is lost on many of their supporters), was absolutely unacceptable, and so a lot of pressure was brought on BitComet. Much of this venom was completely misplaced, though, as what was being claimed was not what was actually happening, and the outrage was based on false information (deliberately or through ignorance was never determined).

The claims were that it could allow anyone, anywhere to get on the torrents and share, seriously undermining the “security” of the sites. The reality is that the only time DHT would be enabled was if the tracker errored for 20-30 minutes. If it returned a message at all, including a ‘not authorised / not a member” message, it wouldn’t start the timer (it would usually stop the torrent in fact), and when the tracker finally responded, it turns DHT off again.

While the person trying to get on could in fact change the tracker address to one that would then give them the error, that still wouldn’t do them any good, because they’d have no-one to communicate with. Until a member did the same thing to become visible, then no outsider will get any peers. The problem then, is the membership not the client – especially as both could quite easily add an open tracker and cut out the wait.

So not only were the allegations spurious, they could only work if an insider AND the outsider conspired together to do a less effective version of a practice that’s still possible today (and much easier, just add an open tracker). Nevertheless, Bitcomet relented, removed 0.60 from distribution, putting 0.59 back out until v0.61 was released without the fallback system. It still continues to be the target of many accusations thought, quite a few of which are unfounded, partly due to the language barrier it faces.

The aftermath of this victory is one of cockiness. After having cowed a client, and mislead a lot of others into thinking they’re looking out for their users rather than themselves, it breeds an atmosphere of contempt for DHT.

More amusingly, they’ve taken lessons from the MPAA and the like, and adopted their strategy – of pretending that not only is this DRM not an issue, it’s actually good for the user, and is about safety and security. It’s not, it’s about control, and getting the best logging on users possible – Without full control of the peers in the swarms, the data collection they do on users won’t mean much. Many site ‘private’ sites deliberately to make users LESS secure, reccomending (or even mandating) old clients, some with well known security vulnerabilities, while they claim to ‘test’ new clients (usually to make sure it won’t break their data collections systems). Perhaps most amusing of all for a long-time observer, is that one of the clients most tend to reccomend, uTorrent 2.2.1, is one that 5 years ago was banned by most, for ‘being unfair‘.

They can do this, because the only major challenge, failed and folded. The Private Flag showdown over Bitcomet was the point at which many Pirates felt DRM was acceptable, and lost the chance to challenge that.