Forgot your password?
typodupeerror
The Internet Privacy IT

New Tool Promises To Passively ldentify BitTorrent Files 265

Posted by timothy
from the checks-for-the-evil-bit dept.
QuietR10t writes "A new technique has been developed for detecting and tracking illegal content transferred using the BitTorrent file-trading protocol. According to its creators, the approach can monitor networks without interrupting the flow of data and provides investigators with hard evidence of illicit file transfers. 'Our system differs in that it is completely passive, meaning that it does not change any information entering or leaving a network,' says Schrader." I wonder if it can specifically identify legal content, too.
This discussion has been archived. No new comments can be posted.

New Tool Promises To Passively ldentify BitTorrent Files

Comments Filter:
  • Encryption? (Score:5, Insightful)

    by hansamurai (907719) <hansamurai@gmail.com> on Thursday February 12, 2009 @03:54PM (#26832245) Homepage Journal

    I'm assuming this has no chance of defeating encrypted connections?

    • Re:Encryption? (Score:5, Informative)

      by fuzzyfuzzyfungus (1223518) on Thursday February 12, 2009 @04:02PM (#26832379) Journal
      TFA confirms it, near the end of the second page. It also only currently works at 100 megabits/second.
    • Re:Encryption? (Score:5, Informative)

      by eldavojohn (898314) * <eldavojohn.gmail@com> on Thursday February 12, 2009 @04:03PM (#26832385) Journal

      I'm assuming this has no chance of defeating encrypted connections?

      The article explicitly says it cannot recognize encrypted files as the method cannot identify them with a hash. Although, I doubt anyone could think of a good way to ID files in encrypted BitTorrent.

      I thought my summary submitted this morning [slashdot.org] did a better job describing this but you should note that this has some key things to overcome before it can be used:

      • Has not been tested for false positives (explicitly stated by a researcher in the article). This has been known to totally render a technology unusable (face recognition, anyone?).
      • Their device only works on up to one hundred megabit per second before it starts to act as a choke point which makes it usefull only on a small scale (not for police/ISPs).
      • Does not work on encrypted files [slashdot.org].

      They seriously need to overcome these obstacles before illegal file sharers should worry about it being used to target people.

      • Re:Encryption? (Score:5, Insightful)

        by Dreadneck (982170) on Thursday February 12, 2009 @04:31PM (#26832895)

        They seriously need to overcome these obstacles before illegal file sharers should worry about it being used to target people.

        I strongly disagree. People need to start raising hell about this Big Brother bullshit now. Technology like this operates under the assumption that ALL users are criminals until proven innocent and blatantly violates the 4th amendment(in the U.S. at least).

        Furthermore, does anyone here honestly believe that this type of technology will only be used to stop copyright infringement and kiddie porn? This technology smacks of oppression and the quashing of political dissent.

        • Re:Encryption? (Score:4, Insightful)

          by Gerzel (240421) * <brollyferret@@@gmail...com> on Thursday February 12, 2009 @05:29PM (#26833943) Journal

          The reason we go after copyright infringement, kiddie porn(well porn in general as it is always lumped in if at all possible to kiddie porn), and things like majauana is to make as much of the general populace guilty of something that is both against the law and seen as deeply wrong with the person.

          Once this is achieved the person can easily be moved to a status of lesser or non-personhood.

          Example is a "Sex Offender" law. Such laws are created inevitably to protect children. However, sex offender includes any offense that is deemed sexual in nature. Public nudity, an argument with a spouse that turns violent which may indeed be an isolated incident and as much at fault with the spouse(I'm not talking about someone who regularly beats their spouse), or just pissing on the sidewalk because there is no where else to go for miles. Everyone is lumped in and assumed to behave like the worst offenders in the group, the serial rapists and violent pedophiles.

        • From what I understand, the technology relies on fingerprinting and lists. It tries to match a bittorrent's 32-bit header data matches to that of a known illegal download on a list. But who gets to set this list? What about false positives? "Yes, this 'Ubuntu' is on the list. This 'WoW-Update-3.0' is also on the list. Well, we're not sure what this 'Ubuntu' is and 'WoW' is obviously a bittorrent of Windows."
      • by Gerzel (240421) *

        1 The False positives problem will be ignored. Already most people and lawmakers consider bit-torrent as a whole to be file sharing and thus piracy. Now they have a way to "ID" the criminals or at least their files. The false positive might work in court but your net connection would be gone long before that case comes due.

        2. This might have a chance to work, provided legislation isn't passed to counteract net neutrality. If such is passed this would easily meet any definition of "reasonable" as would a

      • by noidentity (188756) on Thursday February 12, 2009 @05:31PM (#26833971)

        this has some key things to overcome before it can be used:

        * Has not been tested for false positives (explicitly stated by a researcher in the article).

        Here's my implementation. It also hasn't been tested for false-positives, but I'm hopeful:

        bool is_illicit_content( /* may need parameters in the future */ ) { return true; }

    • Re:Encryption? (Score:4, Interesting)

      by jandrese (485) <kensama@vt.edu> on Thursday February 12, 2009 @04:04PM (#26832423) Homepage Journal
      TFA specifially says that it doesn't work on encrypted traffic. In fact the whole thing seems to have some rather bogus qualities to it.

      It uses a FPGA, but is stuck at a rather pokey 100Mbps. All it does is compare the encoded hash value in the Bittorrent header against a list of known illegal hashes. Hashes you have to program manually.

      I've seen commercial boxes that you can already buy that do a lot more than this and faster. He made a big deal about it not disturbing the network, but that's a standard feature. Unless this thing is dirt cheap or something, I don't really see the application.
      • Re: (Score:3, Insightful)

        by El Torico (732160)

        I've seen commercial boxes that you can already buy that do a lot more than this and faster. He made a big deal about it not disturbing the network, but that's a standard feature. Unless this thing is dirt cheap or something, I don't really see the application.

        I think that the manufacturer will try to pimp this as an "IP Compliance Product" to ISPs and madly lobby every politician they can bribe, err, I mean donate to.

      • Re: (Score:3, Funny)

        by Joce640k (829181)

        It knows every "illegal" hash on the Intertubes?

        If it does that's more newsworthy than the gadget itself.

      • Re: (Score:3, Informative)

        by cerberusss (660701)

        All it does is compare the encoded hash value in the Bittorrent header against a list of known illegal hashes. Hashes you have to program manually.

        That sounds exactly how Snort [snort.org] works.

        I guess if you had a bunch of hashes, you could put these in a configuration and basically have the described functionality.

        I've analyzed Snort more than 6 years ago and also remembered that it couldn't operate on more than 100Mbit. Might've been a change here and there, though.

    • Re: (Score:3, Funny)

      by cbiltcliffe (186293)

      Depends how it works.

      I'm betting something like this:

      $data = read_data_stream($eth)
      if (get_protocol($data) == "bittorrent")
            {
            $illegal_content = 1;
      } else
            {
            $illegal_content = 0;
      }

      In which case, encrypted or not, you're still guilty.

    • Re: (Score:3, Interesting)

      by mochan_s (536939)

      I was wondering, would this defeat this scheme?

      Let bittorrent deliberately make errors in the data transmitted. Hashing is very sensitive to small changes.

      Also, transmit it with error correcting codes so that it can be put back together by the receiver but the hasher gets garbage.

      Finally, so that the hasher doesn't do the error correcting themselves, send the parity encrypted with the keys exchanged beforehand.

      I suppose it's still open to man in the middle attack though.

  • Evil Bit (Score:5, Funny)

    by Lord Byron II (671689) on Thursday February 12, 2009 @03:55PM (#26832251)
    For the record, I have a rule in my iptables that specifically turns off the "evil bit" in any of outgoing packets. Thank God for Linux! =)
  • Till they come up with a good way to figure out whats going across the network encrypted, they will just be wasting their time.

    • by azgard (461476)

      In theory, they could attack encryption with man-in-the-middle during the key exchange. If the protocol is known, the middle man can simulate the other end node for both nodes, and give each one a different key, so they can still see the traffic.

      • by Kjella (173770) on Thursday February 12, 2009 @04:11PM (#26832547) Homepage

        And if they did that, we could start having the tracker negotiate SSL keys for us. If they tried going after the tracker traffic, we could make that HTTPS. If they started faking the certs, we could move to OpenDNS or install a "trusted" torrent root cert. That is a battle they could not win.

        • by azgard (461476)

          Well, eventually, people would have to exchange the trusted torrent root certificates directly (i.e. not over the network). And they could be filtered by the network.

          I think the scheme is in principle possible, but probably very much impractical. You could perhaps create an order of magnitude more music, movies and videogames for the sheer cost of the setup required to negotiate all the encryption keys in the central government server.

        • Re: (Score:3, Insightful)

          by headbulb (534102)

          He was talking about using a man in the middle attack. Both parties think they are talking to eachother.

          It doesn't matter if the tracker sends us a SSL key for us if a man in the middle attack can be used. The only way to be sure the key isn't altered is to get that key directly from the source. How you do that is up to you.

          There isn't much that is open about "OpenDNS". OpenDNS is a bad solution for a non-issue problem. Please stop advertising for them.

          What we should be fighting for is for isp's to be commo

          • by Kjella (173770)

            It doesn't matter if the tracker sends us a SSL key for us if a man in the middle attack can be used. The only way to be sure the key isn't altered is to get that key directly from the source. How you do that is up to you.

            Wrong, wrong, wrong. Or well, if you don't trust the tracker then true but then the whole setup doesn't make any sense. If we both have a secure conneciton to the tracker then the tracker can swap keys for us and there's nothing a man-in-the-middle could do to prevent us from creating a secure peer connection. And if they tried attacking our connection to the tracker, we could use HTTPS and certificates to prevent that. It's you that don't understand.

            • Re: (Score:3, Insightful)

              by headbulb (534102)

              It's funny you just proved my point.

              The internet is in an insecure network. How does anyone know if they have a secure connection? Sure they can know this once a private/public key pair has been exchanged. But how do we know that the public key given to us is good if there is man in the middle to intercept the keys between the "trusted groups"

              I should have been more descriptive. Without physically exchanging the keys with the other parties there isn't a way for an automated system to know; Without testing,

        • by Sloppy (14984) on Thursday February 12, 2009 @04:40PM (#26833067) Homepage Journal

          That's a lot of "we could"s. How about just using the global OpenPGP WoT, and stopping the problem in its tracks?

          Once you have a distributed authentication system (which is what lets you exchange keys safely), email is just one of the applications you can build on it. Sounds like you guys have another. Whatever. The more things it's used for (the more people who connect to the WoT) the better it works for everyone.

          Quit building a redundant but also specialized infrastructure, and instead, join the original.

      • by Shakrai (717556) on Thursday February 12, 2009 @04:19PM (#26832695) Journal

        In theory, they could attack encryption with man-in-the-middle during the key exchange

        In theory, isn't this (or shouldn't this) all be illegal under wiretapping laws anyway?

        As a private citizen I don't have the right to start monitoring my neighbors phone calls (even if those calls are broadcast [wikipedia.org] into my house without encryption) just because I suspect she is dealing drugs. What gives my ISP the right to start monitoring my packets just because they suspect I'm pirating something?

        • Re: (Score:3, Insightful)

          by greenbird (859670) *

          What gives my ISP the right to start monitoring my packets just because they suspect I'm pirating something?

          It's for the children. We must protect the children. Are you one of those evil child porn supporters? If your against this you're a child pornographer.

          All you have to do is add this and all politicians will support it and no publication will speak out against it. Haven't you read Mein Kampf?

  • by alta (1263) on Thursday February 12, 2009 @03:56PM (#26832291) Homepage Journal

    And my $200 24 port gigabit switch from Dell will do it. And that's a cheap piece of crap. For the 3 of you who don't already know, You specify one port on the switch to receive a copy of all traffic on the entire switch, a vlan or a specific port. Then you can hook etherial to that port and monitor all of the traffic without modifying the original. OOOOhhhh, magic eh?

    Anyway, even after I RTFA, I still didn't see anything that this thing does that my cheap port and a P2 running etherial couldn't do.

    • Re: (Score:2, Insightful)

      by tijsvd (548670)

      Two points.

      One: the mirror port (aka span port) on your switch does not buffer the traffic, and will drop packets in any spike. That's true even for expensive Cisco switches. To get all traffic, you need a network tap on a line.

      Two: getting the traffic isn't hard. It's basic sniffing. Analysing the traffic in realtime is what matters.

  • hmm (Score:5, Interesting)

    by Anonymous Coward on Thursday February 12, 2009 @03:57PM (#26832295)

    More restrictions on content? More encryption.

    Better cracking techniques? Better encryption.

    Tyrannical government? Revolution.

  • by RingDev (879105) on Thursday February 12, 2009 @03:57PM (#26832299) Homepage Journal

    Another drawback is that the system cannot cope with encrypted files. "Today, about 25 percent of BitTorrent traffic is encrypted," says Schulze. If such a tool became widely used, then anyone with something to hide would almost certainly switch to using encryption, he says.

    If you make breathing illegal, only criminals with breath.

    -Rick

  • Yawn (Score:4, Interesting)

    by happyemoticon (543015) on Thursday February 12, 2009 @03:58PM (#26832311) Homepage

    From the article:

    Then the system looks at the files' hash, a unique identifying code used to coordinate the simultaneous download of hundreds of file fragments by different users. If a hash matches any stored in a database of prohibited hashes, then the system will make a record of the transfer and store the network addresses involved.

    I mean, you could easily scrape some torrent sites for hashes, but it seems like this system would be fairly easy to circumvent. All you'd have to do is come of with some system for changing the hash on a peer-specific basis.

    • Re:Yawn (Score:5, Informative)

      by blueg3 (192743) on Thursday February 12, 2009 @04:01PM (#26832357)

      If I read the article correctly, what they're really doing is looking at the BitTorrent infohash, which is used when communicating with the tracker and other peers to identify the torrent. (The infohash uniquely identifies the torrent.) Having a different infohash for each peer would require significant BitTorrent reengineering, I would think.

      However, it's defeated by encryption, cannot legally be used in the U.S. or Europe by ISPs, and relies on a blacklist of illicit torrents.

    • by jandrese (485)
      Changing the hash on peer basis would mess the protocol up pretty badly. It's a lot easier just to turn on the encryption stuff.
    • All you'd have to do is come of with some system for changing the hash on a peer-specific basis.

      The hash is how data is verified. You can't just change the hashing mechanism on a peer-specific basis because you're sharing the same data with thousands of different peers. That would require every single peer to host a specific hash for each other peer, or worse, convert between hashes on the fly.

      The flaw in this method is the hashes themselves; the only way to detect the so-called illicit content is by knowing the specific encoding. This stops camcorder films and screener rips because they are enc

  • by fuzzyfuzzyfungus (1223518) on Thursday February 12, 2009 @04:01PM (#26832353) Journal
    So, you're telling me that, given a set of hashes corresponding to "Prohibited content" and access to all the packets moving across a network, you can detect prohibited content? Why, it's a miracle of science!

    Seriously, this is news? It has been possible, with the complicity of the router or physical access to the wire, to unobtrusively and undetectably tap a network since forever. That isn't news. And being able to identifiy files whose hashes you have ahead of time? Also not news, especially since bittorrent uses hashes extensively itself, and was never designed for subtlety or concealment.

    I realize that Technology Review lost interest in technology years ago, and now spends most of its time fellating venture capitalists; but this is pathetic.
  • So... (Score:5, Funny)

    by Adrian Lopez (2615) on Thursday February 12, 2009 @04:10PM (#26832541) Homepage

    So... they invented packet sniffing?

    • by Chelloveck (14643)
      Yeah, but this is completely passive packet sniffing! It doesn't get the packets all sticky like the other tools do.
  • by EdIII (1114411) * on Thursday February 12, 2009 @04:45PM (#26833147)

    It also means that it's impossible for users to tell if a network is being monitored

    "Our system differs in that it is completely passive, meaning that it does not change any information entering or leaving a network,"

    This is nothing new and it's just meaningless marketing drivel. It's impossible to tell that *any* network is being monitored. It's not like you could buy an electronic device in a spy shop that can detect network monitoring. Throttling and "traffic management" are different since that is changing the network traffic.

    There is only one type of network that can prevent a 3rd party from being able to copy the network traffic. Quantum communications provides that type of infrastructure by making it *impossible* to read the traffic without destroying it.

    It's not like network monitoring is really a problem anyways. If you want privacy then just use encryption.

    "Our system does not modify traffic in any way, nor does it interfere in the delivery of traffic either in or out of a network,"

    Ohhh, you mean it's useless right? Everyone involved knows that a large amount of torrent traffic is infringing on various copyrights. The goal of the ISPs is to protect their profit margins. They sell unlimited but expect limited. They don't care whether traffic is illicit or not, just that it does not interfere with their business models. The MAFIAA is interested in the contents of the traffic and could care less about network congestion and bandwidth issues. Until the ISPs actually start caring about content, the goals of these two groups are not the same.

    Enter Net Neutrality. Only when it is in the financial interests of ISPs to care about content will they start to listen to the MAFIAA. Obviously they could not reach an agreement since the MAFIAA is going to the whores in various legislatures to trade our freedoms for the protection of a few group's business models.

    Note, that I don't support piracy on principle. However, I will not give up my rights to privacy and anonymity to protect someone else's copyrights either.

    Schulze adds that the approach relies on having an up-to-date list of illegal files. "The system has to update a huge list of file hashes frequently," he says. "Somebody has to qualify the hashes as copyright infringements or other criminal content."

    That sounds really easy doesn't? Of course there are only a few dozen really popular public trackers out there they can scrape the thousands and thousands of new torrents each day to update their tables. Don't forget about all the private trackers either that add a file or two that changes the hash to be different from the public torrents containing some of the same files.

    Yep. This should be really easy. I can't possibly see how this task could not be reasonably accomplished with just a few salaried personnel on daily basis.

    From a legal standpoint, Schulze says that privacy may be a more significant problem. "Neither the U.S. nor any European country would allow [anyone] to install a device that inspects the traffic of every user just to stop Internet piracy," he says. "In this approach, every user is considered to be suspicious."

    I laughed so hard I almost peed myself at this point. Legal viewpoints change more frequently than the weather. If there is enough pressure from private interests in the U.S and abroad I don't think a little thing like privacy will stop them.

    Even if the legal framework were to allow the technology, it is not quite ready to go. Tests of the system, details of which will be published later this year in a book called Advances in Digital Forensics V, showed that it was effective at detecting 99 percent of illicit files, but only at speeds of 100 megabits per second.

    I just knew there was a p

  • Unclear wording (Score:5, Informative)

    by Rix (54095) on Thursday February 12, 2009 @04:54PM (#26833359)
    This doesn't identify someone downloading a file via bittorrent, it identifies someone downloading a *.bittorrent file (presumably via http).

    This is a non-issue. If anyone actually starts using this, trackers will just start using shttp for their torrent files. They're small and (relatively) low traffic, so it would be a negligible performance issue.

    The only notable thing about this article is that it points out how clueless tech journalists really are.
  • by Anonymous Coward

    There's a well-known technique for dealing with dictionaries of hashes - add some meaningless bits to the content before computing the hash, so that the number of possible hashes increases. This is cheap for everyone except a person trying to keep a dictionary of all possible hashes.

  • This is useless (Score:4, Informative)

    by s_p_oneil (795792) on Thursday February 12, 2009 @05:23PM (#26833851) Homepage

    "Another drawback is that the system cannot cope with encrypted files."

    Even the article mentions that anyone doing something they want to hide is more likely to check the "encrypted only" checkbox. I work on NetSpective WebFilter, which has been passively identifying encrypted protocols that try to hide themselves like encrypted BitTorrent (both standard and Azureus), Skype, and UltraSurf for years. It also lets you choose to block any of these protocols you don't want on your network.

    "If a hash matches any stored in a database of prohibited hashes, then the system will make a record of the transfer and store the network addresses involved."

    Maintaining a list of hashes is not a new idea, as they seem to claim. It was abandoned because the list is insanely painful to manage, and it is insanely easy to get around. These guys aren't even trying to provide a list, which might be worth something (until the hackers put in the time to work around it). They're just sniffing/logging the hashes, which is child's play and worth almost nothing.

  • by Dunbal (464142)

    I like the way the summary tries to equate torrent with illicit. Interesting, on a site full of linux people who have probably torrented more than one distro in their lives.

          Anyway - good luck with that.

  • I am thoroughly amused by articles like this that essential start out as:

    "Hey, look we got! Yackkity, yakkity, yak, yak..." ...And end with something along the lines of...

    "...Well, its pretty damn useless considering xxxxx and xxxx are already in use and defeat it completely."

    Why do people even bother printing such useless information, much less invest millions of dollars into such a product?

  • I wonder if it can specifically identify legal content, too.

    So why would the likes of the RIAA and MPAA want to do that?

    They're interested in finding criminals, not showing that people are innocent.

  • isn't it a packet sniffer? Isn't that illegal tech for these purposes?

"Whoever undertakes to set himself up as a judge of Truth and Knowledge is shipwrecked by the laughter of the gods." -- Albert Einstein

Working...