Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
AI Networking The Internet

MIT Uses Machine Learning Algorithm To Make TCP Twice As Fast 250

Posted by timothy
from the do-it-again-for-four-times-faster dept.
An anonymous reader writes "MIT is claiming they can make the Internet faster if we let computers redesign TCP/IP instead of coding it by hand. They used machine learning to design a version of TCP that's twice the speed and causes half the delay, even with modern bufferbloated networks. They also claim it's more 'fair.' The researchers have put up a lengthy FAQ and source code where they admit they don't know why the system works, only that it goes faster than normal TCP."
This discussion has been archived. No new comments can be posted.

MIT Uses Machine Learning Algorithm To Make TCP Twice As Fast

Comments Filter:
  • by Fishchip (1203964) on Friday July 19, 2013 @11:31PM (#44335065)
    This is how things like Skynet get started.
    • by Sqr(twg) (2126054) on Friday July 19, 2013 @11:42PM (#44335093)

      That's easy to prevent. [xkcd.com]

    • by jamesh (87723) on Saturday July 20, 2013 @02:21AM (#44335553)
      At the very least i'd be doing a grep for things like "kill all humans" in the source code.
    • Skynet has been around for a while, and it is running AT&T. Executive Bonus Recovery Fee tagged on to wireless bills under contract? Humanity is doomed.
    • by kheldan (1460303)

      ..where they admit they don't know why the system works..

      I know you're all being funny in this thread of comments, but I'm not amused at that remark at all, it's a little scary to be honest. If we do start living in a work where AI's are creating things that we don't understand, but use anyway, then everyone should be concerned.

      • by Zalbik (308903)

        . If we do start living in a work where AI's are creating things that we don't understand, but use anyway, then everyone should be concerned.

        That certainly would be scary. But that's not the case here. Researchers certainly understand the algorithm; they just don't understand why it improves TCP performance so much. There is a huge difference.

        A lot of math is like that (we understand something to be true, but can't prove why). However, I'm not concerned that those algorithms are going to jump of the

  • Uh Oh... (Score:5, Funny)

    by Anonymous Coward on Friday July 19, 2013 @11:39PM (#44335087)

    they admit they don't know why the system works, only that it goes faster than normal TCP

    And so it begins...

    • OSPF (Score:3, Interesting)

      by globaljustin (574257)

      It's basically a more complex version of Open Shortest Path First.

      Depending on how you understand the term 'autonomous system' [wikipedia.org] you can have a lot of fun with the idea. It doesn't *explain* everything about how this works, but it puts it into context, in my mind.

      FTA: To approximate the solution tractably, Remy cuts back on the state that the algorithm has to keep track of. Instead of the full history of all acknowledgments received and outgoing packets sent, a Remy-designed congestion-control algorithm (Remy

    • by Richy_T (111409)

      If they don't know how it doesn't work then they don't know when it won't work because of some unaccounted for situation. For an example, consider that pedestrian bridge in London. Engineering is about more than designing for the common case. You also have to consider outliers.

  • by Jane Q. Public (1010737) on Friday July 19, 2013 @11:45PM (#44335105)
    Allow a computer to design a faster TCP? Sure!

    Let them actually implement it without knowing how it works? Oh, Hell no!

    I'm not talking "Skynet" or anything here... but if it breaks, who's going to fix it?
    • Re: (Score:3, Insightful)

      by Anonymous Coward

      FYI: There's a difference in "knowing the precise mechanism for how it works" and "knowing that the algorithm is stable" are two very different things. Presumably they've proven the latter.

    • by RedBear (207369)

      Allow a computer to design a faster TCP? Sure!

      Let them actually implement it without knowing how it works? Oh, Hell no!

      I'm not talking "Skynet" or anything here... but if it breaks, who's going to fix it?

      If it breaks can't we just fall back to the current inefficient algorithms? With the performance and fluidity improvements promised by this approach it could be hugely beneficial to all kinds of networks, even if no one yet fully understands why it works better. They'll figure it out eventually.

      • There is no "can't we just" involved when the network loses its shit because you wanted to try something new. There is, however, plenty of time to figure out why it did afterwards as you stand in the unemployment line.
    • by Intropy (2009018) on Saturday July 20, 2013 @12:12AM (#44335201)
      We're already in that boat. One of the reasons it's so hard to make changes is that nobody really knows why the internet works. We know how and why individual networks work. We can understand and model RIP and OSPF just fine. And we know how BGP operates too. But that large scale structure is a mess. It's unstable. The techniques we use could easily create disconnected or even weakly connected networks. But they don't except for occasionally a single autonomous system falling off. We've built ourselves a nice big gordian knot already. We know what it's made of, and we know how it operates, but good luck actually describing the thing.
      • Come on now (Score:5, Insightful)

        by dbIII (701233) on Saturday July 20, 2013 @01:10AM (#44335357)
        As complex systems goes there are far worse. Go ask an engineer or a scientist.
        • Re:Come on now (Score:5, Interesting)

          by Daniel Dvorkin (106857) on Saturday July 20, 2013 @02:17AM (#44335537) Homepage Journal

          As complex systems goes there are far worse. Go ask an engineer or a scientist.

          I am a scientist--specifically, a bioinformaticist, which means I try to build mathematical and computational models of processes in living organisms, which are kind of the canonical example of complex systems. And I will cheerfully admit that the internet, taken as a whole, is at least as complex as anything I deal with.

          • Several hundred million computers doing pretty much the same thing is not nearly as complicated as a million species each doing their own thing. The internet is more like a single giant organism [abc.net.au].
          • by rthille (8526)

            Yeah, internet is way way more complex than any living organism you deal with...

            because it's composed of all those machines and a crap load of living organisms as well.

      • by Animats (122034) on Saturday July 20, 2013 @01:21AM (#44335393) Homepage

        One of the reasons it's so hard to make changes is that nobody really knows why the internet works.

        We still don't know how to deal with congestion in the middle of a pure datagram network. The Internet works because last-mile congestion is worse than backbone congestion. If you have a backbone switch today with more traffic coming in than the output links can handle, the switch is unlikely to do anything intelligent about which packets to drop. Fortunately, fiber optic links are cheap enough that the backbone can be over-provisioned.

        The problem with this is video over the Internet. Netflix is a third of peak Internet traffic. Netflix plus Youtube is about half of Internet traffic during prime time. This is creating demand for more and more bandwidth to home links. Fortunately the backbone companies are keeping up. Where there's been backbone trouble, it's been more political than technical. It also helps that there are so few big sources. Those sources are handled as special cases. Most of the bandwidth used today is one-to-many. That can be handled. If everybody was making HDTV video calls, we'd have a real problem.

        (I was involved with Internet congestion control from 1983-1986, and the big worry was congestion in the middle of the network. The ARPANET backbone links were 56Kb/s. Leased lines typically maxed out at 9600 baud. Backbone congestion was a big deal back then. This is partly why TCP was designed to avoid it at all costs.)

      • by hedwards (940851)

        No, people know how the internet works. The reason why it's so hard to make changes is that it's unpredictable at what rate the changes will propagate and in which order. And in some parts of the world there are only 2 or 3 lines into that part that would need to be upgraded. So, if they screw it up, they're completely without internet and the ability to google "WTF did I do wrong upgrading the internet?". Whereas in other parts of the world there are much larger number of lines that are probably not all up

    • Re: (Score:3, Interesting)

      by afidel (530433)

      Meh, it's like the AI designed antenna, we don't have to know WHY it works better, just that it does and how to build a working copy.

      • by techhead79 (1517299) on Saturday July 20, 2013 @12:38AM (#44335279)

        we don't have to know WHY it works better, just that it does and how to build a working copy

        But the fact that it does work better means we're either missing a part of the picture that is obviously important or the AI version is leveraging quirks with the system that no current model we have represents. I'm shocked to read that anyone would be comfortable just ignoring the why of something just so we can progress beyond our understanding. If we don't understand the why then we're missing something very important that could lead to breakthroughs in many other areas. Do not let go of the curiosity that got us here to begin with.

        • by blankinthefill (665181) <blachanc.gmail@com> on Saturday July 20, 2013 @01:12AM (#44335367) Journal
          I don't think they just drop the questions and run with it. I'm pretty sure that, when we don't understand how things that are useful work, we just implement them... and study them at the same time. I guarantee you that SOMEONE, at least, is studying why an AI antenna works better than our man-designed ones, and they're doing it for the very reasons that you mention. But I think the point the GP was trying to get at is that we've never let out ability to not understand things hinder our adoption of those very things in the past, and as long as we have good evidence that this thing performs correctly, and we can replicate it, then why wouldn't we use it at the same time we study it?
        • by Ichijo (607641) on Saturday July 20, 2013 @01:18AM (#44335387) Homepage Journal
          So we built a computer that figured out the answer [wikia.com]. Now we just need to build an even bigger computer to figure out the question!
          • by citizenr (871508)

            So we built a computer that figured out the answer [wikia.com]. Now we just need to build an even bigger computer to figure out the question!

            HOLY SHIT you just blew my mind. I've been doing Machine learning for a while now, but I just took it for granted and it never occurred to me that learning the model is in fact learning the answer to a question that is often too complex for humans to grasp.

            All hail Douglas Adams, our prophet and saviour!

        • Re: (Score:2, Informative)

          by Anonymous Coward

          > I'm shocked to read that anyone would be comfortable just ignoring the why of something just so we can progress beyond our understanding.

          ML often works like that.

          You put the inputs into a function... it spits out a model. The model can be considered as an optimal orthonormal basis for the space it was encoding, but its REALLY REALLY hard to understand the dimensions that basis is in. Sometimes, you can take an image categorization model and see "ah, this is the blue shirt dimension. It seems that pe

        • by jkflying (2190798) on Saturday July 20, 2013 @02:01AM (#44335501)

          It's not that we don't understand *why* something like a genetic-algorithm designed antenna works so well. We can evaluate its performance using Maxwell's equations and say, "Yes, it works well." without ever having to build the thing. What we don't have is a set of guidelines or 'rules of thumb' that can result in an antenna design that works just as well.

          The difference is that the computer evaluates a billion antennas for us, doing some sort of high-dimensional genetic optimisation on the various lengths of the antenna components. It doesn't 'understand' why it gets the results it does. We do, because we understand Maxwell's equations and we understand how genetic optimisation works. But Maxwell's equations only work for evaluating a design, not for giving a tweak which will improve it. And we're dealing with too many variables that are unknown to have a closed-form solution.

          As for this algorithm, they basically did the same thing. They defined a fitness function and then repeatedly varied the code they were using to find the best sequence of code. However, unlike the antenna analogy, they used actual equipment to evaluate the fitness function, not just a model. This means that they don't have an accurate model, which means that your complaint that we don't know why this works is entirely valid, and the antenna analogy is not =)

          • Interesting, that when I read your post I couldn't help but think that very same though about politics, and social engineering. People seem to think that any solution that looks good on the surface should work, and yet hardly ever bother to measure the result of any given solution. On the whole we understand that things are f'd up but understanding why is simply beyond us. Makes me wonder what solutions machine learning algorithms would come up with given more and more criteria, and if those solutions wo

        • by seandiggity (992657) on Saturday July 20, 2013 @02:23AM (#44335561) Homepage
          We should keep investigating why it works but, to be fair, the history of communications is implementing tech before we understand it (e.g. the first trans-Atlantic cable, implemented before we understood wave-particle duality, and therefore couldn't troubleshoot it well when it broke).

          Let's not forget this important quote: "I frame no hypotheses; for whatever is not deduced from the phenomena is to be called a hypothesis; and hypotheses, whether metaphysical or physical, whether of occult qualities or mechanical, have no place in experimental philosophy."

          ...that's Isaac Newton telling us, "I can explain the effects of gravity but I have no clue WTF it is."
        • by Daniel Dvorkin (106857) on Saturday July 20, 2013 @02:27AM (#44335563) Homepage Journal

          I'm shocked to read that anyone would be comfortable just ignoring the why of something just so we can progress beyond our understanding.

          If you insist that we know why something works before we make use of it, you're discarding a large portion of engineering. We're still nowhere near a complete understanding of the laws of physics, and yet we make machines that operate quite nicely according to the laws we do know (or at least, of which we have reasonable approximations). The same goes for the relationship between medicine and basic biology, and probably for lots of other stuff as well.

          If we don't understand the why then we're missing something very important that could lead to breakthroughs in many other areas. Do not let go of the curiosity that got us here to begin with.

          I don't think anyone's talking about letting go of the curiosity. They're not saying, "It works, let's just accept that and move on," but rather, "It works, and we might as well make use of it while we're trying to understand it." Or, from TFA: "Remy's algorithms have more than 150 rules, and will need to be reverse-engineered to figure out how and why they work. We suspect that there is considerable benefit to being able to combine window-based congestion control with pacing where appropriate, but we'll have to trace through these things as they execute to really understand why they work."

        • I'm not shocked at all. This is just an automated form of hand-optimisation. Plenty of products and algorithms end up in regular use that have been tweaked intuitively (or algorithmically) without really understanding why the tweaking improved it. Plenty of engineering research is about providing models for existing systems to understand why best in class designs work the best. If we held back empirically proven designs until the theory was completely understood we'd never progress with anything.
        • It's a chaotic system. The optimal responses to that can be another chaotic system, which happens to hit the right symbiosis enough of the time to offer clear benefits. You do not need to understand how the chaos works, and even if you do, it could appear completely nonsensical.

        • by turp182 (1020263)

          Understanding and realizing the nature of something is usually enough. It works for medicine (and penicillin and other accidental discoveries, which over time we have learned much more about).

          In this case we have a heuristic knowledge of the effect, and it applies to complicated networking/data transfer. We don't know the details, but we know it works. The algorithms, or the AI itself, could be applied to similar, but disparate, systems.

          Shoot, we use our brains, but we don't understand them yet. Yet we

        • by AK Marc (707885) on Saturday July 20, 2013 @07:37AM (#44336281)
          It's a mystery because in practice, thousands of sessions being tracked is too hard to deterministically determine in a simple static manner. So we use WRED instead. This is WDED - weighted deterministic early detection. What we don't understand is how this does so much better than random drops, mainly because math is hard. Someone could probably take this, and write a mathematics thesis on this. Determining how to drop packets to keep a minimum queue size and have the lowest impact on performance is something that has been worked on for years. This isn't unknowable, or ever really that hard. It's just different and complicated (within a small area of interest, so less than 1% of the population knows what WRED is, let alone how this is essentially an improvement based on it (at least what I could tell from FTA, as I haven't had time to read the source, let alone understand it.
        • by Immerman (2627577) on Saturday July 20, 2013 @09:30AM (#44336643)

          The canonical example is we have no idea why we're capable of logical thought, yet that doesn't in any way impair us form using it.

          In fact when it comes to most complex systems (economy, ecology, etc, etc, etc) we don't *really* understand how they work, but we muddle through as best we can. Generally speaking when faced with a complex system we take one of two routes:
          * Oversimplify our model of the system so that we think we understand it and work from there (the "professional" approach, which often ends catastrophically when those oversimplifications come home to roost)
          * Trial and error (the historically far more successful approach, and the only one used by evolution)

          Something like the bent-wire antenna with incredible omnidirectional properties is a great example of this: It's not that there's some magical features we haven't discovered about radio, the thing was designed by genetic algorithm within a computer simulation that was 100% limited by our existing models of antenna behavior. But a 10-wire antenna allows for phenomenally complex interactions between those behaviors, and the trial-and-error approach allowed the algorithms to home in on an extremely effective configuration within a problem space far to complex to reason our way through.

          An even better example would be the nozzles used by some companies to create powdered laundry detergent - they spent a bunch of money on engineers to design a nozzle that would spray a liquid detergent solution in a manner that created tiny quick-drying droplets. Despite the simulations all saying it should work great, it failed miserably. Then they just built a bunch of random nozzles, tried them out, and used genetic algorithms to home in on an effective design. The difference from the antenna process being that they actually made physical versions of the nozzles to test, because the best available simulations were clearly incompatible with reality.

    • by Clarious (1177725) on Saturday July 20, 2013 @12:22AM (#44335239)

      Think of it as solving a multiobjective optimization problem using heuristic algorithm/machine learning. You can't solve the congestion problem completely as it is computionally infeasible, now they just use machine learning to find the (supposedly) optimal solution. Read TFA, it is quite interesting, I wonder if we can apply that to Linux writeback algo to avoid the current latency problem (trying copying 8 Gb of data into a slow storage medium such as SD card or USB flash, prepare for 15+ seconds stalls!), the underlying is the same anyway.

      • by Clarious (1177725) on Saturday July 20, 2013 @12:30AM (#44335253)

        A bit offtopic, roughtly 10 years ago I came to /. and was amazed by the technological insight/information in the comments here. And now more than half of the comments are jokes about skynet without any insight of understanding what TFA is about. Of course, informative posts still can be found often, but slashdot has fallen quite low...

        • The jokers are the ones who didn't read TFA. So sad. Especially when TFA is such a good read and is actually "News For Nerds"!
        • by evilviper (135110) on Saturday July 20, 2013 @09:00AM (#44336523) Journal

          A bit offtopic, roughtly 10 years ago I came to /. and was amazed by the technological insight/information in the comments here.

          Yes, all the comments about pouring hot grits on a naked and petrified Natalie Portman had really superb mathematical proofs backing them up...

          FWIW, you're absolutely correct. Before /. tried to become digg, and then reddit, and then a flamefest of AGW/evolution/etc. supporters and deniers, there was a much more vibrant community with a tremendous number of experts in fields from mathematics and physics to biology and psychology, always chiming in on the topic of the day, and providing incredible insight into the field and the specific topic that one wouldn't find anywhere else.

          It seems that model didn't result in enough ad impressions and profits for the parent company, so flamefests it is. /. has only recently backed off of editors posting complete and total crap, so my belated plans to drop this site entirely were aborted, and I remain. These days, there really are only a handful of folks who provide real insightful comments across many articles. It's easy to spot them if you read this site regularly, and it's such a small group I could rattle off a list of names from memory.

          The only reason /. has any relevancy today, and the audience hasn't completely disappeared, is that all other tech sites have HORRIBLE comment/discussion systems that make it hard to follow the discussion, and do not really promote good comments to a wider readership than the first-post crap.

        • Read at 3+, mod at 0+.

    • by ndogg (158021)

      To be fair, I didn't see anything from them saying that everyone should go out and implement this right now. They are going to reverse engineer it to understand what is going on, and once they do, I'm sure they'll only propositions implementations after that based on their findings.

      • by citizenr (871508)

        To be fair, I didn't see anything from them saying that everyone should go out and implement this right now.

        Actually what they said as the opposite - it wont work in mixed networks where other algos (compound/vegas) routinely fill up buffers and create latency bottlenecks.

    • by hairyfeet (841228)

      Why not? You can always go back to the old way if it goes tits up down the line. If the thing gives you X+Y and our current system gives you X and it costs nothing to implement this, as no new hardware or days reprogramming the software to use why not go for it?

      As long as it doesn't require weeks to switch the network back if this doesn't work down the line I see no reason not to give it a shot, not like we couldn't all use a free speed boost, right?

    • by Rockoon (1252108) on Saturday July 20, 2013 @05:14AM (#44335987)
      You should go talk to Intel or AMD about your opinions on the matter, because I assure you that the specific layout of their chips is based on machine learning algorithms. No human can realistically optimize circuits containing a billion transistors.

      As a matter of fact, I recall genetic algorithms being thrown at rather small circuit design problems and producing solutions that were better than any human had come up with. Ah yes, here it is: Sorting networks and the END search algorithm [brandeis.edu].

      -- "Even a 25-year old result for the upper bound on the number of comparators for the 13-input case has been improved by one comparator"
    • They did this experiment with genetic algorithms with FPGAs, and it produced a frequency discriminator with far, far fewer gates than any human design.

      There was a region of circuitry in the middle not connected to anything, so they figured it was redundant and removed it. The circuit stopped working. Stuff like this can discover emergent properties of systems you never even designed in.

  • by tloh (451585) on Friday July 19, 2013 @11:46PM (#44335111)

    they admit they don't know why the system works

    I'm guessing the next big revolution in AI is the quest to figure out how to get digital problem solvers to teach us meat heads how they actually figured this stuff out.

    • by krray (605395)
      When they [the computers] do, I bet they don't... Because, you know -- They're Made of Meat [terrybisson.com]
    • by osu-neko (2604)

      I'm guessing the next big revolution in AI is the quest to figure out how to get digital problem solvers to teach us meat heads how they actually figured this stuff out.

      The thing, we already know exactly how they figured it out -- we wrote the instructions they followed to do so. We know exactly how they figured it out, we just don't understand the solution.

    • by hughbar (579555)
      Skynet jokes aside, it's [my opinion] a general problem with sub-symbolic and also, in a related area, data mining. It works [sometimes], produces a number rather than a model or chain of reasoning and we can't really follow it. I wouldn't be really happy to have something like this in an autopilot, for example. Losing control and depending entirely [no shutdown and back to 'manual'] on systems that we don't completely master [actually with the amount of things that are 'connected' we're heading there anywa
    • by Nemyst (1383049)
      Thing is, they wouldn't know either. This is basically taking a computer, giving it a set of parameters to test, and letting it loose on most/all of the possible combinations. The "AI" doesn't know anything about what it's doing; it's just taking a sample, running it and getting a performance value out of it, which it then uses to classify that sample. Re-run for millions or billions of samples and you end up with the best system out of all the systems the computer tested. Obviously I'm dramatically oversim
  • treason? (Score:2, Funny)

    by Xicor (2738029)
    would it be considered treason for MIT to develop skynet and have it destroy the US?
  • This isn't a redesign of TCP. The network is still just as stupid as it was before; It's just that the local router has had QoS tweaked to be more intelligent. By a considerable margin too. Reviewing the material, it seems to me like it's utilizing genetic algorithms, etc., to predict what's coming down the pipe next and then pre-allocating buffer space; Rather like a predictive cache. Current QoS methods do not do this kind of predictive analysis -- they simply bulk traffic into queues based on header data

    • by Megor1 (621918)
      So much snark!
    • by Lord_Naikon (1837226) on Saturday July 20, 2013 @12:44AM (#44335301)

      Huh? Did you read the same article as I did? As far as I can tell, the article is about a TCP congestion control algorithm, which runs on both endpoints of the connection, and has nothing to do with QoS on intermediate routers. The algorithm generates a set of rules based on three parameters resulting in a set of actions to take like increasing advertised receive window and tx rate control. The result of which is a vastly improved total network throughput (and lower latency) without changing the network itself.

      I fail to see the relevance of predictive/adaptive caching. It isn't even mentioned in the article.

      • Huh? Did you read the same article as I did? As far as I can tell, the article is about a TCP congestion control algorithm, which runs on both endpoints of the connection, and has nothing to do with QoS on intermediate routers. The algorithm generates a set of rules based on three parameters resulting in a set of actions to take like increasing advertised receive window and tx rate control. The result of which is a vastly improved total network throughput (and lower latency) without changing the network itself.

        I fail to see the relevance of predictive/adaptive caching. It isn't even mentioned in the article.

        I think the GP got confused by the "Machine Learning" part of the headline, and thought that the network algorithm uses some sort of adaptive mechanism. What the software actually does is uses genetic learning (i.e. natural selection) to generate sets of network algorithms, each generation of which is better than the one before (that's how Genetic AI works). The actual algorithms in question are a set of static rules, not much different in function than the existing TCP algorithms, just more efficient.

        • by citizenr (871508)

          What the software actually does is uses genetic learning

          No. Genetic learning is learning by iterating over preselected range of parameters in search of a fit.
          Here they are using real data to teach algorithm optimal behavior.

      • That's girlintraining for you.

      • which runs on both endpoints of the connection

        Remy typically runs for several hours and produces a congestion-control algorithm that can be implanted into the sender of a TCP implementation, and then run in real-time. Right now, we do not need to modify the TCP receiver.

    • Admittedly, I've bookmarked this article for later perusal. That said, it strikes me that the following might already foot the bill:

      http://arstechnica.com/information-technology/2012/05/codel-buffer-management-could-solve-the-internets-bufferbloat-jams/ [arstechnica.com]

      Unlike other active queue management (AQM) algorithms, CoDel is not interested in the sizes of the queues. Instead, it measures how long packets are buffered. Specifically, CoDel looks at the minimum time that packets spend in the queue. The maximum time relates to good queues which resolve quickly, but if the minimum is high, this means that all packets are delayed and a bad queue has built up. If the minimum is above a certain threshold—the authors propose 5 milliseconds—for some time, CoDel will drop (discard) packets. This is a signal to TCP to reduce its transmission rate. If the minimum delay experienced by buffered packets is 5ms or less, CoDel does nothing.

      • by citizenr (871508)

        Sounds great in theory, but no one will implement it - dropped packets = some shitty software will complain = user will pick up the phone and complain to isp = isp will complain to cable modem OEM = its in cable modem OEMs best interest to never ever drop packets = we get ridiculous big buffers just so some retard doesnt complain about dropped packets.

        • by Immerman (2627577)

          Actually everyone already implements it - dropped packets are a standard part of TCP/IP, the protocol explicitly deals with how to handle them transparently, along with out-of-order delivery - because packets 1, 2, and 3 may all take different routes, and there's no guarantee 3 won't reach the destination first or that 2 will make it at all.

          They only question is how to decide *which* packets get dropped when congestion overruns the available queues. In fact if I recall correctly there was some research a y

          • by citizenr (871508)

            Large buffers in cable modems is a proof no one will implement it - they added them to avoid dropping packets at all cost, no matter the spec that tolerates it.

        • Active queue management [dropping packets] is already being done by most intermediate routers already. It's been done somewhere along the line in getting this reply to you.

          TCP is a "virtual circuit" [stream] protocol. In every TCP packet, there are upstream/downstream byte offsets (e.g. "I've sent you X bytes", "I've received from you Y bytes"). These are examined by the end-to-end nodes only, not any intermediate node or its associated buffers. At this level, packets [per se], dropped or not, don't mat

  • by mattr (78516)

    Looks interesting. Would be more interesting if possible to model incomplete rollout and rollout at not just endpoints, IPv6 rollout is another rational/superrational mix, and instead we've got ubiquitous NAT instead, right?
    The mention of inability to do an upload during Skype.. is that true?
    Conceivably a whole-network deep simulation could be updated based on real traffic patterns providing suggestions for network upgrades / additional cross connects, so enduser requirements c

  • by CuteSteveJobs (1343851) on Saturday July 20, 2013 @12:36AM (#44335271)
    Kudos, but can't OP say "MIT Researchers Keith Winstein and Hari Balakrishnan". Despite the best efforts of their AI labs, MIT is not the Borg. When someone who works for MIT buys an orange juice, "MIT" has not bought an orange juice.

    And if they have software that can outcode me, COOL! How many professions are this lax with job security? :-)
    • by citizenr (871508)

      They cant because outcome was good. They only use names (Star Simpson, Aaron Swartz) when they want to disassociate it from the MIT brand.

  • by SpectreBlofeld (886224) on Saturday July 20, 2013 @12:56AM (#44335329)

    For some reason, while reading the FAQ writeup, the traveling salesman problem sprung to mind. I don't understand either enough to understand why.

    Can anyone educated enough in both issues explain if they are similar computational problems, and why?

    • by vovick (1397387)

      (Disclamer: I'm no expert in routing algorithms or network problems in general)
      Aside from both these problems having to do with finding a path on a graph, I see little similarity. In the TSP you need to find the shortest path to cover all nodes("cities") on a graph, while here you need to find a path between two nodes ("computers") which would both be fast and not cause traffic bottlenecks. I feel there may be many various metrics to define what an "optimal" path should be, each one better in one regard and

    • I'll tell you the *big* difference between them.

      One is just trying to find good routing.

      The other has to find the *best* routing.

      -- hendrik

  • "they admit they don't know why the system works, only that it goes faster than normal TCP"

    Well, not the best premise for redesigning an infrastructure used by billions. Troubleshooting something you don't quite understand? Right.
    • by Immerman (2627577)

      Hey, we attempt it with the national and global economy all the time, and we don't even have any decent evidence-based models there, much less actual laboratory-controlled experimental data. Just a bunch of wild speculation by economists who want the world to conform to their idea of what should be.

  • If you read the page, you will see that the results are from a simulation and not based on experiments in a real network. And the given performance only works under certain stable conditions. Some remarks seem to imply that if you are moving around (like with a mobile device) the results no longer apply. Still, I believe that machine learning techniques could out perform human coded algorithms, but probably not as much as the 'theoretical' results presented in this research/paper.
    • by Skapare (16644)

      Then we should find some human somewhere, lock him/her to a desk with a computer, and force him/her to write the code to re-implement TCP. Then compare the performance with the AI produced algorithm.

  • by dltaylor (7510) on Saturday July 20, 2013 @03:55AM (#44335791)

    Yet Another Misleading Headline

    The paper states quite clearly that once the simulation has produced an algorithm, it is static in implementation.

    The authors give a set of goals and an instance of a static network configuration and run a simulation that produces a send/don't send algorithm FOR THAT NETWORK, in which all senders agree to use the same algorithm.

    While this looks like very interesting and useful research, it has nothing to do with systems that learn from and adapt to real world networks of networks.

    • by Nemyst (1383049) on Saturday July 20, 2013 @09:39AM (#44336685) Homepage
      Um, what? It most certainly is machine learning. Your "simulation" is an offline machine learning algorithm which, given input parameters, finds the best algorithm in the situation provided. Machine learning isn't strictly online algorithms, and it most certainly isn't "systems that learn from and adapt to real world networks of networks", which I'm having a hard time even parsing.
  • Wonder if can be combined with a previous invention : Boosting the performance of wireless networks by up to 10 times [slashdot.org]

  • I've always wondered why we couldn't use this approach to fix certain types of open-source drivers. We know the APIs, and we have the hardware - set it up and get machines to iterate over it until we get the desired properties for hardware which doesn't need a human their to verify its functioning correctly. I'm thinking for something like network drivers this could work - though you could imagine a system which did it for graphics cards too.

    • by LWATCDR (28044)

      You would want to create a virtual version of the hardware to do this. The problem with systems like this is that they may exploit hardware quirks and only work on one card or chip. Of course you would then want to test on a few pieces of real hardware to make sure your simulation is accurate.

  • We don't use TCP because it's particularly efficient, but because it has been the standard for decades. With our current knowledge, humans could also design faster protocols, we just don't want to waste our time, seeing that not even IPv6 has managed to become adopted.

  • Does Remy assume cooperation with other network users or competition for a scarce resource? If I tune my TCP algorithms but others do not, who wins?

    One positive aspect: This could keep SkyNet busy competing against other systems rather than turning on the human race. Until it realizes that the network could be speeded up by eliminating all the meatbags downloading porn.

"Anyone attempting to generate random numbers by deterministic means is, of course, living in a state of sin." -- John Von Neumann

Working...