200-400 Gbps DDoS Attacks Are Now Normal 92
An anonymous reader writes "Brian Krebs has a followup to this week's 400 Gbps DDoS attack using NTP amplification. Krebs, as a computer security writer, has often been the target of DDoS attacks. He was also hit by a 200Gbps attack this week (apparently, from a 15-year-old in Illinois). That kind of volume would have been record-breaking only a couple of years ago, but now it's just normal. Arbor Networks says we've entered the 'hockey stick' era of DDoS attacks, as a graph of attack volume spikes sharply over the past year. CloudFlare's CEO wrote, 'Monday's DDoS proved these attacks aren't just theoretical. To generate approximately 400Gbps of traffic, the attacker used 4,529 NTP servers running on 1,298 different networks. On average, each of these servers sent 87Mbps of traffic to the intended victim on CloudFlare's network. Remarkably, it is possible that the attacker used only a single server running on a network that allowed source IP address spoofing to initiate the requests. An attacker with a 1 Gbps connection can theoretically generate more than 200Gbps of DDoS traffic.' In a statement to Krebs, he added, 'We have an attack of over 100 Gbps almost every hour of every day.'"
Well (Score:5, Insightful)
The obvious solution is to unplug the Internet. I'm sure the government and the movie people will be thrilled.
Re:Well (Score:5, Funny)
lol (Score:1)
Re: (Score:3)
No. You need to wait at least 30 seconds to make sure the Internet's RAM is cleared and it's ready to reboot.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Notice the sidebar to the Krebs article: "The value of a hacked PC".
I say unplug Windows from the Internet. The world has had enough of this already.
Re: (Score:2)
Yes.. because everyone of those NTP servers that made up the 400Gb/s attack were running Windows...
Idiot.
Then who is generating the forged packets, Einstein? Its not some guy hammering at NTP servers directly from his home PC or hosting account; You still need a very big network presence to generate the calibre of attacks seen lately, even with amplification.
Re: (Score:1)
And where. exactly, does a script kiddie get a "single 1G connection"? Google Fiber? (and do you really think Google wouldn't notice a flooded connection?) Plus, firing off a DoS from a single location makes it down right trivial to kill -- there's only one machine that has to be unplugged.
No they will not be thrilled (Score:3)
How can you push out propaganda if your main distribution method goes away?
Re: (Score:2)
By continuing to use your multiple other methods of distribution?
One system may be fastest or cheapest, but you have to be really, really sure that you're never going to need what you used previously before you unplug it and sell (or throw away) the hardware.
Case in point : I'm working on a 100-million dollar ship equipped with around ten million dollars of the best shiniest and newest of equipment for robotically handling one of t
Re: (Score:2)
By continuing to use your multiple other methods of distribution?
That wont be effective on the next generation ( kids today ). Time has moved on, the method must too.
Re: (Score:2)
Part of the job of us grey-beards is to make sure that when (not if) the things that the kids depend on get broken (including by other kids), then there's a backup system in place. You see, kids today haven't seen things fuck up completely. So they know that it's not going to happen to them because they're immortal and of infinite intelligence.
When (not if) the "kids" encounter their first major fuck up - they have friends killed in a car crash ; their employer goes bust because of something
Re: (Score:1)
CURRENT, low hanging fruit, root cause... take away windows(tm) and they'll target something else.
Root issue is lack of URPF and similar (Score:5, Informative)
Hosting/Colo/Transit providers are the real core issue. There is absolutely no reason that URPF or similar or at least ingress ACL's are not in place. Lets face it if your limiting the prefixes announced you should be filtering on them as well. Anything even close to core can do this in hardware, URPF and similar there is generally no config required more than turning it on. At Hosting/Colo levels do you still have something on the public side that can not do at least ACL's in hardware? Plenty of automation packages can do this stuff in an automated fashion. The root cause is lazy and broken providers that just do not care, DDOS traffic can make some of them piles of cash directly in transit billing or indirectly as the only people with a big enough pipe to do ddos protection.
Re:Root issue is lack of URPF and similar (Score:5, Funny)
Indeed, reverse path filtering should be mandatory, especially because it is so easy.
Also, RFC3514 should become a part of the IP standard. Not setting the appropriate bit from the sender side should then be punishable with eternal flogging.
Re: (Score:2)
Exactly ... is there some perverse incentive at work which makes backbones not try to implement ingress/egress filtering at the internet edge?
AFAICS it would be trivial for them to require it through new contracts, put some fines on not implementing it and all this disappears ... I'm sure there are some owned computers on core networks but I doubt the owners would want to expose them on a vindictive DDOS attack.
Re: (Score:2)
Re: (Score:3)
It would be hard to do ingress filtering by the backbones for those larger companies, but the companies can surely do egress filtering at each edge of their networks ... just a question of sufficient (financial) incentives.
Most of the internet edge could be ingress filtered by the core, the rest should do it's own egress filtering.
Re: (Score:3)
Re: (Score:2)
Well if those companies want to have completely control and just add IP address ranges willy nilly without dealing with admins of their providers it becomes impossible, maybe just more for political reasons than technological but that doesn't really matter.
So I say fine ... if they don't want to let the providers do ingress filtering for them, just make it mandatory for the companies to have egress filtering on their network (with fines for non compliance/due diligence). If there are real financial risks th
Re: (Score:2)
Re: (Score:2)
In cases of multi-homing or failure recovery, the customer has to let their providers know what routes they will/may be announcing in order to get the BGP filtering set up correctly. Might as well set up the source address filtering at the same time.
Re: (Score:3)
The problem with this becomes what if you're a transit provider yourself. The logistics of managing that kind of fitering suck. It's why most peers don't.
There needs to be a middle ground between loose and strict like feasable. I don't want to accept packets for any route I have, nor do I want to drop any packet that doesn't head back the same direction. For reasonable filtering at that level, it needs to be "allow any packets that should reasonably come from this peer per their advisement that I can filter
Re: (Score:2)
That sort of transit is a level above edge. In such a case it is probably sufficient to stipulate in any contract that the transit customer has fully implemented source filtering. Of course, if there is ever any abuse, they'll have to pay penalties and face stricter filtering.
That is already dealt with for BGP.
Re: (Score:2)
The problem is you have to trust that peer to police their network.
It leads to a situation where one bad actor network with content can make it never successful.
Re: (Score:2)
Trust but verify, and cut them off if they refuse to do the right thing.
Re: (Score:1)
As one who has maintained an ISP's peering, it is no where near as complicated as you make it sound. Enterprise class hardware (from Cisco, Juniper, etc.) have builtin support for unicast reverse path filtering (uRPF) that's effectively processing free -- based on the routing table ("FIB" -- forwarding information base) -- very effectively preventing traffic from entering (or leaving) your network that doesn't belong there.
(As an end user, uRPF presents a small problem as the ISP DHCP server is a 10-net ho
Re: (Score:2)
As one who has maintained an ISP's peering, it is no where near as complicated as you make it sound. Enterprise class hardware (from Cisco, Juniper, etc.) have builtin support for unicast reverse path filtering (uRPF) that's effectively processing free -- based on the routing table ("FIB" -- forwarding information base) -- very effectively preventing traffic from entering (or leaving) your network that doesn't belong there.
(As an end user, uRPF presents a small problem as the ISP DHCP server is a 10-net host and I null route 10/8.)
Yes obviously, which can be implemented in 2 modes: strict, which is useless as an upstream peer because you don't necessarily have best path down to them for everything you're hearing, or as loose, which is again useless as an upstream peer because you might as well turn it right off.
Dude, clearly you have no idea what you just read.
Re: (Score:1)
Incorrect. I'll say it again: It's not as complicated as the "haters" claim. I've maintained "BCP38" in an ISP network with transit links (aka. isp-edge routers with default routes.) While it's not perfect -- because "everything" is potentially on the other side, there are steps to be taken. (I ultimately cannot prove a packet with a source address of paypal actually came from them unless I'm directly peered with them.) You know what's inside your network (read: as the network operator, you d*** well bett
Re: (Score:2)
Let me try this as simple as I can. Just because you ran BGP with your provider, does not make you a peer or transit network.
You just said default route. That is a leaf node. You're at the end of the world. You are not peering. uRPF is useful when you're a leaf. It is *completely useless as a real peer* in it's current form.
Let me illustrate this for you with a completely made up scenario: You are Telia, you peer with Abovenet in 3 places, how do you configure uRPF on those links so that it keeps spoofed pa
Re: (Score:1)
(per peer interface) "ip verify unicast source reachable-via any" This is the less desirable "loose" method, and doesn't work if you have a default route (with 0/0 matching everything.) It won't necessarily stop all spoofing, but it will significantly cut it down. "via rx" is always preferred, but in this case, each site may not prefer a given network through it's local connection, instead crossing an internal link to another site. (this is also asymmetric routing.)
Once again... the only way to completely e
Re: (Score:2)
r larger customers that do their own routing from multiple subnets and might legitimately start to send traffic from an IP allocation you have no idea about and thus have blocked because one of their upstream links with a different provider went down.
Assuming they aren't relying on asymmetric routing (a bad thing), if you don't know about a range being sent to you by a customer, how can they receive a reply?
Still, there's no real excuse for not doing this on the edge of networks that are only ever going to have a single known block of IPs behind them though.
Works for dynamically routed networks as well, if they aren't advertising the range through you, don't accept it. That's not going to block any legitimate traffic unless you have route filters and the customer didn't follow the process to get new ranges added to your filters.
The customers and edge ISPs should stop all this. If they don't, they sh
Re: (Score:3)
I agree for edge networks there is no good reason for RPF not being enable but you hit the nail on the head when it comes larger customers that have an AS or multiple AS allocations and ip addressing they may not share with you. Its not really as simple as just throwing a switch at most of the sites which really matter.
As far as the home and SoHo users I don't know how the rest of the world is but I don't know any main stream ISP that isn't doing some kind of reverse filtering. I have not been able to get
Re: (Score:2)
All it takes is to kick those handful of sites off the internet and problem solved.
Re: (Score:2)
RPF wont, ACL's will and it's trivial to take a BGP prefix list and turn it into an ACL. The more it's implemented the better it works, 100% penetration is not required for it to be effective though. As you push these attackers to use the undefended spaces more and more pressure is put on them to clean up there act. Most of the source points for this seem to be Hosting/Colo where the filters are pretty trivial to get in place even if it's just on your own edge outbound.
Re: (Score:2)
I agree that will help a lot but it still won't solve the problem. The problems is the size of the sub that's through skateboarders just upset on out there. You can always weapon eyes local subnet cause her is no router to enforce ACL hosts and talk to each other directly. You spoof a few packets 100 or so little Soho routers out there each with 5 Mb upload and you got quite a lot of bandwidth right there. All of that traffic will indeed be sourcing local network with both the ACL's OraVerse path filter
Weapon eyes (Score:2)
Re: (Score:3)
Is transit billing not a good enough one for you? Selling there own DDOS protection or transit bandwidth to others to do the same. Seems like good reasons for them to not want to.
There are potentially serious issues with tier 1's putting this in place today with there peers etc. Anything that is not a BGP speaker should have his on today, BGP speaking clients should be given a timeline to be ready for this to be turned on (there is some broken bits out there). Tier 1 peers is another story but if everyt
Why not rate limit? (Score:2)
So why don't NTP servers limit their responses to, say, 1 per 10 seconds per IP address? Even if spoofing, it would not take that long to exhaust the subnet of the attack target.
Re:Why not rate limit? (Score:4, Insightful)
They're all buggy commodity routers which are never getting updates.
Re: (Score:3)
They're all buggy commodity routers which are never getting updates.
Relatively recent Juniper JunOS versions respond to ntpdc monlist, as well, so they're vulnerable. The only way to address these, I found.... was to completely firewall off NTP on the loopback interface.
The same for a number of other appliances, that are still technically supported, but the vendors seem uninterested and unconcerned about NTP issues, so much so, that they are only suggesting workarounds such as "turn off NTP", no i
Re: (Score:2)
i see you've also had to deal with this plague. stupid juniper switches are unable to work as ntp clients only. as soon as you configure ntp settings, they reply to client requests. a few weeks ago, i learned this the hard way when all my switches suddenly became overloaded and all my bfd sessions started flapping.
Re: (Score:2)
Juniper advisory:
http://kb.juniper.net/InfoCent... [juniper.net]
JunOSe and ScreenOS unaffected.
Re: (Score:3)
Most modern servers don't respond to the offending command (monlist) at all. Older/misconfigured servers are the problem and there are enough of them to cause trouble.
Re: (Score:2)
So why don't NTP servers limit their responses to, say, 1 per 10 seconds per IP address?
You couldn't bother spending 5 minutes reading to learn that the issue only exists on NTP implementations that allow administrative queries, and on modern NTP implementations that's off by default?
By the way, NTP servers CAN [redhat.com] be configured with 'discard' and 'restrict limited' statements, to restrict the rate at which clients can query, and send KOD packets if a client is querying too often..
But that's not the
How is a 12 year old finding out his IP address? (Score:2)
Maybe this is another reason to use TOR or something more generic to mask IP? Not for privacy, but to hide in the crowd. Google wants to know everything anyway....maybe they should offer a service to be a web-proxy-server.
booster/stresser sites (Score:1)
These services are available for any kid with five dollars. The last one that hit my network knocked us off and our upstream provider. They use spoofed packets to machines with services such as chargen/echo to amplify the attacks. If you contact one of these services they will threaten or try to extort money from you.
Re: (Score:2)
No $5 booter is going to do anything close to this kind of damage. A gigabit maybe, but that's about as high as it would go, despite their claims.
However I'm sure some will be adding NTP amplification to their "services."
Re: (Score:2)
I tried one against myself for a minute last year and saw about 4Gbit/second of port 53 UDP traffic. Enough to cause problems for an amateur-hour webhosting service. Any half decent webhost can handle that these days.
Re: (Score:2)
4Gbit? Did you have a 10 gigabit port or something?
Even so that's not even in the same league as what we're talking about.
Then there's the human end (Score:2)
I can't help but notice all the comments so far are about technical prevention. If it is possible, well, that would be great. But for those who dodge all technical barriers and pull this off, maybe its time for some laws equivalent to those insanely high penalties for file-sharing. It's not like a 200Gbps attacks are inadvertent or accidental; they take some deliberate effort. Make it a criminal-record, no-passport, ruin-your-employability, year-in-jail kind of crime. I suppose the 15-year-old in Ill
Re: (Score:3, Interesting)
The problem with that approach is that a lot of those internet criminals are actually just immature teenagers - all they really need is a slap on the wrist to scare them straight and a good talking-to by their parents. Throwing them in jail is a good way to make sure they turn into real career criminals - if you can't get employment in legitimate work, what other choice is there? It's the same problem with heavy sentences for drug possession.
Almost every decent computer security expert dabbled in black-hati
Re: (Score:3)
"Almost every decent computer security expert dabbled in black-hating a little"
Oh, my. I had no idea that the computer security field was so rife with racists.
Re: (Score:2)
Since you care to differentiate a hatter from a hater, perhaps the word you're looking for is "blackhatting". Note the spelling.
Re: (Score:1)
Almost every decent computer security expert dabbled in black-hating a little when they were learning
Nope, I was never racist about it.
Re: (Score:2)
The problem with that approach is that a lot of those internet criminals are actually just immature teenagers - all they really need is a slap on the wrist to scare them straight and a good talking-to by their parents. Throwing them in jail is a good way to make sure they turn into real career criminals - if you can't get employment in legitimate work, what other choice is there? It's the same problem with heavy sentences for drug possession.
Almost every decent computer security expert dabbled in black-hating a little when they were learning, if only to prove to themselves what they could do or for the fun of adventuring into forbidden places. I used to port-scan for open netbios shares back in the win9x era - found a lot of people who had their entire C: drive open to the world. I left text files on their desktops warning them about the open access.
Ok, public caning in the town square. One lash for each gigabit of wasted bandwidth, plus $100 fine for each of the same.
Re: (Score:3)
Appropriate to what 1961 would have called a science-fiction crime, the punishment taken from Starship Troopers. I like it.
Re:Then there's the human end (Score:5, Insightful)
Problem is, people who do this stuff professionally are pretty much immune from being caught, and the people who do get caught are usually teenagers which, while we like talking about personal responsibility, biologically young brains really do have physical issues when it comes to impulse control and risk analysis. So punishing them harshly does not actually do any good other then satisfying a certain bloodlust.
Re: (Score:1)
DDOS'ing a site? Lock 'm up for, at least, twenty years. The Internet is essential for civilized life nowadays. We wouldn't be lenient on people that blew up power lines, why be soft on the cyber criminals? Arguably they commit worse crimes.
Find them, charge them, incarcerate them. Put it all over the news. Make it known the Internet is not the be tampered with, just like power lines, gas lines, and other essential infrastructure.
That will stop them. Or at least it will stop them for doing it *again*, once caught.
May I remind you the internet is a global borderless network, which makes such laws impossible to implement. Also, it does not solve the problem, because there will always be a new guy stupid enough to DDOS someone. So, forget about laws. Just fix the internet.
Then introduce borders with Son of SOPA (Score:2)
May I remind you the internet is a global borderless network, which makes such laws impossible to implement.
Then perhaps the solution is to introduce borders, to implement something like SOPA except reworded to be not quite as unpalatable to civil libertarian types.
Re: (Score:2)
Y2K (Score:2)
But I am not thinking some nice gradual switch over, but a nice 'if you don't upgrade by X time you loose your insurance and can no longer peer'. If nothing else we could kill at least two birds with one stone... think about the
Re: (Score:2)
But I am not thinking some nice gradual switch over, but a nice 'if you don't upgrade by X time you loose your insurance and can no longer peer'. If nothing else we could kill at least two birds with one stone... think about the massive economic fallout from the Y2K update, all the money that flowed into tech and job for that had a ripple effect through the economy. Requiring a complete upgrade of the internet would put a real dent in the current economic downturn.
Another benefit: we can see the sequel, Office Space 2, and see how Initech inflates the work needed to solve the spoofed-source problem. Will it end in another fire? Will Milton come back?
Solution: Get rid of kids! (Score:1)
Require ISPs to do checks on IP spoofing. Case closed for most DDoS attacks. Optimization always comes at a cost of security. I'm not even an expert and still know the solution, just like a kid can read and click through a premade tool, fill out some forms and do attacks.
Kids don't have the moral subroutines to understand restraint. Anyone with a minimum amount of knowledge can fire off attacks these days, it seems.
Solvable (Score:2)
Re: (Score:2)
You're talking about an entirely seperate issue. This is all about the NTP amplification, as the article says a server on a gigabit port could theoretically push 200 gigabits.
I run a service that has 2 dedis on 10 gigabit ports..what if those got compromised or were owned by those unethical?
Overregulation (Score:2)
iOS [fares] even better because all software approved by Apple for Appstore are screened.
But if most home computers are locked down to run only software chosen by the monopoly "App Store" chosen by the computer's manufacturer, then how will high school students enrolled in an introductory programming class complete their homework?
Finally, penalizing countries that continue to support software piracy will also help.
That or penalizing companies that refuse to sell their products at all in certain countries. In affected countries, copyright infringement is the only way to obtain a copy of the work at all.
No high school iOS developer program (Score:2)
However, as long as the right legal systems and enterprising businesses exist, ventures like Android will keep popping up to balance out (and eventually crush?) 'monopolies' like iOS.
Then where's the 4" Wi-Fi-only tablet that can run applications designed for recent versions of Android as a competitor to the iPod touch? Or are people supposed to just buy a phone, not activate cellular data service on it, and pay for a GSM radio that they'll never use?
[In an App Store world,] how will high school students enrolled in an introductory programming class complete their homework?
As for young aspiring coders, they can use a free student certificate
Since when? This page [apple.com] states that only accredited postsecondary degree-granting institutions, not high schools, are allowed to participate. Besides, the parents would still need to buy the student a Mac on which to run Xcode; it does not ru
Spoofing permits attacks here's the solution (Score:2)
Then make the originating network legally and financially responsible for not filtering the spoofed packets originating from their network with a IDP (Internet Death Protocol) for any networks which do not fix their network within 3 days following a attack launched from their network.
DDOS is always big (Score:2)
DDOS causes more lost money than other "security" breaches. Therefore it is a top priority of companies and by extension public/private partnerships.
Of course, this is an asymmetric attack and you can't stop it. In other words, it is a democratizing attack.
When I worked with the FBI on security issues in the financial sector, I was disgusted by how little attention and funds were available to fix problems like unauthorized transactions but attention is available for issues like this.