DynDNS Drops Non-Delivery Reports 195
jetkins writes "In an email to subscribers, DynDNS announced that they will no longer deliver locally generated non-delivery reports (NDRs) from any MailHop systems. MailHop is a multi-faceted service offering in- and outbound relay services, spam and virus filtering, and store-and-forward buffering. DynDNS makes it clear that they are aware that this goes against RFC 2821 Section 3.7, but explains that in their opinion the increase in spam volume, and the use of NDRs as a spam vector, means that the value of NDRs is now far outweighed by their potential for harm. (DynDNS also points to the far greater reliability of email systems now than when the RFC was approved.) The company notes that other ISPs have quietly dropped RFC 2821-compliant NDRs. Will their public move start a flood (mutiny) of ISPs following suit? Should they have made efforts to have the standard changed instead of defying it?"
Finally, a service provider with a clue... (Score:5, Informative)
Re:Finally, a service provider with a clue... (Score:5, Informative)
A properly-configured endpoint server should check addressee validity during the SMTP exchange, and reject the transfer before it even gets into the system, so the spammer's attempt goes nowhere and "Joe" doesn't get an unwarranted NDR.
Of course that doesn't help proxy providers like DynDNS, unless they have some way of authenticating their clients' valid addresses in real time via a direct connection or regular updates.
Re: (Score:2, Insightful)
Re: (Score:3, Insightful)
"If you think you can create a foolproof system, you are one of the fools" - No idea who I'm misquoting.
Re: (Score:3, Insightful)
We already moved from SMTP to ESMTP. Maybe we can go a step further, with, I don't know, ASMTP or whatever you want to call it. Then impose extra controls on messages that arrive via SMTP or ESMTP.
In any case, there is absolutely nothing we can do about zombies, unless you want to implement some kind of "are y
Re:Finally, a service provider with a clue... (Score:5, Interesting)
Bunk. Even if it was true, it's still no excuse for ignoring your responsibilities.
I run the mail servers for several domains, and brute-force attacks just don't happen. It's fairly obvious why, if you think about it. Dictionary attacks against common names are possible, but I've not seen evidence to suggest that's happening.
Firstly, I want to get back to "responsibilities". By this I mean that anyone who's connected to the internet has a basic responsibility to make at least a good-faith attempt to prevent their system being used against other people. This goes doubly for people who intentionally run publically accessible servers (e.g. mail servers and web servers). You should treat any mail system which indiscriminately generates NDRs the same way you'd treat an open relay, because that's effectively what it is. You are deliberately making a server available which will accept mail from anyone on the internet, and send it to anyone else on the internet*. This is reckless irresponsibility.
* - most NDR messages include at least part of the original message's text; at the very least, the subject line. So a system which generates backscatter does in fact carry some payload chosen by an anonymous third party.
Even if brute-force attacks on your mail server's address list do occur, there are ways to mitigate the effects of it that don't turn your system into a spam engine.
Having a look through the last 48 days logs on one of my servers, there's 2,308 addresses which were rejected because they're non-existent. The vast majority are either formerly valid addresses (i.e. of people who used to work here), or slightly mangled versions of valid addresses (presumably badly parsed). Particularly common are things starting with "3D" (presumably parsed from quoted-printable data which contains =3D) or people's surnames (smith@example.com) -- our email addresses are in the format firstname.lastname@example.com, and it would appear that some harvesters consider periods before the @ to be invalid.
The second part highlights why brute-force is impractical: the namespace before the @ is absolutely massive, and only a tiny fraction will be valid addresses. If you have no idea what format email addresses in the target domain take, you have no choice but to try everything, and that will take far longer than a week. Add to this the proliferation of very small domains with only a handful of email addresses (i.e. personal domains, promotional domains). Even with a vast botnet, trying to harvest addresses by brute force against a mail server is so horribly inefficient as to not be worthwhile. There's much easier ways to harvest addresses.
Then there's technical issues with that kind of harvesting. First, any reasonable mail server will start responding slower to a client which is making repeated errors, before finally shutting them off. This means you have to make lots of connections. Second, brute force or dictionary attacks stick out like a sore thumb versus normal mail traffic, making it trivial to block any IP which is trying to harvest addresses in this manner. The only possible way to do these sorts of attacks would be to use a vast distributed botnet, and even then it's not going to work. It would be easy (and fun) to build a system that watches for such attacks and blacklists any IP involved. Anyone harvesting in this way would then be betraying the IPs of most of their bots during the harvest! Then there's lots of clever things one can do: once you have a known harvester, start okaying its invalid addresses and add them to your list of spamtraps. Not only is the spammer not collecting any valid addresses, but you're poisoning their address list!
Brute-force attacks are too easy to detect, and too easy to use against the attacker. There's much, much easier and more efficient ways to harvest email addresses. Possibly it could be used if you're targeting a specific company or domain and can do some research into their configuration, but even then there
Re: (Score:3, Informative)
Ignoring a problem isn't the same as fixing it... NDRs serve a useful purpose assuming the original message was actually useful. The problem isn't sending out NDRs. The problem is sending an NDR in response to spam!
I've had to deal with the whole joe-job+NDR+DDOS scenario on several occasions... I have found that 65~80% of th
Re: (Score:2)
I'm certain you've seen the syndrome: Speak to the business owner and his management team about the problem in easy-to-understand terms, and their eyes glaze over
Re: (Score:3, Interesting)
What I'd like to see... (Score:2, Interesting)
Re:What I'd like to see... (Score:5, Insightful)
Re: (Score:3, Insightful)
Re: (Score:2)
In order to spoof a remote IP address you'd have to basically have to share a wire someplace between the mail server and your spoofing target, or exploit some secondary flaw on a router/host along that same path. It could be done, but there are easier ways to DoS, and most of those ways are effective beyond the single-host-to-single-mailhost-for-mail-service-on ly scope that is targeted with the attack y
Re: (Score:2)
That's a non-trivial attack though -- it's not as though you can send mail with uni-directional traffic.
You don't have to send mail with unidirectional traffic. You just have to make sure that the traffic doesn't point back to you. In other words, if you send mail from a botnet, you're still free and clear as long as you don't use too much of your botnet at once.
Re: (Score:2)
I'm not suggesting you couldn't get some box other than your own desktop blocked, or that blocking by IP would be effective at stopping spam. I was just refuting the original statement that you could use IP-scoped blocking in r
Re: (Score:3, Insightful)
I wouldn't probably think to check a forum for an announcement on a free-shipping sale or a closeout on last year
Re: (Score:2)
I do that. (Score:2)
After X failed addresses, block the sender.
Except you have to make exceptions for things like gmail and hotmail and other major ISP's and mail delivery services.
Instead of sending and NDR though, I just reject at SMTP time. If the ISP's were a bit smarter, they'd see X rejections (5xx-series) and shut down ALL outbound email from that account.
And I want a pony and a plastic spaceship and
RFC-Ignorant.org (Score:5, Interesting)
Stupid bastards.
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
Re: (Score:2)
It's easier to get forgiveness than permission, I suppose.
Re: (Score:2)
Based on a 4-digit SlashID, I'd say not...
Re: (Score:2)
Re: (Score:3, Informative)
Why are you accepting a message for a nonexistent user in the first place? As soon as the sending SMTP connection specifies RCPT you should be able to check if it is valid and terminate the connection if it is for a nonexistent user. This can all be done before the DATA command is issued. Why waste cycles virus scanning, spam screening and bouncing a message for a user you don't even have? You're not just RFC ignorant, you're ignorant of how to properly run a mail server!
Note that the method above get
Re: (Score:3, Insightful)
Re: (Score:2)
Possibly, but it does prevent the backflow DOS problem.
Re: (Score:2, Informative)
Re:RFC-Ignorant.org (Score:4, Insightful)
So USE that information. (Score:5, Insightful)
#1. The spammer already HAS the account name and is checking to see if it still works. Defeat this by generously distributing SpamTrap accounts. And accepting email to them. And then opt'ing out of the email that they receive.
#2. The spammer is trying to guess a new name. Good luck with that. Sure, maybe SOMEWHERE there is an email account of "frank@example.com" but good luck finding it. If you want to have some FUN, watch your logs for examples of this. Then setup some of them as SpamTraps. And follow #1 above.
I use both of these approaches. It makes filtering spam VERY easy.
Re: (Score:3, Insightful)
If someone is going to pull off a dictionary attack against the SMTP server, then you just discard connections to them after a specific number of invalid users.
Almost all mainstream MUAs support this sort of thing now.
At the end of the day, if you actually accept the message for delivery and later reject it, you should do so silently.
Re: (Score:2)
That works real well when the incoming e-mail is a complaint to sexual harassment anonymous hot line and the sender thinks the e-mail went through, but we silently dropped it due to a mistake on the spelling.
I hate sending and e-mail and having no idea if it ever went through or not.
So I setup all my outgoing e-mail to have delivery and read receipts to try and discover lost e-mail.
Re: (Score:2)
If the incomming e-mail is actually an anonymous complaint then there's no way to actually notify the sender in any event, is there? The best case would be the receiving MTA rejecting it immediately because it was mispelled, but if it doesn't, how do you expect it to talk to the original sender anyway
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
Re: (Score:2)
Many ISPs block 25 outbound to be good netizens and avoid their lusers'botnets spewing spam. Legit users can get the block lifted.
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
Actually in this case it doesn't matter. If all email addresses @example.com are indeed mapped to your (valid) real email address, then foo@example.com IS valid, as far as the mail
Re: (Score:2)
Outright NDR ban is just plain stupid, akin to curing headaches with guillotine. If they must do something, why not place a cap and delay on the NDR traffic.
Re: (Score:2)
Yes
Re: (Score:2)
I'm not one of the people that shouts how "email
Re: (Score:2)
Probably
The concept is nice, but I get scores of them every day from ignorant mailservers telling me that the spam that I didn't send, but had my address on it didn't get delivered. I filter them off into a folder, which frankly, I just purge every week or so. I don't have the time to read through them.
Re: (Score:2)
NDR ban sounds like a solution in search of a problem which will hurt legitimate users if this thing catches on.
Re: (Score:2)
Frankly, this article has me considering the possibility of refusing/blackholing NDRs on my own servers. I'm betting it might drop nuisance mail by as much as 5-10%
Re: (Score:2)
Re: (Score:2)
Utterly retarded idea, and an utterly worthless list.
Re: (Score:2)
Change or Defy (Score:4, Insightful)
If a RFC said all boxes should have a port that users could telnet into with root access, and people start abusing that would you leave it and wait for the standard to change?
No (Score:2)
Re: (Score:2)
Re: (Score:2, Insightful)
By "abbreviated," I mean mail servers should look at incoming apparent NDRs, drop most of the message content, and provide summary information instead. So instead of getting a fake NDR with a SPAM payload, you'd get something like "Your message addressed to fakeaddress@someplace.com, with subject beginning 'First three words,' could not be delive
SPF? (Score:5, Insightful)
And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.
Re: (Score:2)
Re: (Score:2)
Granted, you can pick up alot from logs, but not all
Re: (Score:2)
Your email will go into a catchall mailbox and it will be forwarded to the appropriate person. Yes this is tedious however 1 missed email could be a missed chances at *TONS* of business. Often times people won't email you again if they get an NDR back.
Re: (Score:2)
If SPF were more widely implemented, or required to be implemented, wouldn't this problem be solved?
Yes.
Don't send NDRs to domains without SPFs or when SPF fails.
A fair point.
And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.
Welcome to 2007. I hate to say it, but this is the state we're in. When I used mailhop, I used it for secondary MX, so I would not really have cared too much about the off chance that when my primary MX was down, you sent mail with typo in the To address. Failure recovery doesn't need to be 100% perfect for me to appreciate having it.
I like SPF a lot... (Score:2)
That said I DO wish more people would use it so that it's overall impact would be increased (as people began to rely on it more). TMDA aside (which has a whole batch of problems, I know) it's my next
Re: (Score:2)
"If I typo an email address, I damn well better be getting an NDR from the recipient domain,"
And if your typo matches a real person's email, you won't be getting an NDR. Heck, I've gotten tons of email from people who have sent their stuff to the wrong person - including the new password for someone whose name misses mine by one letter.
If the mistake originated with you, don't expect someone else to take responsibility for fixing it.
Re: (Score:2)
No, SPF/Sender-ID are bad ideas, which even their creators don't put much trust into. Don't believe me? Try sending a brand-new newsletter to Hotmail and MSN subscribers. Make sure all your Sender-ID and SPF records are in place and verified with Microsoft's own Sender-ID checker. Make sure all your WHOIS data is current, valid and not obfuscated for privacy. Setup your mail servers on freshly-allocated
Re: (Score:3, Insightful)
And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.
Which is why you run a real secondary MX that can either do recipient callout or use valid recipient lists in order to reject during SMTP. DynDNS is a che
to defy the laws of tradition (Score:5, Insightful)
maybe by defying it, the standards will now be reviewed, and eventually changed.
Re: (Score:2)
And your daily lesson in passive aggression comes to you from chef_raekwon today.
Not very Wu of him, if you ask me.
Re: (Score:2)
RFC (Score:2, Funny)
Re: (Score:2)
The Problem Is Not NDR's (Score:5, Insightful)
The main problem is a you have a system based on blind trust.
Second trust based duct-tape systems are simply too cumbersome for the average user.
I don't have the answer but I do know that email in it's present state is broken.
Start at the top, with secure DNS. (Score:2)
Re: (Score:3, Informative)
Way to confuse envelope-from, header-from and reply-to.
Besides, my home-brewed Linux-based mail server has a published SPF record, and anyone receiving mail can verify that machine is entitled to generate envelope-from with that domain. The SPF also spells out my relay provider, since my DSL line is in DSL blocklists.
What it really needs, at the least, is for people to stop accepting bogus HELO/EHLO addresses and other unverifiable envelope information. If there isn't even an A record for the HELO ad
Re: (Score:2)
Re: (Score:2)
I'm reposting this to make certain that if one point gets made here, this is it.
Re: (Score:2)
E-mail works fine, with the various hacks that have been added on to fight entropy. Dealing with normal spam is no worse than the annoyances of closed networks - you still get spam on facebook etc!
Compare and contrast e-mail with the alternatives - you get instant messaging, which solves a different problem and *still* sucks, or you place yourself at the mercy of a third party. No thanks.
If you can't use e-mail chances are you don't:
Run an well configured server (or pay an insignific
Re: (Score:2)
Re: (Score:2)
I have an answer (Score:3, Insightful)
Yeah, whiners on Slashdot say CAN-SPAM is horrible, because it legalizes spam. What they forget is that CAN-SPAM only legalizes it under certain rules, which spammers are ignoring because there's no enforcement. According to this article from last year [techweb.com], only 0.27% of all junk mail actually complies with CAN-SPAM, which means the other 99.73% is clearly illegal. On top of that, the 0.27% is deliberately easy to filter out i
Re: (Score:2)
Then if for whatever reason it has to be email, rather than courier, fax, snail mail, in person, etc, you could always pick up the phone and ask for verbal confirmation of receipt, or assume that not getting a reply confirming receipt is evidence of non-delivery.
Anyone who blindly relies on email (or anything else) being delivered, received, understood and acted upon correctly for a critical business venture without some kind of conf
Re: (Score:2)
Standards and Implementation (Score:5, Insightful)
Going against standards can cause a bit of chaos as well, which is why it's good to avoid deviation - but sometimes a deviation makes sense, and you do it. Publicly announcing this (non-critical) deviation, and explaining exactly why, is the proper way to go about it.
Re: (Score:3, Insightful)
The process of modifying standards is a bit more complex [rfc-editor.org] than that, but there is a process for change. You just have to become part of it rather than just picking and choosing which standards annoy you the least and then hoping that someone else will fix the ones that don't work the way you think they should.
Re: (Score:2)
I tend to differ...
What they're doing is making a change to a service that they provide so that their problem is resolved (which they have a right to do IMO).
It's kind of a move towards an 'ignorance-is-bliss' policy rather than fixing a problem for their customers... after all, if they aren't aware of a spam problem that their customers are experiencing then there isn't a spam problem.
I'm a f
Turn off original message in the bounce??? (Score:3, Interesting)
This is how it goes on all our mail servers. All bounced messages have the original content stripped off. You get the error message with the reason the message bounced and that's it.
NDR are still usefull. There is PLENTY of mail servers not configured properly or messed up on the Net, even from big ISPs. Calling the current system as a whole, reliable, is a joke.
Re: (Score:2)
Basically, our spam law says it's illegal to send unsolicited commercial e-mails to private individuals - there's nothing to say that you have to be the author of the spam to fall foul of that law, you are still guilty if you send me an unsolicited commercial e-mail by bouncing it to me from a third party when I'm being joe-jobbed.
A nicely worded 'please change your settings, or I'll tell the information commissioner to fine you £5,000 per m
Re: (Score:2)
NO. Once a spam MO becomes commonplace, that technique will NEVER go away.
You seem to be implying that if the "effort" is wasted in vain, then spammers will deprecate their old technique. They won't - they'll just ADD new techniques. The NDR loophole will never die.
Long deprecated in practice (Score:2)
not all sending servers can generate NDR (Score:2)
sendmail and postfix both do this. don't know about MS exchange or courrier. a default qmail install (without patches) certainly don't. i believe there's a patch to implement this
Are DynDNS cluebies? (Score:5, Insightful)
Excuse me, but due to the vast amount of spam handling, modern e-mail systems are substantially less reliable than they used to be.
If you redirect email for your domain name to Hotmail, chances are good that it will disappear without a trace. (No NDR, not in the spam box either.)
Someone else already mentioned the problem of people typoing email addresses. This is a common problem.
Email can be bounced for other reasons, too, such as a full mailbox, or that there is a relaying mail server (yes, DynDNS, they still exist, and in abundance!) which gives up on delivery after a week of timeouts for the destination host.
And so on.
Someone at DynDNS needs a good whack with the clue bat.
Re: (Score:2)
If you have a DynDNS account, chances are good that you don't forward all your e-mail to a HotMail [live.com] account. In fact, you might run your own mailserver; in that case, you can make sure that your own server returns whatever bounce messages you feel are appropriate. Even the forwarding service will normally be pointed at RFC-compliant servers, which may c
Problem is legitimate, solution is not (Score:2, Interesting)
First, by their own admission this is only a serious problem for what they call their MailHop Backup MX service. Their other services, MailHop relay and forward are "mostly immune" to DSN issues.
T
Re: (Score:2, Insightful)
Not necessarily. Backup MX services could do address validation if they're given a userlist. Of course, this entails some security concerns (example: why trust a backup service with a userlist?), but that can be figured out satisfactorily (answer: use a backup service you can trust, and engineer a secure solution).
Further thoughts:
There is little reason to avoid address validation these days. As for the argument against address validation --
Bad idea (Score:2)
Unilaterally deciding to ignore an RFC (or part of an RFC) just because you don't like it is almost never a good idea. When Microsoft does it, everyone (correctly!) gets up in arms. DynDNS shouldn't get off any easier.
At most, I would agree with a temporary block of NDRs to a particular user or domain if a large joe-job run is recognized. But this should never be made permanent or blanket.
Having read TFA... (Score:2)
The problem is one of architecture. There is no excuse in the modern world for running a secondary MX server that lacks knowledge about local recipient addresses. While this architecture may have been OK 10 years ago, it no longer is. Just don't run a secondary MX unless you have a way to transfer your account list to the secondary in a way that the secondary can have local knowledge of valid addresses even if the primary is unreachable.
NDR's are not evil (Score:5, Insightful)
The problem is not with NDRs. The problem is that their servers *accepted* the message that eventually had to be NDR'd in the first place, then after accepting responsibility, decided they didn't want that responsibility, so discarded mail that they promised they would deliver.
If their servers checked validity of local recipients, scanned and filtered the message, etc BEFORE accepting it (via 2xx series SMTP accept response), and instead properly REJECTED it with a 5xx series response, these messages would never be bounced. The NDR mechanism is not at fault - rather, the fact that they can't properly configure their servers to reject the message up front is at fault. If you properly REJECT the messages at the SMTP level instead of accepting the message for delivery, the only thing left to NDR are perfectly valid cases, such as mailbox over quota, etc.
Once you *accept* responsibility to deliver a message (via a 2xx series SMTP response), you MUST deliver it somewhere, else you have shirked your responsibility - either deliver it to it's destination, or bounce it. To do anything else would be to LOSE mail, which is the ultimate sin of any mail server. The key is not to throw out bounce messages, but to minimize or eliminate unnecessary bounces in the first place by rejecting instead.
Note that by properly REJECTING the message, you also effectively defeat most spam bots, since they can't "bounce" the mail that you reject to the "real" local sender.
I always hate it when providers like this take the short cut of *losing* mail intentionally rather than fixing their broken systems to work right.
One caveat to my comments - unfortunately, some mail software is symply not geared toward todays Internet, such that it can do the scanning and filtering of messages realtime fast enough to prevent a sending server from timing out while it's doing this scanning, so they queue the mail to process it for spam, etc later. Using such software is the first mistake most places make, and is the real reason why there are so many NDR's in the first place.
Re: (Score:3, Insightful)
A similar problem occurs when you submit outbound mail to your ISP -- unless it's going to someone else at the same ISP, the local SMTP server can't verify that delivery will succeed. At the ISP level it's still probably reasonable to generate bounce message, at least for local users. That way you don't have to do the final delivery right away, users can still get error messages, and you don't risk sending
If email is so reliable these days... (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
High-availability systems generally accept degraded performance in a double-fault situation. Really, email only needs to rise to the level of "high-availability" (as opposed to "fault-tolerant"), given users' current expectations. If you need anything better than that, users generally rely on "layer 8" (human) acknowledgment (largely b
Good for DynDNS; TZO.COM did this 3 months ago (Score:2)
This step might not have been necessary if everyone customized (read: FIXED) their Microsoft Exchange installations, but that's never going to happen.
TZO stated that 80% of outbound relayed mail was DSN from spammer attempts.
With a lot of Exchange installs, even if that server is NOT an "open relay", they WILL send out DSN's for spam relay attempts. NO mail server should send out DSNs for domains that are not their own - just reject it up front. Unfortunately tha
POTS (Score:2, Insightful)