DynDNS Drops Non-Delivery Reports 195
jetkins writes "In an email to subscribers, DynDNS announced that they will no longer deliver locally generated non-delivery reports (NDRs) from any MailHop systems. MailHop is a multi-faceted service offering in- and outbound relay services, spam and virus filtering, and store-and-forward buffering. DynDNS makes it clear that they are aware that this goes against RFC 2821 Section 3.7, but explains that in their opinion the increase in spam volume, and the use of NDRs as a spam vector, means that the value of NDRs is now far outweighed by their potential for harm. (DynDNS also points to the far greater reliability of email systems now than when the RFC was approved.) The company notes that other ISPs have quietly dropped RFC 2821-compliant NDRs. Will their public move start a flood (mutiny) of ISPs following suit? Should they have made efforts to have the standard changed instead of defying it?"
Change or Defy (Score:4, Insightful)
If a RFC said all boxes should have a port that users could telnet into with root access, and people start abusing that would you leave it and wait for the standard to change?
SPF? (Score:5, Insightful)
And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.
Re:What I'd like to see... (Score:5, Insightful)
to defy the laws of tradition (Score:5, Insightful)
maybe by defying it, the standards will now be reviewed, and eventually changed.
The Problem Is Not NDR's (Score:5, Insightful)
The main problem is a you have a system based on blind trust.
Second trust based duct-tape systems are simply too cumbersome for the average user.
I don't have the answer but I do know that email in it's present state is broken.
Standards and Implementation (Score:5, Insightful)
Going against standards can cause a bit of chaos as well, which is why it's good to avoid deviation - but sometimes a deviation makes sense, and you do it. Publicly announcing this (non-critical) deviation, and explaining exactly why, is the proper way to go about it.
Re:RFC-Ignorant.org (Score:3, Insightful)
Once that initial connection is gone though, the receiving mailserver should not assume that it has any way to talk to the sender again.
Re:RFC-Ignorant.org (Score:3, Insightful)
Re:What I'd like to see... (Score:3, Insightful)
Re:RFC-Ignorant.org (Score:3, Insightful)
Re:Standards and Implementation (Score:3, Insightful)
The process of modifying standards is a bit more complex [rfc-editor.org] than that, but there is a process for change. You just have to become part of it rather than just picking and choosing which standards annoy you the least and then hoping that someone else will fix the ones that don't work the way you think they should.
Are DynDNS cluebies? (Score:5, Insightful)
Excuse me, but due to the vast amount of spam handling, modern e-mail systems are substantially less reliable than they used to be.
If you redirect email for your domain name to Hotmail, chances are good that it will disappear without a trace. (No NDR, not in the spam box either.)
Someone else already mentioned the problem of people typoing email addresses. This is a common problem.
Email can be bounced for other reasons, too, such as a full mailbox, or that there is a relaying mail server (yes, DynDNS, they still exist, and in abundance!) which gives up on delivery after a week of timeouts for the destination host.
And so on.
Someone at DynDNS needs a good whack with the clue bat.
NDR's are not evil (Score:5, Insightful)
The problem is not with NDRs. The problem is that their servers *accepted* the message that eventually had to be NDR'd in the first place, then after accepting responsibility, decided they didn't want that responsibility, so discarded mail that they promised they would deliver.
If their servers checked validity of local recipients, scanned and filtered the message, etc BEFORE accepting it (via 2xx series SMTP accept response), and instead properly REJECTED it with a 5xx series response, these messages would never be bounced. The NDR mechanism is not at fault - rather, the fact that they can't properly configure their servers to reject the message up front is at fault. If you properly REJECT the messages at the SMTP level instead of accepting the message for delivery, the only thing left to NDR are perfectly valid cases, such as mailbox over quota, etc.
Once you *accept* responsibility to deliver a message (via a 2xx series SMTP response), you MUST deliver it somewhere, else you have shirked your responsibility - either deliver it to it's destination, or bounce it. To do anything else would be to LOSE mail, which is the ultimate sin of any mail server. The key is not to throw out bounce messages, but to minimize or eliminate unnecessary bounces in the first place by rejecting instead.
Note that by properly REJECTING the message, you also effectively defeat most spam bots, since they can't "bounce" the mail that you reject to the "real" local sender.
I always hate it when providers like this take the short cut of *losing* mail intentionally rather than fixing their broken systems to work right.
One caveat to my comments - unfortunately, some mail software is symply not geared toward todays Internet, such that it can do the scanning and filtering of messages realtime fast enough to prevent a sending server from timing out while it's doing this scanning, so they queue the mail to process it for spam, etc later. Using such software is the first mistake most places make, and is the real reason why there are so many NDR's in the first place.
Re:RFC-Ignorant.org (Score:3, Insightful)
I see this as causing more people to be sending messages with delivery/read receipts so that they will know that the message was properly delivered and/or read. I am afraid that this will lead to an increase in traffic due to these receipts being requested more. But then I guess that servers will start dropping the receipts as well and then sending an email will be no more reliable than sending a snail mail. It is supposed to be a guaranteed system of delivery, but there is a huge dead letter barrel that gets way too many things.
Re:Finally, a service provider with a clue... (Score:2, Insightful)
Re:Problem is legitimate, solution is not (Score:2, Insightful)
Not necessarily. Backup MX services could do address validation if they're given a userlist. Of course, this entails some security concerns (example: why trust a backup service with a userlist?), but that can be figured out satisfactorily (answer: use a backup service you can trust, and engineer a secure solution).
Further thoughts:
There is little reason to avoid address validation these days. As for the argument against address validation -- that spammers can determine valid addresses by seeing which ones don't fail, and that's a reason not to validate -- that's not much of an argument:
- Accept and discard, and spammers think all addresses are real.
- Accept and generate an NDR, and spammers never get it, because their sender address/domain is usually bogus.
- Reject unknown addresses, and reduce massive CPU overhead from spam/virus-scanning. Bonus!
Finally, implement some form of connection throttling based on address validation. Some connection's doing what appears to be a dictionary attack, trying a jillion usernames starting with the "a"s? Or even multiple connections doing it, subdividing the alphabet between themselves? Trip a threshold of, say too many connections or too many bad recipients per minute, and refuse further SMTP connections from the given IP address -- or even slip it into a packet-filter rule to blackhole -- for an interval of time, and the problem's solved.
Once spam makes it past the DNS blocklists, the connection throttling, etc. there's enough intelligence in the anti-spam middleware to accurately identify the ham from spam 99% of the time or better.
And of course there's DomainKeys Identified Mail (DKIM) which is now RFC 4871 and is excellent for this middle section of the anti-spam defense wherein one also has to determine whether a message is an elaborate phishing scam or the real deal. SpamAssassin can do a bit of this as well as SPF, etc., and there's a high-performance dkim-milter implementation on sourceforge, too.
SMTP isn't dead, just a tad more complex on the server level, and defenses are improving against the soft areas of the protocol. What I've suggested works pretty darn well as a starting point.
Re:RFC-Ignorant.org (Score:4, Insightful)
So USE that information. (Score:5, Insightful)
#1. The spammer already HAS the account name and is checking to see if it still works. Defeat this by generously distributing SpamTrap accounts. And accepting email to them. And then opt'ing out of the email that they receive.
#2. The spammer is trying to guess a new name. Good luck with that. Sure, maybe SOMEWHERE there is an email account of "frank@example.com" but good luck finding it. If you want to have some FUN, watch your logs for examples of this. Then setup some of them as SpamTraps. And follow #1 above.
I use both of these approaches. It makes filtering spam VERY easy.
Re:NDR's are not evil (Score:3, Insightful)
A similar problem occurs when you submit outbound mail to your ISP -- unless it's going to someone else at the same ISP, the local SMTP server can't verify that delivery will succeed. At the ISP level it's still probably reasonable to generate bounce message, at least for local users. That way you don't have to do the final delivery right away, users can still get error messages, and you don't risk sending bounces across the Interwebs.
But it gets tricky for forwarding hosts like DynDNS. The only way to reject during the session would be something like:
A) Inbound connection sends message to forwarding server
B) Forwarding server leaves connection open after the DATA command
C) Forwarding server forms outgoing connections to every domain specified in the envelope addresses
D) Forwarding server attempts to deliver the message to all final recipients
E) Forwarding server rejects original connection from (A) if any deliveries in (D) fail
And even if you overcame the technical complications of such a system, it completely dis-allows the use of mail hosts that don't have 24/7 availability. Moreover it eliminates the ability to use secondary hosts, because even if they can validate addresses (which is not trivial) they can't guarantee things like available disk space on the destination server.
Rejecting messages in-session is certainly a good idea, but it's ridiculous to think that it's possible to eliminate bounce messages just by validating local addresses before accepting a message.
Re:RFC-Ignorant.org (Score:3, Insightful)
If someone is going to pull off a dictionary attack against the SMTP server, then you just discard connections to them after a specific number of invalid users.
Almost all mainstream MUAs support this sort of thing now.
At the end of the day, if you actually accept the message for delivery and later reject it, you should do so silently.
Re:Finally, a service provider with a clue... (Score:3, Insightful)
"If you think you can create a foolproof system, you are one of the fools" - No idea who I'm misquoting.
Re:What I'd like to see... (Score:3, Insightful)
I wouldn't probably think to check a forum for an announcement on a free-shipping sale or a closeout on last year's tires, but if it comes in an email, it gets my attention, and I appreciate that.
My ISP offers spam-filtering, but I have turned it off because too many of these mailing lists I like were getting caught in its trap. So I agree that mailing lists make spam filtering more difficult, but I personally see them as being worth the hassle.
Re:RFC-Ignorant.org (Score:3, Insightful)
Re:Change or Defy (Score:2, Insightful)
By "abbreviated," I mean mail servers should look at incoming apparent NDRs, drop most of the message content, and provide summary information instead. So instead of getting a fake NDR with a SPAM payload, you'd get something like "Your message addressed to fakeaddress@someplace.com, with subject beginning 'First three words,' could not be delivered. [...]"
People will still (temporarily) get a number of bogus NDR messages, but once spammers see nothing but the first three words of the NDRs are getting through, they'll give up on using the technique. Any messages crafted to exploit the first three words of these summary/abbreviated NDRs would be fairly pathetic compared to normal SPAM, plus they would be easily captured by updated spam filters. (Subjects like "pr0z@c ch3@p www.cheap-prozac.com" would be a total give-away.)
Re:SPF? (Score:3, Insightful)
And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.
Which is why you run a real secondary MX that can either do recipient callout or use valid recipient lists in order to reject during SMTP. DynDNS is a cheap hack for geek vanity domains - you're getting exactly what you pay for.
POTS (Score:2, Insightful)
Re:Finally, a service provider with a clue... (Score:3, Insightful)
We already moved from SMTP to ESMTP. Maybe we can go a step further, with, I don't know, ASMTP or whatever you want to call it. Then impose extra controls on messages that arrive via SMTP or ESMTP.
In any case, there is absolutely nothing we can do about zombies, unless you want to implement some kind of "are you human" test for every single e-mail that is sent. And how could we do it between relays ?
I have an answer (Score:3, Insightful)
Yeah, whiners on Slashdot say CAN-SPAM is horrible, because it legalizes spam. What they forget is that CAN-SPAM only legalizes it under certain rules, which spammers are ignoring because there's no enforcement. According to this article from last year [techweb.com], only 0.27% of all junk mail actually complies with CAN-SPAM, which means the other 99.73% is clearly illegal. On top of that, the 0.27% is deliberately easy to filter out if you choose.
We don't need a new law to make spam illegal; CAN-SPAM already makes it illegal. We just need to start actually prosecuting people who break the law.
Yes, some spam comes from other countries where the FBI has no jurisdiction, but not as much as you might think [spamhaus.org], and I believe the FBI already has partnership agreements with agencies in several other countries to work together on fighting spam - they're just not doing enough of it.
Why won't this happen without an act of Congress? Because without a direct congressional mandate, the FBI has better things to do with its time and money. I don't blame them, really - raiding meth labs or catching serial killers is certainly important. But fighting spam is important too, and there's no reason the FBI couldn't do both.
So that's the answer. Spam is a social problem, more than it's a technical problem. We can try to fight it with technology, but spammers are fighting back, and they have the huge advantage of not being limited by morals or legality. We can't win with the odds stacked that high in their favor. The only way to win is to throw them all in jail.