Fight Spam With Nolisting 410
An anonymous reader writes with the technique of Nolisting, which fights spam by specifying a primary MX that is always unavailable. The page is an extensive FAQ and how-to guide that addressed the objections I immediately came up with. From the article: "It has been observed that when a domain has both a primary (high priority, low number) and a secondary (low priority, high number) MX record configured in DNS, overall SMTP connections will decrease when the primary MX is unavailable. This decrease is unexpected because RFC 2821 (Simple Mail Transfer Protocol) specifies that a client MUST try and retry each MX address in order, and SHOULD try at least two addresses. It turns out that nearly all violators of this specification exist for the purpose of sending spam or viruses. Nolisting takes advantage of this behavior by configuring a domain's primary MX record to use an IP address that does not have an active service listening on SMTP port 25. RFC-compliant clients will retry delivery to the secondary MX, which is configured to serve the role normally performed by the primary MX)."
Attacks on 2ndary relays (Score:3, Informative)
Since in our case, the 2ndary MX was a dumb sendmail relay only without knowledge of the user DB, it shot the traffic load out thru the roof with bounces to junk spam that, because they couldn't be rejected during the actual delivery attempt, hammered our backup relay.
This is just a dumb idea.
That's "greylisting". (Score:5, Informative)
It's been pretty much defeated now because so many spammers have their machines try to hammer the message through until it does go through.
I'm using greylisting right now and the only advantage is that many times a spammer will end up on an RBL during the 15 minutes that I'm refusing his messages.
Remember, the spammers have, effectively, unlimted bandwidth and unlimited processing power at their disposal.
Not as good an idea as it sounds (Score:4, Informative)
Re:Temporary Solution (Score:3, Informative)
Re:Oblig. (Score:3, Informative)
Whiney Mac Fanboy is a subscriber. They (subscribers) get to see the articles before us mortals. First post isn't hard when you can reply to the article before the article is available to the unwashed masses.
Re:Won't work. (Score:3, Informative)
he has results that dispute it.
If you take a look at his page, he says that he used DNSBL.
DNSBL host != spam-bot
Spam-bots are a subset of the hosts that would be listed in a DNSBL.
Next time, before attacking someone, you might want to work on your reading comprehension skills. You'll look like much less of a fool.
Just like MailHurdle (Score:2, Informative)
It works wonderfully. We've been using for about a year at my organization. It works by initially rejecting all incoming mail from unknown servers. If the server is legit, it will retry the email, and on that retry, MailHurdle will allow the mail through.
It instantly eliminated well over half of our incoming spam. Very clever technique, and it certainly works.
Re:Temporary Solution (Score:3, Informative)
I handled other domains on the same servers, so I'd still see the requests come in
Re:That's "greylisting". (Score:4, Informative)
see: http://it.slashdot.org/comments.pl?sid=132222&cid
From the FAQ (http://www.olympus.net/doubleVerifyNL):
DoubleVerify gets two chances to automatically identify mail. When mail arrives at our mail server the first time our server requests the sending mail server to send it a second time. Spammers rarely comply. Legitimate mail servers typically resend the mail about fifteen minutes later. Once OlympusNet receives mail the second time, it immediately delivers that mail and continues to immediately deliver mail from that sender. The DoubleVerify process works invisibly and is handled automatically by the mail servers.
Re:Spammers IGNORE the MX priority (Score:3, Informative)
Re:Just like MailHurdle (Score:1, Informative)
Re:That's "greylisting". (Score:3, Informative)
The trick is that I don't just use greylisting, I use greylisting + spamtrap driven RBLs, so that once the greylisting period runs out the RBLs have a much greater chance of having been hit by the same spammer and thus they catch him.
Grylisting on its was a temporary fix, but it makes spamtrap driven RBLs pretty much bulletproof.
You could get pretty much the same result simply by tarpitting connections that would have been greylisted for 15 minutes rather than giving the immediate error code and then checking the RBLs before receiving the body of the mail.
Re:That's "greylisting". (Score:4, Informative)
A better solution would be to ignore the problem, because those appliances are broken and need to be replaced or fixed no matter what.
Re:Temporary Solution (Score:4, Informative)
This means that servers *must* be RFC-compliant to deliver mail to a no-listed server - they must try to deliver to servers in the published order, and must try at least two.
The big advantage with no-listing is that if the sending server immediately tries the secondary after the primary fails, here is almost no delivery delay.
The big disadvantage of course is that an RFC-compliant spammer gets almost no delay either.
Re:That's "greylisting". (Score:3, Informative)
Re:That's "greylisting". (Score:4, Informative)
However, I still believe that the best way to handle this situation is by not working around it. When users complain that a good fraction of their mail gets bounced for no apparent reason, there may be action. When you implement a workaround, things will remain as they are.
This does not only affect greylisting. I have seen bad SMTP bugs in NAI's virus checker, "SurfControl E-mail Filter", "logsat spamfilter for ISP", and another spamfilter whose name I forgot. tried to issue bug reports via their support system. It often is near impossible to submit a bug report when you are not a user of their product, and once you get through they are completely uninterested when you are not Microsoft or Sendmail. Pointing them to the RFC does not work at all, they fix bugs by the "if it delivers mail then it must be OK" paradigm.
Re:I run a high volume mailserver, this is a bad i (Score:4, Informative)
I run a fairly low-key server, which I only use for my family, so I am not sure how relevant my data is.
I remember at one point last year checking on the usage my backup MX gets and was surprised to see a lot of mail coming through it. Surprised because my primary server is (almost) always available. Upon a closer inspection I was astounded by what I found: all the email that came through the backup MX was spam for the past year was spam. No exceptions!
Certainly, mine is an extreme case, but I think the trend is very clear.
Re:Temporary Solution (Score:3, Informative)
http://www.joreybump.com/code/howto/unlisting.htm
Re:MOD PARENT UP +5 THE FUNNAH (Score:2, Informative)
Except it wasn't filled in consistently.
These are incorrectly checked:
(X) Many email users cannot afford to lose business or alienate potential employers
(x) Dishonesty on the part of spammers themselves
(x) It will stop spam for two weeks and then we'll be stuck with it
(x) Asshats
The plan loses no email that is distributed by an actual mail server. Even the crappiest actual mail server out there follows the rules by checking another MX server, and if it doesn't it's going to lose a lot of mail anyway. Supporting multiple MX records isn't some obscure part of the standard, it's a major requirement, and all actual mail servers do.
And spammers can't 'lie' their way around it. They can use software that operates correctly in the first place, but the years have demonstrated exactly how long it takes them to switch. I have no idea how long the spam software pipeline is, but spammers have operated software that is broken in many ways, and people have been consistently using that brokenness to block spam for years.
If this reduces spam for a time and then stops reducing spam, I'm failing to see what the problem is. I'm still checking that the MAIL FROM domain is a real domain, and it's astonishing how much spam doesn't even bother to do that. Or checking that the HELO is not a negative number. (I have no idea what that's about.)
And we won't be 'stuck' with it. It doesn't change anything. People can point to a fake MX server for however as long as they want, and then switch back to just having their one real one, whenever they want.
And the 'asshats' check box is used to mean people can abuse or break the system. I have no idea why it was checked.
About the only one correctly checked complain is:
(X) Eternal arms race involved in all filtering approaches
Yes, it's an arms race, and, yes, it will lose power over time as spammer's crapware adapts. Aaaand? At the very least we cost spammers money, and upgrading spam software is insanely expensive. We didn't hurt ourself in the slightest.
I WTFA... (Score:3, Informative)
...and encourage readers to RTFA, where I've addressed many of the issues brought up in these comments. I also encourage people to try the technique, if they are in the position to do so (admins only, this is not a solution for endusers), and evaluate it for themselves. Or not. It's true that most new antispam solutions are dreamed up by crackpots. I might be a crackpot. If this possibility concerns you, don't be an early adopter. Wait and see.
It's true, in my experience, that Nolisting stops some spam with no false positives (in my experience). And that's a Good Thing. But it doesn't stop significantly more spam than a combination of other techniques, which I also implement. Some of those techniques use a lot of resources, such as content filters (often powered by perl) and virus scanners. Nolisting provides a way to free up some of those resources, possibly resulting in better performance and even hardware savings. These savings can be significant at large sites that currently scan each and every message that arrives.
Nolisting can be bypassed. I don't make any wild claims. Spammers can get past it easily by going directly to the secondary MX. Guess what? They already do that, and have been doing that well before greylisting was introduced. Nolisting significantly reduces the percentage of spam my MX processes, thereby freeing up resources. It's just one part of a layered solution.
I've limited secondary MX access by extending Nolisting into Unlisting (Port Knocking for SMTP): http://www.joreybump.com/code/howto/unlisting.html [joreybump.com]. It's wildly effective, except for one serious problem: A retry might originate from a different IP. This appears to be legal, and seems to be the result of load balancing strategies adopted by some important sites. For that reason I don't recommend it. It will randomly block messages from gmail, for example. You can't reasonably predict the IP a multihomed host will use for a retry, so be very skeptical of any approach that claims to have solved this problem.
Unwanted email is annoying. When it carries a payload, it is potentially dangerous. But I don't really view this as a security issue. I don't buy the argument that Nolisting is security by obscurity, and therefore bad. It's a form of access control, a gatekeeper, a prophylactic. It's an apple a day, not a cure for cancer. It's not addicting, fattening, or life-threatening. Try it, if you're looking for ways to improve the health of your mail system. Discontinue use immediately at the first sign of complications. Side effects include more sleep and time spent with your kids.
Nolisting rarely introduces delays. As I point out in the article, most relays retry immediately. Any relay that cannot get beyond Nolisting is seriously, seriously noncompliant. While I don't suggest Nolisting as a complete replacement for Greylisting, it is a viable alternative for sites that experience problems with Greylisting and find the delays it introduces to be unacceptable. As the name implies, Nolisting is meant to used without dependence on whitelists. Wider adoption and testing will determine if this ideal has been realized.
Like Greylisting, Nolisting breaks infrastructure to some degree. Many admins find this distasteful. I know I do. If Nolisting becomes widely adopted, logs will become fatter with "Connection refused" errors when the primary MX doesn't respond. I'm sorry for that. But our logs are already fat with 45x errors from Greylisting, RBL disconnections, SpamAssassin scores, etc. Nolisting might even help to make logs smaller, if you currently see a lot of these messages. Time will tell. Keep an open mind, and remember that we often make concessions to improve a system's overall health. Just reducing the possibility of another zombie being created on the Internet creates benefits for everyone.
Try it before you draw a c
Re:Oblig. (Score:2, Informative)
The acknowledgements don't say anything about running specific software, or making any changes to software.
I've been doing nolisting for about three months now, and it required:
1) Making three A records, mx1.example.com, mx2.example.com, and mx3.example.com, all of them pointed to my IPs (Don't abuse the internet by directing traffic randomly elsewhere, people.) with mx2 being my already existing mail server and the other two being IPs without mail servers.
2) Set all the domains I felt like it to use those 3 MX records in order.
That was it. I didn't touch my mail server at all, I didn't even bother with firewalls, because my server already has a firewall setup.
Re:Oblig. (Score:3, Informative)
How did this get marked insightful? Sending a temporary failure SMTP response code is not a bounce, and should not result in the generation of a bounce message except from psychotic MTAs.