Fight Spam With Nolisting 410
An anonymous reader writes with the technique of Nolisting, which fights spam by specifying a primary MX that is always unavailable. The page is an extensive FAQ and how-to guide that addressed the objections I immediately came up with. From the article: "It has been observed that when a domain has both a primary (high priority, low number) and a secondary (low priority, high number) MX record configured in DNS, overall SMTP connections will decrease when the primary MX is unavailable. This decrease is unexpected because RFC 2821 (Simple Mail Transfer Protocol) specifies that a client MUST try and retry each MX address in order, and SHOULD try at least two addresses. It turns out that nearly all violators of this specification exist for the purpose of sending spam or viruses. Nolisting takes advantage of this behavior by configuring a domain's primary MX record to use an IP address that does not have an active service listening on SMTP port 25. RFC-compliant clients will retry delivery to the secondary MX, which is configured to serve the role normally performed by the primary MX)."
Re:Oblig. (Score:5, Interesting)
> (x) It will stop spam for two weeks and then we'll be stuck with it
There is another anti spam technology called (doubleverify?), if a message smells like spam the smtp server rejects it saying unavailable and waits for the sender to send it again (an hour or so later). For people who use it it works fine, but people who use it are in the minority, thus spammers won't bother writing new systems that keep track of what was rejected etc. They appeal to the (cheap) masses.
Same here, unless this becomes widely popular few spammers will adopt it. Thus there's a chance for this to work (hopefully, unlike doubleverify this is not patented)
Re:Temporary Solution (Score:4, Interesting)
I used to received about 6 million spams a day across 3 relays for this domain.
I removed all MX records for the domain, and the hostnames have nothing to do with the domain (so A record lookups won't help), but 30 days later I still was receiving over 2 million spams a day. After about 6 months the number really started falling off.
Re:hmmm (Score:2, Interesting)
as a spammer writing your own SMTP engine, why wouldn't you just write in basic queue management into your client to get around nolisting/greylisting/nastyhacklisting...?
I run a high volume mailserver, this is a bad idea (Score:5, Interesting)
We have thousands of domains pointed to our mail servers and secondary MX servers. Looking at the long run stats, I'd be tempted to completely disregard this technique.
When we take a primary down for maintenance, the secondaries and alternate primaries (same weight MX) see the load almost immediately.
I second the opinion that if this has any effect, it's only for low volume applications, with few/one domain.
We generally see more hits straight to the secondaries by spammers hoping for less rigorous checking. It would be interesting to profile IPs connecting to secondaries without being seen at the primary assuming a primary is always available - I bet that a very high percentage of these connections to secondaries could be viewed as spam.
The problem remains that most tricks of this sort - including greylisting - are eventually circumvented by spammers once the trick gains critical mass. Lets not forget that there are a lot of broken, yet not open relay, mail servers out there. Good engineers and administrators quickly find that Jon Postel's words ring true with their customers "Be liberal in what you accept, and conservative in what you send." - don't let your RFC enforcing configuration be responsible for delaying/blocking the delivery of that big contract your PHB was waiting for!
Address Book (Score:3, Interesting)
One for mail addressed to someone in your mailbox.
One for everyone else.
90% of my spam problem would be solved by this simple recipe.
They will respond (Score:4, Interesting)
Many people were already using this trick, probably hoping it wouldn't show up as lead story on slashdot.
In some ways, selfish ways, it's like the story of the two hikers who face a bear. The first hiker immediately sits down and starts putting on his running shoes. The other says, "What are you doing? You can't outrun the bear!" The first hiker says, "I don't have to outrun the bear. I just have to outrun you."
Many spammers, faced with a failed attempt at sending mail, do not bother to retry or try other MX. Instead, they just move on to the next target in the list, since trying a new target is just as easy as retrying an old target. No real difference to them. But it means you just push your spam attempts onto other people who haven't elected to bend the standards to divert the spammers.
The "good" spam sending programs run many threads, timeouts don't punish them, their limit is more the bandwidth. Attempts to divert spammers onto others who have not tried the tricks should create an ethical question. Are we just arranging for the bear to eat our friend?
Their customers are the ones at fault here. (Score:2, Interesting)
A number of recent studies have shown that most of the major purchasers of goods advertised via spam are from the United States. One particular report offered statistics showing that most spam-advertised goods were bought by people in the Oklahoma, Arkansas, Mississippi, Alabama, Tennessee and Missouri region of the US. Another major area for the purchasers of spam-advertised items was London, England.
If anyone is responsible for spam, it is all the people who actively go forth and continually buy the items that are advertised via email spam.
Re:That's "greylisting". (Score:5, Interesting)
I sincerely doubt that most of them would ever try more than the primary MX when delivering mail either.
Non-complience with the standards by email handling programs just makes it easier for the spammers by taking away a postmasters anti-spam tools
Re:Temporary Solution (Score:3, Interesting)
It's not hard to think that spammers are probably keeping lists of IP addresses rather than DNS names. They don't care about correctness, so there is no need for them to try the correct SMTP server. Therefore, why bother with the overhead of DNS? Or at least, why do the lookup more than once every month or so, especially when IP addresses of mail servers tend to be pretty stable. (You might even call them "static".)
Because spammers may be directly targeting an IP address, one other possible way to fight spam is to change the IP address of your SMTP server regularly. If you change the MX records (well, really the A records they point to), legitimate traffic will pick up the changes. To be safe, you can continue to listen on the old IP address for a week or so while you make the transition to the new IP address. That ought to give stale DNS entries plenty of time to expire.
And, of course, you keep rotating, so that out of, say, 254 possible addresses, you're only using each one for maybe 1% of the time. The other addresses are, of course, not responding to any TCP packets received on port 25.
All this will achieve in the long term is force spammers to use DNS and/or carefully prune their list of IP addresses they try to send spam directly to. Well, that and any message sent to an IP address that hasn't been current for, say, 1 month is a message that is a very strong candidate for being sent to an RBL.
It's not a huge win, and the spammers will adapt, but until someone figure out some idea which is a huge win, there is some value in continuously forcing spammers to adapt. It makes spamming less easy.
Spammers and MX records (Score:1, Interesting)
Re:Address Book (Score:5, Interesting)
Flowchart:
Re:What's with the breakage to fight spam? (Score:3, Interesting)
Because spam has broken the infrastructure. A working broken solution is better than a fully broken solution.
I now use my work e-mail and nothing else. Mail from outside lands in the junk folder as low priority stuff to be sifted later.
My home private e-mail hasn't been checked since October. It's been hammered to the point of being useless. I've gone to reach me by pager, phone, or business radio.
I no longer spend 20 minutes a day sorting spam. My mailbox is 12 years old. It's on way too many spam lists. The backlog is so deep, I don't bother looking.
Some people are looking for a working solution to the tidal wave of spam. Some get a new address every 6-12 months. Others have gone to IM. Some have given up private ISP provided mail entirely.
Nolisting + Port Knocking? (Score:4, Interesting)
Instead of blocking the connection to the primary at a firewall or using an "unused" IP address, the primary SMTP server could give a greeting banner and then immediately return a "temporarily unavailable" status code (and cache who was connecting there).
In other words, an RFC compliant MTA should be connecting to the higher priority host as defined by DNS first, then fail over to the lower priorty host, in order. If an MTA tried to connect directly to the secondary MX first it could be rejected with a temporary failure status code which a spammer is likely to ignore. It would require the SMTP receiver to keep a cache of who had connected to what IP addresses within a certain time period which would eat up some memory depending on traffic load. We already cache reverse DNS lookups and RBL lookups, so it could probably be done.
With this setup you would have two MX records for your primary mail server that your SMTP server would be active and listen on. It would just track the order of connections to ensure that the remote MTA was following the rules before it allowed the source to get past the greeting banner.
Re:That's "greylisting". (Score:5, Interesting)
From the link...
--snip log example--
This spammer got stuck for 47 minutes. Current spamd sets its socket receive buffer size to one character, forcing the sender to send one TCP packet for each byte of data, even if its a non-compliant "dump and disconnect" mailer. Of course, the spammer nearly immediately tries to retransmit the spam. Repeatedly.
Re:Oblig. (Score:4, Interesting)
First, at least some botnets will hit secondary MX-es first. The reason for this is because one person too many out there think that the secondary MX gets invoked only when the first one fails and do not put full sets of antispam software on it.
Second, as far as detecting SPAM is concerned the fact that a system has tried your first MX is valuable information. So while the first MX may not accept the message it should still be available to record the attempt. As a result, if you have multiple level different priority MX-es you can vastly improve on standard greylisting. The first MX resets with the usual "greylisted for 300 seconds, come again". After that system expects that you appear on the second, third, etc in the correct order and try on all MX-es of equal value before going up. In other words your connection pattern should follow the one of a normal MTA. Zombie writers are too lazy to do that (and that takes too much resources as far as they are concerned) so they fail the test and get their greylist timeout pushed up. Normal MTAs get their greylist timeout adjusted down and may even be allowed in on one of the last MX-es. I have done that using exim/mysql and I know a few other people who do that as well (trivial actually). In fact, looking at my mail logs it looks like yahoo does something similar for receiving mail and I can bet that they are not the only ones.
Re:Oblig. (Score:3, Interesting)
Most e-mail servers resend within 15 minutes (usually like 5-10), so it doesn't cause for much delay. Besides, once an e-mail made it through, we would simply allow all future emails from the same sender to the same recipient for up to 7 days from the last successful mail. Thus, if you frequently e-mailed someone, the greylist was completely transparent to you.
It really is quite a successful method, but only until spammers start resending messages.
Re:Oblig. (Score:2, Interesting)
Another, related, problem is when the secondary mail server belongs to your ISP. Spammers will target this, making the (almost certainly correct) assumptions that:
i. The ISP will have less rigorous Spam checking.
ii. You won't block SMTP connections from your ISP's mail server.
In the end these factors actually lead to more certain ways of detecting, and thus blocking, certain Spam.
Zero Spam is easy... (Score:4, Interesting)
*If* you are serious about getting rid of the spam then just do it. The technical part is readily available.
I deployed that almost a year ago and never looked back. I still see the occassional spam in a
mailing list folder because those go through unfiltered for obvious reasons but I couldn't care less.
My inbox has been spam-free since then and that's what matters.
I don't quite get why people are still bothering with greylisting, spamassassin, razor, dcc, bayes and
the ilk. I tried them all and they're more trouble than it's worth. You get false positives, false negatives,
it's a stupid game that you can't win.
Re:Oblig. (Score:2, Interesting)
Except that most ISPs nowadays block SMTP to anything but the most expensive (full class C or even higher) connections, and put their own SMTP server in between. In that case, rejecting the mail means that the receiving SMTP returns an error code to the intermediate (ISP) SMTP server, which will then send a bounce mail to the person whose address was being spoofed.
Re:Oblig. (Score:3, Interesting)
I do have the numbers to back this up... check out the stats at slowspam.com [slowspam.com] - this exploits the fact that some spammers target low priority MX hosts and then holds them in a tar pit for as long as they keep the connection open - 671 hours in one case.
More of an explanation here [blogspot.com].
Re:Oblig. (Score:3, Interesting)
Read them both. While the statistics are correct (mine roughly the same), the technical bit is typical "I shall use naked Postfix or die" technological rococo (not to use harsher words).
I am aware that implementing a generic expandable grey/black/integrity-listing framework is much more difficult in "naked" Postfix compared to Exim and Sendmail, but it is not that difficult. Postfix has a policy server and it mostly works. In fact I know quite a few people who have taken my grey/black/connection-sequence stuff for Exim and have ported it to the Postfix policy server in less than a day or so (with testing included).
As far as Unlisting that is even more rococo and looks hideously ugly. It is of course a matter of taste, but I would rather use a database behind the MX-es to exchange state data and do it properly. It is NOT more complex. In fact it is less complex and much more reliable. 5-10 lines worth of Exim config or 5-10 lines worth of Milter perl code.
This doesn't work (Score:3, Interesting)
Re:Temporary Solution (Score:3, Interesting)
I've got a similar story. When a good local ISP got bought up by a crappy CLEC who ran it into the ground, I switched over to the ILEC's DSL offering. However, they never closed my e-mail account, so I kept reading from it. After a while they switched their authentication so that I had to log in as "user@domain.net" instead of just my user name, but it still accepted my password.
Naturally, all I got was spam on that account. But then the CLEC dropped the old domain name, which got snatched up by an ISP in New Zealand. So now there were no MX or A records pointing to that mail server any more under the old domain name. The only way to send mail there was with a "%" hack ("user%domain.net@newdomain.com"). Yet the spam still kept coming in. It must have been at least two years more before it finally wouldn't let me log in any more, and there was still a ton of spam coming in daily.
It does make me wonder if the New Zealand ISP got a lower than normal amount of spam during that time.
Re:Oblig. (Score:2, Interesting)
I'm really surprised he hasn't mentioned the other obvious thing to do.
Some spamming software is 'clever' and tries only the last MX record. Some is not and tries on the first MX record.
What I did: Three MX records. Mail server actually listens on the middle one.
And even if they try the secondary first, even using his scheme unmodified won't add any spam. It's not like they were originally looking up the domain, saying 'There's only one MX record, I guess I won't send them any spam.'
Re:This doesn't work (Score:3, Interesting)
That's what I thought, too. But then I thought, 'make your highest AND your lowest priority servers dummies.'
Re:This doesn't work (Score:3, Interesting)
Teergrubing your Third MX Record? (Score:3, Interesting)