Catching Spam by Looking at Traffic, Not Content

AngryDad writes "HexView has proposed a method to deal with spam without scanning actual message bodies. The method is based solely on traffic analysis. They call it STP (Source Trust Prediction). A server, like a Real-time Spam Black list, collects SMTP session source and destination addresses from participating Mail Transfer Agents (MTAs) and applies statistics to identify spam-like traffic patterns. A credibility score is returned to the MTA, so it can throttle down or drop possibly unwanted traffic. While I find it questionable, the method might be useful when combined with traditional keyword analysis." What do you think? Is this snake oil, or is there something to this?

Greylisting

[Daemonstar]

This is similar to greylisting [greylisting.org] that has been around for a bit.

Greylisting is a simple method of defending electronic mail users against e-mail spam. In short, a mail transfer agent which uses greylisting will "temporarily reject" any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again to send it later, at which time the destination will accept it. If the mail is from a spammer, it will probably not be retried, however, even spam sources which re-transmit later will be more likely to be listed in DNSBLs and distributed signature systems such as Vipul's Razor. Greylisting requires little configuration and modest resources. It is designed as a complement to existing defenses against spam, and not as a replacement.

Re:sounds good to me

[Anonymous Coward]

That may be just another tool to circumvent Spam. My primary email spam filtering is Spamd @ openbsd.org/spamd. The service-based spamd is known as Spam Assassin. This is a daemonized version that was ported for Openbsd by the gods. It can be troublesome to configure if you are a first timer. But remain vigilant with google groups and documentation provided by openbsd.org and the man pages within spamd.

Re:your specific idea sounds damned good to me

[fifedrum]

I work for an email hosting company and our standard with ISP customers is they use IMAP or SMTP auth, worst case, POP before SMTP. It's amazing how much spam is blocked going from an open relay for an ISP to authenticated-only.

spambots are bad, but my biggest problem is with fraudsters, both 419ers and standard credit card fraud types.

These sleazebags cause more trouble than the bots, and it's illegal to kill them. I'm not sure why they cause more trouble, they send out less email than the bots, perhaps the scammer's email is better targetted to real people, as opposed to directory harvesting type attacks.

Anyway, definately agree with you there, smtp auth, imap or whatever, all piped through SSL or nothing at all.

Great idea, just several years late ;)

[Thorizdin]

For everyone screaming that this isn't feasible, will kill mailing lists, and other wise render effective communication via SMTP impossible you might want to consider that about a quarter of global email volume is already flowing through a system very much like what the OP describes.

Ironport (recently purchased by Cisco for $830 million US) has been doing this kind of service for large providers for several years.
Their statistics site is publicly viewable, but using their stats requires a subscription fee.
http://www.senderbase.org/ [senderbase.org]
Its interesting to look at how well or poorly the MTA's you use are scored. All of the stats are gathered by the systems they sell to ISP's and enterprise customers. These boxes perform the spam filtering for that organization's customers and provide statistical data back to senderbase.org, which allows all Ironport customers to "know" about problems for all other Ironport customers.

The link to their PDF on their metric's is here:
http://ironport.com/pdf/ironport_wp_reputation_bas ed_control.pdf [ironport.com]

We evaluated their system last year as a possible replacement for a third party spam/virus scanning provider and may end up purchasing their equipment once everything with the Cisco purchase shakes out. Their solution, while not perfect, behaves far better than some of the things that large service providers *coughAOLcough* have tried and are (or were when we tested) comparable to most of the content based scanning systems in terms of spam filtering with a lower rate of false positives.

This technology has been around for years

[j0yc3]

IP Reputation filters are not a new idea by any stretch of the imagination.
CipherTrust TrustedSource [trustedsource.org]

Re:This is painfully obvious and hopelessly naive

[rel4x]

You sir, have no idea what you're talking about. They get paid by the sale for products, by the lead for mortgages, or a percantage for stocks. Go to bulkerforum.biz and look around.

Re:Botnets?

[gregmark]

Sending spam the old fashioned way (sans botnet) is still very effective. My company uses two throttling appliances, IronPort and Symantec 8160. Both score senders based on their spamminess and throttle appropriately. When we first turned on our 8160s last year, some people in our company thought we had eliminted spam completely. We'll be moving to the IronPort solution soon as its scoring system appears to be a great deal more thorough and reliable; we expect our spam numbers to drop even further when the go live.

Botnets make rate-limiting (which really, is all STP is, besides Stone Temple Pilots and motor oil) an imperfect solutions, but if you can eliminate the old school spammers, trust me, you will take a giant chunk out of your daily spam volume, giving your true anti-spam software more CPU cycles to do its thing, like catch that blasted image spam.

Re:I am curious...

[Phroggy]

Are any of you people still living with spam? Do we really need another solution?

Anyone who's a mail server administrator is living with more spam than you could probably imagine. During a four-week period, across two of the (very small) servers I manage, 38,728 connections were refused because of RBLs. Of the messages that were accepted, 8,102 were assigned a SpamAssassin score above 15 and sent to a system-wide quarantine folder that users never see. Another 13,619 messages were assigned a score between 5 and 15, and sent to a user-accessible quarantine folder for review. I use Rules Du Jour to keep rules from the SpamAssassin Rules Emporium updated daily, and I spend quite a bit of time writing and tweaking my own custom rules to catch spam that everything else misses.

After all of that, I STILL get about 5 per day. Bayesian filtering in my e-mail client usually catches these, but since it occasionally catches false positives, I have to check it anyway.

Nothing wrong with new ideas in the battle, but I thought that for anyone who cared it was already won.

No, the battle is already lost. We absolutely cannot keep up with the spammers if all we have are technical solutions. The only real solution is increased law enforcement. In the mean time, we need all the help we can get with technical solutions.

Re:I have a better idea:

[sheepdog43]

Funny, my BSD dedicated server was recently hacked and was spamming. It was hacked through a php/mysql exploit through a poorly written script a customer was using.

Many DDOS attacks are often carried out by Linux boxes as well, the .com I worked at had it happen to their server. Stop blaming Windows for the problems of the world. Besides if Windows did not exist, you would just have to blame something else. The most likely candidate would be Linux.

I would almost bet that your Linux or Mac box has no anti-virus protection on it, so how does that make you any better?

Re:this and other effective weapons

[macdaddy]

I'd suggest you look into Canit-Pro from Roaring Penguin [roaringpenguin.com]. It's from the author of MIMEDefang. Actually it's MD's commercial big brother. They make an appliance but I still run the app locally on Fedora boxes. They give you the full source code. It's extremely extensible. It makes Barracuda Networks' products look like child's play. Basically it will take the knowledge you already have and give you a platform to extend and build upon it. Canit-Pro is slick. The auto-tempfail by recipient and IP is great. The regex and user controls are worth their weight in gold. By far the most essential feature that is lacking in most other canned spam filters is the ability to scan incoming messages during the SMTP transaction. That way you can reject the message as spam before you actually accept it. This eliminates the need for DSNs. Give the demo a try sometime. You'll like it.