Stopping Spambots: A Spambot Trap 312
Neil Gunton writes "Having been hit by a load of spambots on my community site, I decided to write a Spambot Trap which uses Linux, Apache, mod_perl, MySQL, ipchains and Embperl to quickly block spambots that fall into the trap. "
Re:Elements of good design I'd missed (Score:5, Interesting)
Block? Are you kidding? (Score:5, Interesting)
No way, man.
If you realize you're serving to a bot, go on serving. Each time the bot follows the "next page" link, you
Give it thousands, millions of addresses this way.
How I track spammers using PHP (Score:5, Interesting)
As it turns out, I really haven't received that much mail to this address. About the only mail I've ever received to it is someone from trafficmagnet.net, who tells me that I'm not listed on a few search engines and that I can pay them to have my site listed. I need to send her a nasty reply saying that I don't care about being listed on Bob's Pay-Per-Click Search Engine, and that if she had actually read the page, she would have noticed that she was sending mail to an invalid address. Besides, the web server is for my inline skate club and we don't have a $10/month budget to pay for search engine placement.
I think I've received more spam from my Usenet posting history, from my other web site, and from my WHOIS registrations than I've received from the skate club web site.
re: spidertrap (Score:4, Interesting)
removing mailto: a bad solution (Score:5, Interesting)
Removing mailto: links is a bad solution to the problem. It might be the only solution, but it is bad.
I hate the editor in my web browser. No spell check (and a quick read of this message will prove who diasterious that is to me), not good editing ability, and other problems. By contrast my email client has an excellent editor, and a spell checker. Let me pull up a real mail client when I want to send email, please!
In addition, I want people to contact me, and not everyone is computer literate. I hang out in antique iron groups, I expect people there to be up on the latest in hot tube ignition technology, not computer technology. To many of them computers are just a tool, and they don't have time to learn all the tricks to make it work, they just learn enough to make it do what they want, and then ignore the rest. Clicking on a mailto: link is easy and does the right thing. Opening up a mail client, and typing in some address is error prone at best.
Removing mailto: links might be the only solution, but I hope not. So I make sure to regualrly use spamcop [spamcop.org].
Re:Block? Are you kidding? (Score:5, Interesting)
Similar to how the new ORBZ works? (Score:4, Interesting)
Removing the Mailto: may not be the best plan.. (Score:5, Interesting)
<script>document.write("<A CLASS=\"link\" HREF=\"mailto: " + "myname" + String.FromCharCode(64) + "mydomain"</script>
Seems to work fine. Anyone know of any reason it shouldn't, or have any other way to keep down spam without totally removing the Mailto: ? I know this won't work with *every* browser, but it beats totally removing mail links. And I don't think spammers can get it without having a human actually look at the page...
Comment removed (Score:2, Interesting)
Similar setup without SQL requirements (Score:4, Interesting)
Setup details at http://www.bero.org/NoSpam/isp.php [bero.org]
Re:Block? Are you kidding? (Score:4, Interesting)
Zero resources, except for memory.
A much better solution would be to point the bot at a set of "servers" with IP addresses where you're running a stateless tarpit.
Re:Similar to how the new ORBZ works? (Score:4, Interesting)
This is the same method I have been using for a while. I have an e-mail account called "cannedham" that I had posted on several web sites as a mailto: anchor on a 1x1 pixel graphic. Any e-mail sent to that address updates my Postfix header_checks file to protect the rest of my accounts. It works like a charm.
Re:Block? Are you kidding? (Score:3, Interesting)
The bots don't fall for it anymore. Some dorks in Washington state decided to make a couple requests a second to it once, but in the two years I've had it up, they're the only ones.
burp (Score:1, Interesting)
None of that Perl nonsense, either. All in pure C on a BSD host, with a damn good attention to potential overflows. That was also the site which had my own custom MTA (I only knew sendmail, so it seemed a wise decision), demanded full W3C compliance (we would test it on about 10 platforms), and got used as evidence in the DoJ case against Microsoft.
Sigh, those were the days. Now, all I see is rehashing of old ideas. So, I view this news is 6 years old -- perhaps even a record for Slashdot?
Re:Block? Are you kidding? (Score:5, Interesting)
I agree. And, come on, how much technology do you need?
This is my solution to stopping spambots. It's in a JavaServlet technology and I am posting it here to prevent my company's site from being slashdotted. It does not prevent the spammer from harvesting emails it just slows them down.... a lot :) If everyone had a script like this, spambots would be unusable.
Feel free to use the code in anyway you please (LGPL like and stuff)
Put robots.txt in your root folder. Content:
User-agent: *Disallow:
Put StopSpammersServlet.java in WEB-INF/classes/com/parsek/util:
package com.parsek.util;import java.io.File;
import java.io.StringWriter;
import javax.servlet.ServletContext;
import java.net.URL;
import java.util.Enumeration;
import java.lang.reflect.Array;
public class StopSpammersServlet extends javax.servlet.http.HttpServlet {
private static String[] names = { "root", "webmaster", "postmaster", "abuse", "abuse", "abuse", "bill", "john", "jane", "richard", "billy", "mike", "michelle", "george", "michael", "britney" };
private static String[] lasts = { "gates", "crystal", "fonda", "gere", "crystal", "scheffield", "douglas", "spears", "greene", "walker", "bush", "harisson" };
private String[] endns = new String[7];
private static long getNumberOfShashes(String path) {
int i = 1;
java.util.StringTokenizer st = new java.util.StringTokenizer(path, "/");
while(st.hasMoreTokens()) { i++; st.nextToken(); }
return(i);
}
public void doGet (javax.servlet.http.HttpServletRequest request,
javax.servlet.http.HttpServletResponse response)
throws javax.servlet.ServletException, java.io.IOException {
response.setContentType("text/html; charset=UTF-8");
java.io.PrintWriter out = response.getWriter();
try {
ServletContext servletContext = getServletContext();
endns[0] = "localhost";
endns[1] = "127.0.0.1";
endns[2] = "2130706433";
endns[3] = "fbi.gov";
endns[4] = "whitehouse.gov";
endns[5] = request.getRemoteAddr();
endns[6] = request.getRemoteHost();
String query = request.getQueryString();
String path = request.getPathInfo();
out.println("<html>");
out.println("<head>");
out.println("<title>Members area</title>");
out.println("</head>");
out.println("<body>");
out.println("<p>Hello random visitor. There is a big chance you are a robot collecting mail addresses and have no place being here.");
out.println("Therefore you will get some random generated email addresses and some random links to follow endlessly.</p>");
out.println("<p>Please be aware that your IP has been logged and will be reported to proper authorities if required.</p>");
out.println("<p>Also note that browsing through the tree will get slower and slower and gradually stop you from spidering other sites.</p>");
response.flushBuffer();
long sleepTime = (long) Math.pow(3, getNumberOfShashes(path));
do {
String name = names[ (int) (Math.random() * Array.getLength(names)) ];
String last = lasts[ (int) (Math.random() * Array.getLength(lasts)) ];
String endn = endns[ (int) (Math.random() * Array.getLength(endns)) ];
String email= "";
double a = Math.random() * 15;
if(a if(a if(a if(a if(a if(a if(a if(a if(a if(a if(a if(a if(a email = email + "@" + endn;
out.print("<a href=\"mailto:" + email + "\">" + email + "</a><br>");
response.flushBuffer();
Thread.sleep(sleepTime);
} while (Math.random()
out.print("<br>");
do {
int a = (int) (Math.random() * 1000);
out.print("<a href=\"" + a + "/\">" + a + "</a> ");
Thread.sleep(sleepTime);
response.flushBuffer();
} while (Math.random() out.println("</body>");
out.println("</html>");
} catch (Exception e) {
out.write("<pre>");
out.write(e.getMessage());
e.printStackTrace(out);
out.write("</pre>");
}
out.close();
}
}
Put this in your WEB-INF/web.xml
<servlet><servlet-name>stopSpammers</servlet-name& gt;
<servlet-class>com.parsek.util.StopSpammersS ervlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>stopSpammers</servlet-name& gt;
<url-pattern>/members/*</url-pattern>
</servlet-mapping>
Here you go. No PHP, no APache, no mySQL, no Perl, just one servlet container.
Ciao
Take this one step further... (Score:4, Interesting)
You'd have a standardized spambot trap (like the one described in the article) on various webservers. The new spambot info could go into a "New SpamBots" database (which wouldn't be blocked). Once a day, the webserver would connect up with a central database and submit the new spambot info it's obtained. Then the server would download a mirror of the updated "SpamBots" database which it would use to block spambots.
The centralized SpamBots database would take all of the new SpamBot info every day and analyze them in some manner as to detect abuse of the system (ensuring that only true spambots are entered). E-mails could be fired off to the abuse/postmaster/webmaster for the offending IP address. Finally, the new SpamBot info would be integrated into the regular SpamBot database.
This way you'd be able to quickly limit the effectiveness of the Spambot-traps across many websites.
Attn Spambot Authors (Score:5, Interesting)
Thanks again for your interest. I hope that we were able to help you write the spambots of the future that will be able to detect and sidestep as many of the above protection schemes as possible. We tried to work all of our knowledge into one convienient thread for your development team to peruse.
Thanks for your interest in SlashDot, home of too much information.
Re:Removing the Mailto: may not be the best plan.. (Score:4, Interesting)
<img src="myemailaddress.jpg" alt="me at domain dot com">
that way people who use browsers that speak (ie. the blind) would still hear your address correctly, so long as spambots don't start to pick up on the spelling out of "at" and "dot".
Re:Elements of good design I'd missed - P.Solution (Score:2, Interesting)
As described at http://www.joemaller.com/js-mailer.shtml [joemaller.com] you can combine JavaScript and Images to protect your mail. Made very good expiriences with this one....
But, as stated on the Website: this game is an arms race...
What I use (Score:3, Interesting)
<A HREF="mailto:hosting%40slickhosting.com"
onMouse
onMouseOut="window.status='';">hostingsli
<!-- Spam trap
abuse@ (your domain) HREF="mailto:abuse@ (your domain) "
root@ (your domain) HREF="mailto:root@ (your domain) "
postmaster@ (your domain) HREF="mailto:postmaster@ (your domain) "
uce@ftc.gov HREF="mailto:uce@ftc.gov"
-->
Re:Block? Are you kidding? (Score:3, Interesting)
Good idea but, I'm sure spam software has been rejecting 127.0.0.1 for many years.
How about a few people volunteering real FQDNs that all resolve to 127.0.0.1? I realize that people would be volunteering horsepower and bandwidth for DNS lookups, but it would be in the name of dramatically reducing spam. Then, keep a list of all the "loopback FQDN's" and let the rest of us feed those FQDN's into spam-trap generators. Eventually, there would be so many real-looking spam trap email addresses that the spam software wouldn't be able to keep up with the list of loopback FQDN's.
To take it to the next level, you could hide the list of "loopback FQDN's" by making a reverse DNS lookup against a couple of volunteered IP addresses return a random FQDN from the list of loopback FQDN's at the time that the spamtrap page is dynamically generated.
Spammers would never know the entire list of FQDN's that resolve to loopback.
Don't stop spambots, feed them with Sugarplum (Score:3, Interesting)
Better yet, use a Spam Troll-box (Score:2, Interesting)
A troll-box gives Spam-bots a place to send their spam. When this box intercepts the spam, it reports it to the Vipul's Razor network, and everyone else on this network becomes aware of that spam (if they are also using Vipul's Razor to filter, which, chances are they are, it will filter that spam if they get it).
If Vipul's Razor isn't enough, one can even use something like SpamAssassin [taint.org] in conjunction with Vipul's Razor to get even better results.
Of course, this isn't cutting off Spam-bots at their source... but if enough sites were to cut them off at their source, then I'd imagine the Spam-bot authors would get wise to this and devise a way around it. Whereas with something like a SPam Troll-box, the Spam-bots seem to still be working to those running the Spam bots
Let's feed the serpent its own tail (Score:2, Interesting)
Well, I didn't trust (1), and (3) just got me a voice mail box instead of a person I could chew out, which I didn't use. That left (2), and I had a wicked idea:
I hit 2, and input the number that I should call if I was interested in the fax (which appeared in BIG text right above the little text). Their own response number should start eventually getting faxes from them or, as I tend to experience, hangups.
Cute story, I know, but what does this have to do with defeating spambots?
I went to the page indicated...
And I scrolled to the bottom, and looked at the source code, and noted two faaaaaascinating things:
First, the HTML on that page is rather clean; I can see no evidence of anti-spambot code on their page.
And second, the "Contact Us" link at the bottom is a mailto:.
By all appearances, their page is vulnerable to their own spambot.
So I had the thought... what if those generated-random-email-address pages were geared to produce not-so-random email addresses? What if the email addresses on those generated-page traps were geared to generate random email addresses at the domains of the various spambot-- (err, I mean) harvester producing companies? Let them see what it's like when less than discerning spammers use their software for evil. Hundreds of Viagra-substitutes! Thousands of hangover cures! Tens of thousands of opportunities to refinance their home mortgage!
This is just an off-the-top-of-my-head idea. Opinions?
What about a Terms of Service page (Score:2, Interesting)
The page could have a form with "Accept TOS" and "Reject TOS" buttons. I wonder how many spambots would submit a form?
And to catch spambots that did submit the form, your TOS could have some clauses that make it a violation for evil spiders (ones that don't honor "robots.txt") to use the site. Maybe you could make||lose a few bucks suing the spambotters who go through the TOS and still harvest your email addresses.
New Program - Mailwasher (Score:4, Interesting)
Anyway, AFAIK, it's WinBlows only, and available at http://www.mailwasher.com [mailwasher.com], although right now it seems the site is down, all I get is a 404!
Re:A better solution: obfuscate the mailto: link (Score:5, Interesting)
Some spambots will render that correctly. Less likely, though, is if they'll render an email that has had this [jracademy.com] done to it: it's encrypted through javascript.
It is a rather impressive piece of work. Uses honest-to-god RSA.
You could also encrypt all email addresses, and then in your spambot trap, put really really CPU intensive javascript. You'll win either way: either the spambot doesn't do javascript, and it won't get your addresses, or it does do javascript, and they've just spent an eternity wasting time. It would work the same way as a tarpit, but it wouldn't eat nearly so many resources on your end.
If you're really clever, you could have the javascript do useful work, and then have the results of that work encoded into links in the page. You could then retrieve the results when the spider follows the link.
There was an idea called hashcash floating arount a while back. The idea was that an SMPT server would refuse to deliver email if the sender didn't provide a hash collsion of so many bits to some given value. The sender has to expend way assymetrically more resources to generate the collision than it takes the reciever to check it. That way on can impose a cost on sending a lot of email. It's not so much to be a burden on ordinary users, but if you need to send thousands of emails, it will add up.
How about trying this (Score:2, Interesting)
How about writing something for these spambots using a special web server that slowly responds to it's requests (sends out a small packet every 10 seconds) so it won't time out and won't consume much cpu time, and just feeds it a line or two lines of junk with each packet. Have it randomly generate a never ending supply of useless information to keep the spambot happy. While it's busy with the useless site, it's not bothering other people nor is it getting any real addresses.
Re:Spam the spammer - sales@softwareking.com (Score:1, Interesting)
Re:Removing the Mailto: may not be the best plan.. (Score:3, Interesting)
We embed this JavaScript code on each page that needs mailtos:
<script type="text/javascript" language="JavaScript1.3">
function n_mail(n_user) {
self.location = "mailto:" + n_user + "@" + "yourdomain" + "." + "com";
}
</script>
And then make email address links of this form:
<a href="javascript:n_mail('foo');">foo<!-- antispam -->@<!-- antispam -->yourdomain<!-- antispam -->.<!-- antispam -->com<!-- antispam --></a>
Our addresses even show up correctly in lynx, but are "clickable" only in JavaScript-enabled browsers.
Of course, it's probably only a matter of time before spambots can compensate for this code. A more secure approach would be to put email addresses "components" in borderless cells of tables, or as a previous poster suggested, in images.
Re:Block? Are you kidding? (Score:2, Interesting)
I always enter postmaster@warez.slashdot.org in spamforms