August 26, 2005

Concerning Spam

Updated August 29, 2005. Originally posted in 2004.

Spammers have discovered bloggers and sooner or later if you allow comments or trackback pings on your weblog you will get spammed.

Blog spam appears in many flavors:

1) Basic comment spam. The spammer leaves a short uneventful message in a comment field in one of your entries. The spam comes from the URL placed in the comments URL field. These URLs link back to every conceivable scam. The spammers leave URLs here to create a link from your site to theirs, thus increasing their Google ranking. Spammers are also now linking to legitimate sites that have not cleared their pages of comment spam, thus increasing the Google rank of those spam links. This all goes to show you that you really do need to check the links of anyone who leaves a comment on your site.

2) Comment spam flooding. The spammer uses an automated computer bot to flood your blog with comment spam messages, up to hundreds in an hour. The spammer doesn't necessarily leave a URL, but can leave garbage messages, almost like a graffiti artist. The comment spam can put a severe load on the server hosting your blog software to the point that it crashes.

3) Trackback Spam. Spammers have discovered how to take advantage of Trackback. TrackBack spam is very similar to comment spam. The spammer sends TrackBack pings to your site that direct viewers to a totally unrelated URL.

4) Referral spam. The spammer links to your site from their site, and then pings your site through their link, thus creating a reference and link to their site on the statistics referral log of your website. When you are reviewing your stats and see the reference to an odd site (ex. Paris Hilton), clicking on the link takes you to their site. Many people list "referrals" on their site publicly, so by spamming referral logs, not only does the spammer get a link on your referral log (which is picked up by Google) but may even get a link on your main page.

How do you fight spam on your blog?

Fighting Comment Spam

MT 3.2 offers much better spam fighting capabilities than MT 3.1, MT 2.661, and earlier versions. With MT3 you have the option of Typekey authentication which gives you more control over who can comment on your blog. MT 3.2 allows you to moderate Trackbacks as well.

  • SpamLookup. SpamLookup is a Movable Type plugin, developed by Brad Choate, that uses several techniques to identify spam, and then uses user-supplied choices to either moderate or block it. SpamLookup is now an integrated part of Movable Type 3.2, so if you have installed the latest version of MT, there is nothing more you need to install. SpamLookup utilizes several blacklist services to check incoming comments and trackbacks against known spammers. It allows you to either "junk" or moderate comments and trackbacks based on different settings for links and keywords. You can even "white list" domains or IP addresses. To adjust the settings on SpamLookup, simply open up your Plugins menu from the System Overview of your Movable Type editing window. Scroll to the bottom and select "Show Settings" from any of the SpamLookup modules. SpamLookup works with MT versions 3.1 and 3.2.

  • MT-Blacklist. If you are using an earlier version of MT, Jay Allen's MT-Blacklist Plugin is your first form of defense. MT-Blacklist does not work with MT 3.2. So if you have upgraded to MT 3.2 and you had Blacklist installed, you can disable it or remove it all together. Alternatively, you can try this plugin to get your BlackList plugin to work with MT3.2.

    Once installed, MT-blacklist checks comments and trackbacks against a known list of spam URLs. If a comment or trackback contains one of these URLs, the comment is blocked before it ever appears on your site. If you get a comment with spam that is not already listed, when you receive an email notification of the comment, you can click a link to invoke MT-blacklist, remove the comment, and add the commenter's URL to your blacklist. There is a master blacklist that is maintained by Jay Allen and contributed to by hundreds of MT bloggers. You can update your own blacklist with the listings from the community blacklist. You can also use MT-blacklist to screen content and block comments for use of foul language. See Jay's special instructions for dealing with trackback spam if you are using Blacklist version 1.64. The plugin is easy to install and use. If you are using MT3.1, you can get the Blacklist plugin from the Plugin Pack. An earlier version of the blacklist, MT-Blacklist v1.64, will work with MT version 2.661 and you can get it from Jay Allen's website.

  • Use a "Captcha". A captcha is a security code that a commenter must enter in order for her comment to load. The benefit is that it screens out automated comment spam bots. The downside is that it keeps visually disabled people from contributing a comment. James Seng has posted a captcha security plugin for Movable Type. Arvind has released an updated version of this plugin to work with MT 3.2 - MT-SCode 1.0.

  • Require approval before a comment posts. One way to ensure that your readers never have to see a spam message is that you personally approve comments before they are posted. If you have a low comment volume site, this may be viable option. For MT 2.661 users, Scripty goddess has posted a script/MT hack to do this. MT3 has the sophisticated comment moderation features of TypeKey built right in. (See TypeKey Authentication for Comments.)

  • Close old comments. David Raynes' plugin allows you to close entries to comments for any entries older than a given number of days (defaults to five). Other similar solutions: Conversation Killer and MT-Closure. (See Closing Comments.)

  • Force "preview" before allowing comment submissions. Forcing site visitors to preview their comments before submitting them will not only give you more error-free comments, but will put yet another hurdle up against automatic comment spam bots. Just remove this line of code:

    <input style="font-weight: bold;" type="submit" name="post" value="&nbsp;Post&nbsp;" />

    from your Individual Entry Archive and your Comment Listing Template.

Fighting Comment Spam Flood Attacks

One way that spammers can cause trouble is by repeatedly pinging your server, hundreds of times an hour, trying to leave their comment spam. This can cause server CPU overloads and crashes and can even have your web host shut down your account.

SpamLookup is an effective defense against Flood Attacks as well.

Fighting TrackBack Spam

The primary measures to fight TrackBack spam are similar to comment spam - SpamLookup and Trackback moderation, available now in MT 3.2. To moderate Trackbacks, open the settings for your weblog. Select the Feedback settings. Under Trackbacks, select, "Hold all Trackbacks for approval before they are published."

Fighting Referral Spam

Fight referral spam by ammending .htaccess file. Referral spam is annoying, but it doesn't affect the public display of your site unless you are publishing your referral log. If it bothers you enough that spam companies are benefiting by creating backlinks to their sites on your referral logs, you can ammend your .htaccess file (see What is .htaccess?) with the following lines of code:

SetEnvIfNoCase Referrer ".*(casino|gambling|poker|porn|sex|nude|xxx|hilton|pics|video).*" BadReferrer
order deny,allow
deny from env=BadReferrer

See this Killing Referrer Spam/a> article for more info on using the htaccess method to fight referral spam.

Six Apart Guide on Combatting Comment Spam
MT Support Forums thread on referrer spam
MT Support Forums thread on send mail spam
Good post on comment flooding including helpful links.
Al-Muhajaba's MT tips on comment flooding spam.
Stepping Stones to a Safer Blog - pointers to reduce crap flooding.
Referer Spam - Notes on how to fight referer spam by John's Jottings.
New Comment Spam Technique - Adam Kalsey notes that spammers are creating comment spam with links to legitimate sites that have been spammed to get the page rank up for those links.
MT-Closure - Tim Appnel's script to close comments.
ARIN WHOIS Database Search - Look up the ISP of the IP address of the person spamming you and report the spammer behavior.
MT-Blacklist Forum - take your MT-Blacklist questions to this forum, not the regular MT support forum.
Bloggers Declare War on Comment Spam, but Can They Win? - article from the USC Annenberg Online Journalism Review.
Trick the Comment Spammers - an interesting twist from MT Hacks.
Mod Rewrite method to divert spam bots to a 403 error
Killing Referrer Spam

Has this tutorial been helpful? Please consider linking to Learning Movable Type at . Thanks!

Posted by Elise Bauer on August 26, 2005 to Comments and Trackbacks, Security
| Email to a friend | Printer-friendly version


If you would like to send a trackback
please use the following URL:

» Down with comment spammers from Funtime Franky
Eagle-eyed Photo Matt has noticed that The Register has interviewed a “professional” comment spammer. Interview with a link spammer | The Register And comment spam is becoming a serious problem now, thanks to idiots like “Sam”. ......[read more]

Tracked: February 2, 2005 12:17 PM

» Anti-comment-spam tip from Flashes of Panic
I’ve mentioned this in passing once before, but it bears repeating and calling attention to. A few months ago I changed the name of my Movable Type comment script to foil comment spammers. Some of them have bots which can......[read more]

Tracked: March 12, 2005 01:07 PM

» Movable Type 3.16 from Jacques Marneweck's Blog
Movable Type 3.16 is out! I keep telling myself that I should be writing my own blog software which would do the things that I want. I think that the spamlookup plugin from the sounds of it looks great!......[read more]

Tracked: April 19, 2005 09:21 AM

Email to a friend

Email this article to:

Your email address:

Message (optional):