Saturday 9 December 2006

Spam Volumes: What's Really Going on Here?

The sky is falling! The sky is falling! Spam has doubled / spammers are winning / spam is 80% of all mail / 90% of mail / 110%, etc. etc. etc...

Yawn.

I'm getting bored with self-serving anti-spam vendors flinging dubious statistics around. Yes, spam volumes have increased recently, but doubled? Much of this seems to be counting from an artificially-small base during a quiet summer for spam.

Here's my take on what's happening. A bit stream-of-consciousness, so please excuse. Grateful for your thoughts.

The growth in spam is chiefly down to two factors:

  1. Demand-side -- stock kiting gangs wanting access to more and more sending capacity
  2. Supply side -- new, bigger botnets with more sophisticated command and control mechanisms, which are more resistant to being shut down and can send fewer messages per zombie (because they're bigger), so stay under the radar longer
This is compounded by bad statistics, which make the growth seem bigger than it actually is:
  1. New botnets spewing spam from PCs not on blacklists, so a smaller proportion of spam gets rejected (and thus never seen in quarantines)
  2. New botnets resistant to anti-spam techniques such as greylisting (because they have real, autonomous MTAs), so a smaller proportion of spam gets rejected (and thus never seen in quarantines)
  3. New botnets employing content morphing tricks that are fooling many vendors' content filters, so more spam reaches the inbox -- then naive commentators wrongly assume that a doubling of spam in the inbox equals a doubling of spam on the Internet
The image spam messages tend to be about 10x bigger than "normal" (say median 30K compared with 3K), so spam volumes are now much higher in terms of bits on the wire.

Some anti-spam vendors are coping quite adequately with the new techniques, but seem to have broken PR departments ;-)

I trust Commtouch's and MessageLabs's data more than most -- my reading is that spam volumes increased measurably about a month ago, but not to the extent that Chicken Licken would have us believe.

Thoughts?

6 comments:

Anonymous said...

I couldn't agree more. MX Logic is particulary bad about this type of bad spam PR.

In a similar vein, I can't stand the survey by similar self-interested parties where they do DNS lookups on the F500 TLD such as "ge.com" and then say that only a small % is using authentication - but in fact most F500 companies ARE using authentication just not on their TLD, but instead on mail.ge.com or gecorp.com or some other domain.

Anonymous said...

Hi Richi,
If you ask me the spammers spent their "quiet" summer vacations developing new tactics and started launching them in the Autumn. My personal ISP hosted email, that I've used for over 2 years, suddenly started getting hit in September. Most of it was Stration strains. So I guess my ISP has pretty good anti-spam, but the newer tactics are leaving them impotent. At anyrate, I just got done closing my account with that ISP. If you want any data from Commtouch's spam or virus detection, just give me a buzz. We analyze massive worldwide email traffic so we get some pretty interesting data. Peace

Anonymous said...

Well, I happen to agree with just about all of what you said here. You seem to be pretty knowledgeable about all this, so do you actually have any suggestions to "fight" spam?

Anonymous said...

I work for an anti-spam vendor (not going to say which one, but it's a very well-known name). I'm not a marketer, I'm an engineer, and I see the raw network traffic stats (and produce some of them), so I'm in a good position to comment on this post.

First, I don't consider it to have been a quiet summer. We broke a number of network traffic records during the summer months. Second, our volume really has doubled since that time.

When I say volume, I'm not referring to number of bytes, but number of messages. Message count has doubled, byte count has not. Image spams are big, yes, but they don't make up the majority of spam. Our overall spam catch rate has held up well (although image spam is problematic and difficult to filter; we have trouble with it, like everyone else).

I know as well as you do that marketing departments throw around a lot of bogus or mis-applied statistical info, but from my position on the front lines, I can tell you that the ones saying volume has doubled since the summer are accurate.

Richi Jennings said...

Hi Anonymous. Of course, I respect your right to disagree with me. Everyone sees different data. That's why I verified my data against those gathered by several other vendors, who's methodology I trust.

I also respect your desire to remain anonymous. Of course, I know exactly which vendor you work for. And I can say that yours had one of the most vocal PR departments that said "spam has doubled".

Bear in mind that your business has been growing. So one would hope that the total traffic you're seeing has grown faster than the aggregate growth of spam!

Anonymous said...

Registrars are doing their part to reduce spam quietly as well in all this, by reacting faster than ever to spam complaints.

Bill Krahmer: what you can do is to look at the full header of the spam, and then trace it to the true origin. Find the spamming domain involved, and then report it to the registrar as listed in the Whois database. Also, while you are at it, report the spamming domain's nameservers to their registrars as well.

This technique is proven effective, as you can see in many of the anti-spamming forums (http://web.tebweb.com:8080/cgi-bin/spm_forum/Blah.pl?) just to post one as an example.

Depending on the position of the registrar with regards to spam, you can have more or less success. But generally speaking this is a tried and true way for "ordinary" folks to really have an impact in reducing spam!

Post a Comment