Monday 2 May 2005

Why Challenge/Response is bad

Challenge/response (C/R) is disliked by users and legitimate bulk mailers alike. Unfortunately, anti-spam technologists who should know better keep re-inventing it.

The most recent example that I've come across is SquareAnswer. Whenever I hear about a new anti-spam vendor with "secret," "revolutionary," "patent-pending" technology, that suffer "zero false positives," I roll my eyes and prepare for yet another C/R product.

What is it? Briefly, if a C/R recipient is sent email "from" a sender that it's never heard of before, it auto-replies with a challenge. Until the sender has satisfactorily responded to the challenge, their mail doesn't get through to the recipient's inbox.

Although possibly useful in some environments, it's basically a terrible idea. It's generally worse than today's state of the art spam filters, which use techniques such as Bayesian filtering, heuristics, and "out of band" connection data analysis. Here's why...

  1. Users hate receiving challenges; especially if their email address has been forged by a spammer and they've never even heard of the person it came from, let alone emailed them. A significant number of people just don't respond to challenges, which means that the false positive problem is worse than with conventional filtering.
  2. Legitimate mailers hate it because they can't deal with the flood of challenges when they send out newsletters. Again, the false positive (or "deliverability") problem is worse. Much worse, in this case.
  3. C/R shifts the cost of spam from recipients to the senders of legitimate mail. How dare you make me prove that I am who I say I am? I've already published an unambiguous SPF record that says that my IP address is permitted to send email from my domain; what more do you want? We won't win the war against spam until the costs are shifted to the spammers.
  4. Users who employ C/R are seen by some as spammers in their own right. It's part of the phenomenon known as "backscatter." Imagine if your email address was used by spammers to forge the "sender" of their pill-pushing messages. You would expect to receive many non-delivery reports from mailboxes that no longer exist, "we don't want your spam" bounces from badly-configures spam filters, and challenges from people running C/R systems. How is this better than the spam we're trying to kill?
  5. If you run a C/R system, you are likely to be blacklisted for spamming, and your ISP will receive abuse complaints about you. You may even lose your connectivity as a penalty for violating your ISP's Terms Of Service or Acceptable Use Policy.
Vendors: enough with the C/R reinvention already!

Users and IT managers: don't buy it. There are much better ways to filter spam without the problems that C/R will cause you.

Categories: , , .


Anonymous said...

Challenge/Response is bad? No it is the ONLY system available today that actually works! Don't tell me filters work, spammers are morphing just as fast as filters. My main business email was getting over 200 spams a day. I modified my filters daily, and oh yeah I had to continually check my trash bin anyhow to catch things that got filtered out that shouldn't have. I was wasting over an hour a day dealing with spam, then I switched to a challenge/response system. Almost instantly, problem solved. I have had a wonderful experience, and most of your reasons why C/R is bad are full of holes.
1. Users hate receiving challenges: Legitimate users have to answer a challenge AT MOST one time. Not much of a burden, and most users never get a challenge because I've already added them to my white list. All my normal correspondents never have seen a challenge from me. The only people who get a challenge are people who are contacting me who I've had no prior contact with. The challenge only asks them to prove that they are human, and not some spambot. I have never had anyone complain when I explain the magnitude of the spam problem I've got.
2. Legitimate emailers hate it: A challenge is much better then being routed to a black hole, like so many of your wonderful filters do. At worst you only get back as many challenges as you send out newsletters, and in reality it will be MUCH smaller amount.
3. C/R shifts the cost of spam to the SENDER. Funny, that's just how normal postal mail works. Filters leave the cost entirely with the recipient. How is that better?
4. C/R makes you a spammer? Huh??? I am only sending out as much as I get that is of questionable legitimacy. If 50-60 challenges a day (my average load) makes me a spammer, what does that make a "legitimate mailer?" You can't have it both ways. My challenges are legitimate mail. And as for spoofing of email addresses, I want to know if someone is using my address illigitimately. Yeah it would be a pain if everyone in the world used C/R, but with folks like you around that's not gonna happen. It would also be an excellent motivator to get people to change the currently easily spoofed system.

C/R works. Period. Nothing else currently available does. It is no different than using caller ID to decide whether you will pick up the phone. Marketers don't like it because it leaves the decision of whether to receive a message with the recipient, where it belongs.

Richi Jennings said...

Well, it's point of view.

Simple filters don't work, as you've discovered. That's not a good argument for replacing them with somthing that shifts the burden onto other parties--possibly innocent ones.

Today's state of the art spam filters can easily achieve 95% effectiveness, with very small false positives. I would wager substantial amounts of money that you're getting a much higher false positive rate.

There are also two main points I want to clarify, as a result of your comment:

3. C/R shifts the cost of spam to the sender. It would be better if an anti-spam filter REMOVED the cost altogether.

Note also that the "sender" is probably forged, so innocent people receive challenges. This leads to point 4...

4. People who use C/R are spammers. You are sending challenges to forged email addresses. i.e. they are innocent 3rd parties who have to receive your challenge and decide what to do with it. If everybody did this, email would become completely unusable.

It's a "tragedy of the commons" argument. Saying "it works for me" implies "everyone else should do it" -- if they did, you'd regret saying that!


Anonymous said...

Alot of what Dave Bennett said, I agree with. Richi basically assumes that C/R users are maliciously incompetant and fails to consider that like a lot of other technology, whether C/R technology works depends on what you want out of it. One should start by assuming that users will be reasonably conscientious about maintaining their whitelists when they giving out their email address. Users hate recieving challenges? Which users? From the above it wouldn't be those I gave my email address to. So it would have to be people who want to email me without having me know them first. And they can't be bothered to take the time to verify that they're not some POS spammer? Cry me a river. In that case, C/R has done it's job perfectly in that I don't want that email. Legitimate bulk emailers hate it? Is their a legitimate bulk emailer whose mail I shouldn't expect beforehand? Not in my opinion. If C/R can get most bulk emailers to join us in 2006 and adopt blogging technology, that alone would make it a wonderful thing no matter what else it does.

C/R is useful for personal email where one tends not to receive email from new people frequently. It's use is to prevent spam from businesses who think they can email you because they bought your personal info from an f-ing credit bureau or something. No it shouldn't be used for every email account, but no one is suggesting that. No it won't stop Viagra spammers from spoofing headers but no one is suggesting that either. It's part of a balanced breakfast: PGP/ S-Mime / possibly Domain Keys to prevent spoofing, server and client spam filters, and possibly C/R depending on the account.

And BTW Ip based spoof prevention sucks because it can break mail forwarding if just one server doesn't do it right or isn't on board with CID. The best way to do it is with either PGP or SMime.

Post a Comment