Naive Bulk Emailers Howl in Protest

This is Andy Oram: pianist, CPSR member, and O'Reilly book editor. Andy's latest weblog post is a quiet rant about how difficult it is for new bulk email senders to navigate around a twisty maze of spam filters.

For example, he writes:

Just this morning, board members of a non-profit I volunteer for were complaining to me that email to board members gets trapped as spam
...
Ryan Bagueros ... told me lots of promising social networking companies are stymied because the emails they send members and prospective members get trapped by spam filters–especially at the major email hosting sites.

My sympathies. But there are two sides to every story.

On the other hand, some social networks behave idiotically and totally deserve to have their mail eaten.

Case in point: tagged.com, which -- let's be charitable -- was less than transparent in its description of what happens when new users signup.

Actually, no. Let's not be charitable. Let's tell it how it is. Email from Tagged.com is spam. It asks new users for the password to their [Hotmail|Yahoo|AOL|Gmail] account. Then, without warning, it spams all the addresses in their address book.

I carefully went through the signup process, using a test Gmail account. This is not a case of clueless users blindly clicking OK.

While I'm on the subject, a general point about email n00bs.

There's a pervasive naivety about what it takes to successfully send legitimate bulk email. It's not as simple as popping a default install of Sendmail onto a DSL connection someplace and expecting the whole world to be overjoyed that you're sending them mail.

Often, people don't know they need help, blindly assuming it's their "right" to have their email delivered to anyone they choose, regardless of how poorly they send it.

Two examples; there are plenty more:

1. Get your FCrDNS right. Don't know what that is? Look it up in Wikipedia. Still don't understand? You probably need help.
2. Behave correctly when presented with a greylisting tempfail. Don't know what that is? Look it up in Wikipedia. Still don't understand? You probably need help.

As I say, plenty more where those came from...

BlackSpider Acquired... Again

Less than nine months after SurfControl bought BlackSpider, it seems that Websense is buying SurfControl. Wow.

BlackSpider is an email- and Web-filtering service. Variously described as "managed", "hosted", "on demand", or "in the cloud", the BlackSpider service competes with the bigger fish such as MessageLabs, Postini, and Microsoft -- the service it acquired with FrontBridge.

In related news, BlackSpider revenues are now about £5,500,000 (US$11M at today's excruciating exchange rate). 40% growth ain't bad.

My chums at BlackSpider must be rubbing their hands with glee. James, Jeff, John, Jonathan, Kevin, et al -- the beers are on you.

PS: calling SurfControl and Websense's PR/AR teams: how come I had to read about this in the Sunday Times first? (Bizarrely, in an article about Peter Gabriel's We7.)

Thoughts on Network Neutrality

Update: clarified the point about IPv6 (see the comments)

Our old friend Curt Monash has a think-piece about net neutrality up on his weblog. In it, he argues for ISPs to sell a premium service tier for high-bandwidth/latency-sensitive applications:

The current Internet [can't] well support … communication-rich applications such as entertainment, gaming, telephony, telemedicine, teleteaching, or telemeetings.

A bold assertion. Certainly if t’were true the anti-neutrality camp would have a point.

I’m not 100% in either camp, but my gut tells me that today’s IP routing technology is holding up well. It’s the lack of investment in sufficient peering bandwidth and router horsepower that’s letting the side down. That and the criminally glacial progress towards RSVP-TE (RFC3209) with label switching and IPv6, a combination that would allow much better traffic prioritisation.

He later clarified his assertion:

I think that in a high fraction of applications that amount to real-time communications, good quality will entail seriously sub-second latency. I don’t think it will soon be affordable to provide that kind of QOS for all traffic. Ergo, tiering. Until we get to unmeteredly-cheap, just-like-being-there transmission of full-room-sized sounds and images, there will be a place for differentiated QOS.

Here's the thing... Those of us that live the other side of the Atlantic live with 250ms latency every day, when we connect to services hosted in North America. I dare say the same is true for those on the other side of the Pacific. There's not much getting around the speed of light.

Yet stuff still works just fine. TCP is designed to get the best out of a latent connection. Even if that latency is unpredictable/chaotic (as is often the case with latency caused by congestion).

I recently ditched my old ISP. It got taken over by a company that appear to be running into the ground. Typical latencies over the first hop grew from 20ms to 500ms (often higher). But even though there was severe network congestion, stuff still worked fine.

Of course, if an application writer makes assumptions that ignore realities such as the speed of light or temporary congestion, their application's going to behave badly. But no premium QoS in the world is going to help that.

My sense is still that the ISPs that are complaining about net neutrality are simply being greedy and don't want to invest money to cope with the growth in usage.

There are perhaps some lessons to be learned from the experience of UK ISPs when many users' connection "suddenly" jumped from fixed-rate DSL connections in the 512Kb/s - 2Mb/s range to an "up to 8Mb/s" service that offers the highest speed that the phone copper can support. How the various ISPs coped with the associated jump in demand makes for a salutary tale.

But that's the subject of another post, some other day...

Update: clarified the point about IPv6 (see the comments)

Stupid Blog Service

Sorry for all the old posts re-appearing as new ones in my feed. Looks like a news version of Blogger decided that adding a category makes an old post new again. Sigh.

More About the CEAS Spam Control Bake-off

Last week, I wrote about the CEAS 2007 Live Spam Challenge (CEAS is the Conference on Email and Anti-Spam). I opined that fair comparative testing of spam control technologies is extremely difficult, especially when behavioural analysis techniques such as greylisting and OS fingerprinting are part of the spam control technology mix.

I wanted to clarify that the test isn't intended to evaluate the relative strengths and weaknesses of existing spam control products (that would be extremely difficult to do fairly, as last week's post pointed out). The intention is to compare some promising new content-based filtering techniques -- techniques that might be employed as components in a cocktail of techniques used by a spam control product.

As Gordon Cormack, one of the test's co-organizers, wrote:

An open competition attracts all sorts of techniques that can be vetted. The methods that are uncompetitive can be discounted, and the "greatest hits" can be tested ... in combination with greylisting ... and other intrusive techniques.

...

One popular fallacy that I run into all the time is, "this test has limitations, so it shouldn't be done." All tests and experiments have limitations, and the scientific method involves identifying them and constructing specific experiments to see how much the limitations matter, not witholding all tests until the perfect one can be done (which, of course, it can never be).