Thoughts on Network Neutrality

Update: clarified the point about IPv6 (see the comments)

Our old friend Curt Monash has a think-piece about net neutrality up on his weblog. In it, he argues for ISPs to sell a premium service tier for high-bandwidth/latency-sensitive applications:

The current Internet [can't] well support … communication-rich applications such as entertainment, gaming, telephony, telemedicine, teleteaching, or telemeetings.

A bold assertion. Certainly if t’were true the anti-neutrality camp would have a point.

I’m not 100% in either camp, but my gut tells me that today’s IP routing technology is holding up well. It’s the lack of investment in sufficient peering bandwidth and router horsepower that’s letting the side down. That and the criminally glacial progress towards RSVP-TE (RFC3209) with label switching and IPv6, a combination that would allow much better traffic prioritisation.

He later clarified his assertion:

I think that in a high fraction of applications that amount to real-time communications, good quality will entail seriously sub-second latency. I don’t think it will soon be affordable to provide that kind of QOS for all traffic. Ergo, tiering. Until we get to unmeteredly-cheap, just-like-being-there transmission of full-room-sized sounds and images, there will be a place for differentiated QOS.

Here's the thing... Those of us that live the other side of the Atlantic live with 250ms latency every day, when we connect to services hosted in North America. I dare say the same is true for those on the other side of the Pacific. There's not much getting around the speed of light.

Yet stuff still works just fine. TCP is designed to get the best out of a latent connection. Even if that latency is unpredictable/chaotic (as is often the case with latency caused by congestion).

I recently ditched my old ISP. It got taken over by a company that appear to be running into the ground. Typical latencies over the first hop grew from 20ms to 500ms (often higher). But even though there was severe network congestion, stuff still worked fine.

Of course, if an application writer makes assumptions that ignore realities such as the speed of light or temporary congestion, their application's going to behave badly. But no premium QoS in the world is going to help that.

My sense is still that the ISPs that are complaining about net neutrality are simply being greedy and don't want to invest money to cope with the growth in usage.

There are perhaps some lessons to be learned from the experience of UK ISPs when many users' connection "suddenly" jumped from fixed-rate DSL connections in the 512Kb/s - 2Mb/s range to an "up to 8Mb/s" service that offers the highest speed that the phone copper can support. How the various ISPs coped with the associated jump in demand makes for a salutary tale.

But that's the subject of another post, some other day...

Update: clarified the point about IPv6 (see the comments)

Stupid Blog Service

Sorry for all the old posts re-appearing as new ones in my feed. Looks like a news version of Blogger decided that adding a category makes an old post new again. Sigh.

More About the CEAS Spam Control Bake-off

Last week, I wrote about the CEAS 2007 Live Spam Challenge (CEAS is the Conference on Email and Anti-Spam). I opined that fair comparative testing of spam control technologies is extremely difficult, especially when behavioural analysis techniques such as greylisting and OS fingerprinting are part of the spam control technology mix.

I wanted to clarify that the test isn't intended to evaluate the relative strengths and weaknesses of existing spam control products (that would be extremely difficult to do fairly, as last week's post pointed out). The intention is to compare some promising new content-based filtering techniques -- techniques that might be employed as components in a cocktail of techniques used by a spam control product.

As Gordon Cormack, one of the test's co-organizers, wrote:

An open competition attracts all sorts of techniques that can be vetted. The methods that are uncompetitive can be discounted, and the "greatest hits" can be tested ... in combination with greylisting ... and other intrusive techniques.

...

One popular fallacy that I run into all the time is, "this test has limitations, so it shouldn't be done." All tests and experiments have limitations, and the scientific method involves identifying them and constructing specific experiments to see how much the limitations matter, not witholding all tests until the perfect one can be done (which, of course, it can never be).

Chatroom Pimping: New Spam Technique

Interesting spam from Taiwan. Just a single line containing a link to a Skype chatroom. What caught my eye was the chatroom subject embedded in the link:

Special price of the wrist-watch

Random thoughts:

• Clearly what's going on here is the need for spammers to remove as much Bayesianable text as possible from their messages.
• There probably needs to be a way that we can report these to Skype so they can quickly take 'em down. Spamcop doesn't work -- Skype is refusing abuse complaints.
  
• Presumably if you were foolish enough to join the chat, you'd get bombarded with ads. for fake Rolexes and the like. Possibly also a vector for malware.

I was foolish enough, but the host was offline. Watch this space for updates on what happens...

IDC's Spam Stats are Conservative?

Mark Levitt is the "Program VP for Collaborative Computing and the Enterprise Workplace" at IDC. His name is on a new report that includes some stats that have raised a few eyebrows.

Ars's Nate Anderson said:

New research from IDC claims that this will be the year in which spam outnumbers person-to-person e-mail for the first time.

Huh? Don't we hear from anti-spam vendors all the time that spam is 60, 70, 80, 99% of all email? Is Levitt living in a timewarp?

Well, reading between the lines of IDC's press release, it seems to me that we're comparing apples with oranges. I think Mark is including the number of legitimate messages that stay inside an organization. This is typically a whole lot more than the amount that comes in from outside. It might easily double the number of messages a user receives.

According to my latest estimates, an "average" email user (whatever one of those is) receives around 40 spam messages per day and 15 legitimate. Me? I get more like 500 spams/day, but that's including several spamtraps.

I love Brad Linder's comment:

Spam filters are a lot better than they used to be, so really what this means is that nefarious companies will continue to send messages that nobody will read this year.

Oh yeah, and the Ars story got dugg, too...

[Hat tip: Techmeme]