Friday 23 December 2005

FixingEmail: CAN-SPAM is working?


The FTC is trying to convince Congress that CAN-SPAM has caused spam levels to drop. True? Well, yes and no. Let's look at the facts...

1. Spam levels are dropping? False.

The number of spam messages sent continue to rise. It's possible that spam might be leveling off as a percentage of spam, but the number of legitimate messages is rising faster.

2. But people are getting less spam, right? Irrelevant. [read more]

Tags: .

Dinner with heads of Gmail, Hotmail, and Yahoo! Mail

The WSJ's Lee Gomes talks to the heads of Gmail, Hotmail, and Yahoo! Mail. Much discussion of Ajax mail clients ensues. The Yahoo! guy (Ethan Diamond) was the Oddpost guy before they were bought out. He makes the point that their technology allows users to scroll through a big inbox very naturally -- Gmail and Hotmail still can't do that, relying on ugly "next/prev" buttons.

Similarly, the Scalix web client is the only enterprise product that does this -- Exchange's OWA can't, even in Exchange 12.

Read more here.

Predictions for 2006

Everyone else is doing it (see today's IT Blogwatch), so it's bandwagon time...

  1. More high profile lawsuits cause spammers to think again
  2. Huge shakeout in the anti-spam market, as VCs cry, "Show me the money!"—mergers, acquisitions, and failures galore
  3. Symantec (SYMC) stock price overcomes resistance at $16; slides some more
  4. Exchange 12 doesn't get released this year
  5. Companies without blogs seem like companies without websites were a few years ago
  6. Mobile operators in the US and UK will all but drop handset subsidies
  7. Someone like Linksys or D-Link sells a Media Center killer based on MythTV
  8. The Buteyko breathing technique finally recognized as a "cure" for asthma
  9. Big punch-up at the INBOX 2006 security vendor showdown
  10. Some governments offers incentives to ISPs to clean up the zombie problem
Merry Christmas-Chanukkah-Kwanza-Yule-Saturnalia-Brumalia-Solstice and a safe, happy and prosperous New Year (assuming you follow the same calendar)...

Thursday 22 December 2005

Richi'Blog review of 2005

Bah, humbug. So farewell then, 2005. We hardly knew you. Just for fun, here's a quick list of my favourite posts of the year:

January: are spam volumes really levelling off? (errr, no)

February: how to irritate users of public Wi-Fi (screw up their popstate)

March: I was on TV (well, Microsoft's Security360 webcast)

April: I am a fool

May: I hate challenge/response anti-spam (the first of a series) plus I get Slashdotted (and Nokia hates me)

June: My daily blogging column for Computerworld (IT Blogwatch)

July: The first Bastardcast (my occasional podcast of bootlegs and mashups)

August: Spam quarantines need to be better (just keep the grey area spam)

September: What is Ajax? (apart from a floor cleaner)

October: I'm the conference director for INBOX 2006

November: In defence of CAN-SPAM (mostly)

December: I laugh at eBay's anti-fraud desk ("yes, we sent that phish")

Next year, more of the same nonsense. Not only here, but at Ferris Research, Computerworld, and FixingEmail. Season's greetings, everyone!

Wednesday 21 December 2005

Spam Quarantines Should Be Sorted by Score

When spam filters decide what's spam and what's legitimate email, they often assign a score to the message. You can think of this score as the confidence that the message is spam. For example, filters based on SpamAssassin typically assign a score of more than 5.0 to indicate spam. However, spam filters can make mistakes and occasionally flag legitimate messages as spam (known as a false positive). Usually these false positives have a relatively low score.

Most spam filters maintain a quarantine or spam folder where they put the spam messages. Users or administrators can browse the quarantine folder in an attempt to find false positives.

Searching for false positives is a laborious task. It's very helpful to sort the quarantine list by the messages' score. This means that any false positives are likely to be near the top of the quarantine list. The Pareto Principle -- the "80/20 rule" -- applies. In other words, in order to get 80% of the benefit, the user only need browse the first 20% of the quarantined messages.

An example of a quarantine that does this is Electric Mail's PerimeterProtect hosted service. A surprising number of spam filter quarantines don't even allow this sort order as an option.

Tags: .

Monday 19 December 2005

mailgeek sez: Hire Richi


In case you are in the market for a good consultant in the computing field, consider the following posts...

Read more at Documenting Insanity

Wednesday 14 December 2005

Structured Blogging

Structured Blogging is a way to get more information on the web in a way that's more usable. You can enter information in this form and it'll get published on your blog like a normal entry, but it will also be published in a machine-readable format so that other services can read and understand it.

Think of structured blogging as RSS for your information. Now any kind of data - events, reviews, classified ads - can be represented in your blog.

So let's try this. Here's my hCard:
Profile photoHi, I'm Richi Jennings, of Richi Jennings Associates, where I'm the Principal. I'm based in Berkshire, United Kingdom. You can call me on +44.7789.200701 or leave a message on +1.806.993.5712. You can also email me at , or IM me using AIM/YIM/Skype.

Friday 9 December 2005

What is Mobile WiMAX?

What a lot of empty talk about Mobile WiMAX this week. If you're finding it hard to understand, here's a translation:

  • It's WiFi on steroids
  • It has great range and speed
  • When you're far from a base station, the speed degrades gracefully
  • You can easily roam from one base station to another without hiccoughs
  • Don't expect it until 2007 at the earliest

Tags: , , , , ,, , , , , .

Thursday 8 December 2005

eBay phishing saga; in summary...

Last week I noted a problem reporting a phishing email to eBay. I'm pleased to report that the phishing website -- -- is now down. However, I'm not pleased to report how long it took. The detail behind the delay is instructive...

From first report to takedown took 13 days (November 25 to December 7), which is simply unacceptable. However, despite the hilarious response from their "Trust and Safety Department," you should note that eBay wasn't the main factor in this delay. Indeed, the company claims that it first started takedown proceedings on November 8.

The main issue was that the phishing webserver was hosted on a botnet of virus-compromised PCs. The DNS entry for the web site served up a sequence of IP addresses, so that requests for the webpage could go to one of many machines. In other words, taking down "the website" wasn't an option.

Removing the DNS entry was the only practical takedown option. However, the DNS registrar for the domain --, a small company based in Switzerland -- was completely unresponsive to all requests to investigate. Finally, it seems Verisign -- the controller of the .net top-level domain stepped in and removed authority for away from Now requests for the web site come back "no such host."

This sorry saga illustrates the fact that it's important for domain registrars to act quickly and responsibly when abuses such as phishing are brought to their attention. Authorities upstream of the registrar need to be able to exercise some sort of leverage if they don't act.

Tags: , , .

Wednesday 7 December 2005

Hurrah, is no more

At last! Looks like Verisign pulled rank on

Query for type=255 class=1
net SOA (Zone of Authority)
Primary NS:
Responsible person:
refresh:1800s (30 minutes)
retry:900s (15 minutes)
expire:604800s (7 days)
minimum-ttl:900s (15 minutes)

Tags: , , .

Tuesday 6 December 2005

Digged (dugg?)

That was a hairy few days of web server load. Hello to all my new readers, anyway.

My eBay story was picked up in several places including:

Digg (on the home page for a while)
PC World
Yahoo! (was the #1 most viewed tech news item for a while)
PC Magazine (although they truncated it and I'm not mentioned)
PC World Greece
Computerworld (no link -- and this is the thanks I get for ITBW?)
Upsize This
ComputerWorld Australia
The Ross Show
Tulip Tools
CSO Magazine
PC World Australia
ComputerPartner Netherlands
InfoWorld Netherlands
Winnipeg Sun
Computerworld New Zealand
PC Advisor (UK)
Auction Info
PA Pundits

And to top it all off, my hosting provider kindly decided to take the server down for two hours of maintenance on Tuesday, without telling anyone. Great...

Tags: , , .

Saturday 3 December 2005

eBay's anti-phishing desk sucks

I reported a phishing attack last week. Nothing new there, I do it a lot as part of my ongoing research into spam for and others.

A scammer put up a fake eBay site and sent spam encouraging people to go there. Predictably, it prompted for the user's eBay username and password. Both the email and the website were very credible-looking. Nothing new there, either.

Naturally, I reported the attack to, expecting them to work with the host of the fake website to take it down quickly. Three days later, I received a reply, basically telling me I'm an idiot because this email was in fact sent by eBay.

Now, idiot I may be (and frequently am), but there's just no way the message could be legitimate. Consider the facts:

  • The email didn't include my eBay username
  • It wasn't sent to an email address that corresponds to an eBay account
  • The site puts up a signin page that's not encypted and isn't hosted at
  • The site was hosted on a consumer cable TV connection
  • The site's domain contact information -- the whois data -- was obviously forged

If eBay can't tell the difference between their own messages and phishing, how's a poor consumer supposed to know?

A week after my report, the phishing website is still active.

If I was a victim of this eBay phish, I'd be hopping mad. It's vitally important for brands like eBay to run a fast-response "takedown" service, which can accurately identify phishing and work with hosts, registrars and ISPs to remove fraudsters from the Internet.

Update: for those of you asking for more details, I'm not going to post the phishing site directly, for fear of entrapping the gullible. However, if you're determined to research it, understand that I cannot warrant that the site is malware free. Unless you agree that you take full responsibility for your actions, do not go to www(dot)ebaychristmas(dot)net.

Yesterday, the site was hosted at RoadRunner -- cpe-065-190-247-092.triad.res.rr(dot)com -- but now it's somewhere in China. Looks like it might be hosted on a botnet. The domain was registered through with a bogus email address.

Here's what eBay said:

Thank you for writing to eBay with your concern about this email. My name is [redacted], and I am happy to address your concerns. I can confirm that the message you received was an official email message sent on behalf of eBay. This message was sent because you indicated in your preferences that you wished to receive these types of messages. [followed by a description of how to check "My Messages"]
Followups to eBay and have gone unanswered.

To the folks who wanted to contact me about this, see the Contact Me page (also linked over on the right column).

This blog post is now old news. Future updates will be in newer posts, so you probably should now go to the home page

Digg this Tags: , , .

Saturday 26 November 2005

Is your Xbox 360 crashing?

Could be that your power supply is overheating (click here).

Tags: , , , .

AOL "Triton" IM client contains shovelware

AOL recently announced that its next-generation IM client, Triton, is now out of Beta and ready for downloading. The new AIM has some interesting new features and eye candy, but hides a disturbing aspect: it insists that you install a whole new web browser.

When you run the client you get an AOL Explorer web browser window, displaying the AIM Today web page. Note that AIM is simply loading a web page, but not using the user's existing preferred web browser. There are other parts of AIM that also cause this new browser to start.

AOL Explorer nags users to switch their preference, and does it in such a way that it's easy for them to accidentally grant it default browser status. Also, there's also no uninstaller!

Web browsers are complex network applications, with big potential to contain security vulnerabilities. AOL Explorer appears to be based on Firefox, but probably won't get security fixes as fast as Firefox does.

In IT shops, inadvertently installing an unknown web browser is not a good idea. Because of the potential security issues, IT managers should consider forbidding their users from installing the new AIM on corporate PCs and laptops -- users can be instructed to stick with version 5.9.

[Update: tidied up some of the language]

Thoughts about SMS text message spam

SMS text message spam is barely a problem in the US and Europe. Certainly nothing like what some doom-mongers were predicting a few years ago. Operators are incented to control it to maximize customer satisfaction -- number portability has made customer churn a potential business killer.

In Europe, the economics of sending text messages are different from those in the US -- in most countries the sender pays, not the recipient. The US is unusual in this respect -- in the US, one can usually send an SMS by emailing to an SMS gateway. This is free of charge to the sender but paid for through the receipient's plan. In most other countries, sending access to the SMS network is restricted to those who have a financial arrangement set up with the carrier -- receipt of messages is free. This destroys the indiscriminate spammer's business model.

Tags: .

Thursday 24 November 2005

Should We Publish Email Addresses?


Since posting my tips on how to frustrate spammers, I've received some questions and comments about publicizing email addresses on websites.

When I wrote How do spammers find your email address?, I said that harvesting was the main way that spammers find addresses. Harvesting is the automatic extraction of email addresses from web pages, online forum posts, and the like. Some people wrote things like, "If the #1 way of getting email addresses is to harvest them from web pages, surely the #1 way to frustrate spammers is not to publish email addresses on the web?" Well, yes, but...

No. [read more]

Tags: .

Tuesday 15 November 2005

What can we do to make it harder for spammers?


A couple of posts ago, I talked about how spammers find your email address. I promised to carry on and talk about how we can all make it harder for these low-life spammers. Here are my top four tips for killing spam: [read more]

Tags: .

Saturday 12 November 2005

Bastardcast Nov 12 - Violated Halo

Jimmi Jammes - Violated Halo

This is track 4 on the new Depeche Mode Violated compilation. A mash of DM's Halo with bleeps and breaks from other DM tracks (can you spot them all?)

I'm not familiar with Jimmi's work, so I'll be checking him out at

Categories: , , , , , , . My Odeo Channel (odeo/d3ddc2a176243581)

Thursday 10 November 2005

Which South Park character are you?

No, not some lame quiz, but a tool to design a South Park character. Here's as close as I got to my fair countenance...

Not a bad likeness. I particularly like the mooning Cartman T-shirt. I wonder if I can get Cafepress to make some?

Go to Das Planearium to build yours.

Tuesday 8 November 2005

Espion International

Interesting brace of announcements from Drew Burdsall's company recently:

  1. ESPION International, INC. Secures $1.1 Million in Funding
  2. ESPION International, Inc. Shows Strong 2005 Sales Growth

It seems that they've brokered a deal with Bellwether Venture Capital Fund I, Inc. and Bellwether Advisers, adding one Ryan D. Smith as President and board member.

Saturday 5 November 2005

Mobile Operator Subsidies will Decline

In some countries, the least expensive way to buy a mobile phone is with a contract. Even if you're Paris Hilton. When one adds up the cost of a 12 monthly payments, the phone often still costs a consumer less to buy than they could buy it without a contract. Look at the secret deals a mobile operator will often offer at the end of a contract to customers who are quite happy with their current phone. Threaten to switch operators you might get offered a contract which is effectively free for the following year.

The reason, of course, is operator subsidy. The operator sells the consumer a phone for below cost price -- sometimes even free -- in order to lock them into a 12, 18, or 24 month contract. Some 30% of the published monthly contract price is accounted for by subsidy clawback. This is a classic razor and blades business model.

Operators have huge buying power and so can source the devices for far less than a consumer would pay. Operators will also usually ensure that the phone is locked to their network, making it difficult for a consumer to switch service to another operator -- known as a SIM lock. Such subsidy business models are common in countries like the UK, Germany, and to an extent the US (at least for GSM operators). They are however illegal in some other countries.

However, this is changing. Although many consumers are happy to upgrade their phone every year -- essentially as a fashion statement -- increasing numbers of people are wising up to the way the industry works. They've realized that they can pay far less by keeping their phone and negotiating a lower tariff. This shift has been enabled by two key factors:
  1. Back-street unlockers -- the ability to remove the SIM lock, allowing the user to switch operators.
  2. Number portability -- the ability to move to another operator yet keep one's number, which is a strong bargaining chip.
In the future, subsidies will become smaller and much less common. For example, in the UK we've already seen subsidies disappear for phones without a contract (i.e. pay-as-you-go). Contract subsidies are sure to follow over the next few years.

Wednesday 2 November 2005

Inside a 419 scam


You may have heard of "419 scams." You've almost certainly received spam containing a 419 come-on.

419s are also known as Advance Fee Fraud. It gets its name from section 419 of the Nigerian criminal code. The basic idea is that someone gains your trust, promises you lots of money, but needs you to cover a few expenses first. Naturally, they take the "expenses" and disappear.

We've all made the occasional slip-up in email -- sending things we didn't mean to send to people we didn't mean to send them to. Our 419 friends are no exception. Recently, we received the following email from Nigeria. It seems to be one scammer talking to another. Enjoy! [read more]

Tags: , .



Computerworld has been sending me articles for several years, and today, I finally read one - and found - surprise - blogs! But good ones. Really, really good ones. Say Hello To Blogwatch [more]

Judi and I humbly thank you. And grin foolishly for a bit.

Tuesday 1 November 2005

More on CAN-SPAM

Yesterday, I blogged about the CAN-SPAM Act and how some people think it's a lame duck. Here are some more thoughts...

It's a myth that "unsubscribing from spam gets you more spam." It's certainly true that tests have shown that submitting new spamtrap addresses to some spammers' unsubscribe forms means that spam gets sent to those addresses, but the nature of spammers is that it's more trouble than it's worth to try and weed out dead addresses. It's unfortunate that the industry has told people not to unsubscribe.

It's also a myth that CAN-SPAM allows you to "spam until you get an opt out" -- legal, legitimate direct marketers may only send unsolicited email to those who have given permission for their email to be sold to such marketers.

There are many potential holes in the above, but in my experience users can tell fairly well which solicitations are legitimate and which aren't. In other words, they're quite capable of unsubscribing from most legitimate DM.

CAN-SPAM simply does not permit spammers to spam. More to the point, it clearly codifies the spam problem as US residents experience it: the vast majority of spam breaks these rules and so is illegal in the US:

  • Thou shalt not harvest
  • Thy subject shalt not be deceiving
  • Thou shalt not be untruthful in headers
  • Thou shalt include thy physical address (not a PO box)

Tags: , .

How do spammers find your email address?

In my last post, I talked about how spammers send spam. This time, let's see how they decide whom to send it to. First, they need your email address. How are they going to get that? It's not as if you go to their website and ask to be spammed:
Spam Form
Hopefully not, anyway. There are three main ways that spammers get your email address... [read more]

Tags: .

Monday 31 October 2005

Jury still out on CAN-SPAM

This fine chap, Dana Blankenhorn says The CAN-SPAM Act enables spam. Come again, Mr. B?
It legalized specific forms of spam, it overturned stiffer state laws, and it has gone unenforced. The primary enforcement of this "law" has come from private parties. Microsoft, which urged the act's passage, has been the most aggressive ... The likelihood of this being effective in stopping spam is nil ... Shaming corporations into policing their distribution channels and re-sellers would get rid of another hunk. [more]
I think Dana goes too far. This open letter says why...

1. CAN-SPAM sets a good minimum behaviour bar

Few laws please all the people all the time, but CAN-SPAM does bring some useful tools to the spam fighting table, including making the following illegal:
  • harvesting
  • forging headers
  • misleading subjects
  • contracting with spammers (and ignorance is no defense)
It also enables law enforcement to "follow the money" so offshore spammers and those who contract with them aren't safe.

2. Legitimate, permission marketing is not spam

I simply can't agree that "the CAN-SPAM act has done more to enable spam than any other act by anyone." It did not legalize any form of spam that I recognize. I can't understand how you can feel like that, unless you are of the opinion that any form of permission direct marketing is bad. True, it gives no remedy to those who have inadvertently given permission to resell their email address to 3rd parties, which has been better addressed here in the European Union.

3. Civil enforcement and state pre-emption

As to private or civil actions, that's a good thing in a capitalist society. These are organizations that are more motivated to go after spammers than Big Government.

I don't agree that CAN-SPAM has "gone unenforced." All the high profile actions to date have been against the state laws (now largely pre-empted). There are several CAN-SPAM actions in the pipeline right now.

Gathering evidence, finding spammers, and prosecuting them takes time. If you'll pardon the pun, the jury's still out on CAN-SPAM. I'm much more hopeful than you that it and other laws will significantly dissuade spammers.

Tags: , , .

Sunday 30 October 2005

New Pace Twin update

At long last there's a firmware update being broadcast for the Pace Twin (a Freeview/DVB-T hard disk PVR). Early reports indicate:

  • EPG works better
  • FF and Rew more reliable
  • Less picture stuttering
  • Fix for the 0 second recording problem after power fail
  • Faster MHEG text
  • Hasn't broken TwinRIP
Broadcast scheduled only for the weekend though. If you want to be on the bleeding edge, don't delay. I expect it'll be repeated later.

Update: It's over, but it looks like the next broadcast will start 10am November 17th until 9am November 21st.

Digital Spy discussion is here.

Friday 28 October 2005

Bastardcast Oct 28 - Uptight Maggie

GHP - Uptight Maggie

Sorry it's been a while since I posted a mashup. Nothing really caught my ear. But here's another one from Go Home Productions (aka Mark Vidler):
Stevie Wonder / Rod Stewart 2.28
Taken from the GHP XFM Remix Superchunk (Oct '05)
Had this particular pairing sitting on the hardrive for a dogs age before finally polishing it up for the superchunk.
Pretty simple in all honesty. A couple of loops from 'Maggie May' fleshed out with some breakbeats and percussion.
Stevie sounds about 9 years of age...

Categories: , , , , , , . My Odeo Channel (odeo/d3ddc2a176243581)

Governments should offer ISPs incentives to clean up zombies

Most spam is sent by zombies -- PCs infected with viruses, which allow spammers to remotely control them. It's a big problem, but one that most ISPs are doing nothing about, with one or two notable exceptions, such as AOL.

ISPs are in a great position to slash the number of zombies operating today, so why the lack of action? Basically, ISPs have little incentive to identify zombies and help their users clean up their PCs. It requires an investment in time and technology for which there's little payback in their business model. Margins are razor-thin in a competitive, commoditiy marketplace. Few consumers will choose an ISP based on how good they are at cutting off infected PCs.

What if governments encouraged ISPs to actively help in this area? Perhaps via tax breaks. ISPs could be encouraged to instrument all outgoing email traffic so that they can spot patterns. If a subscriber appears to be sending spam, the ISP should cut off their ability to send mail until the subscriber can be contacted and remedial action taken. This could be triggered if a PC sends more than, say, 50 messages per day.

With thanks to the participants in the Message'05 spam roundtable and to Ovum's Graham Titterington for chairing the meeting.

Tags: , , .

Monday 24 October 2005

Kate Bush - King Of The Mountain video

Not a bootleg. Kate is back! The video is now online.

Tags: , , , .

Book review: "Ending Spam" by Jonathan A. Zdziarski

Ending Spam
Bayesian Content Filtering and the Art of Statistical Language Classification
by Jonathan A. Zdziarski
July 2005, 312 pp. ISBN 1-593270-52-6
No Starch Press -

Zdziarski is the creator of the open source spam filter, DSPAM. As such, he is a vocal proponent of the school of statistical filtering, as popularized by Bayesian filtering. No surprise then that his book focuses on statistical filtering, painting it in its most positive light.

The book's structure is well-thought-out. If a chapter becomes too heavy-going -- and some chapters do go into some hair-raising mathematical detail -- the reader can simply skip forward without much trouble.

However, Zdziarski makes little or no effort to tackle the issue of false positives. Generally he glosses over the problems caused by legitimate mail being filtered as spam without acknowledging that such "errors" are much more expensive than the error of unfiltered spam. There were also several places in the book where I'd have preferred the editors and proofreader to have done a better job. It was as if they sometimes misunderstood the point that Zdziarski was making and thus obscured it.

Overall, this book is an excellent primer on spam, spammers, and spam fighting, but the casual reader might get indimidated. It shouldn't be relied on to give a complete and balanced look at spam fighting techniques.

Tags: .

Bloglines burps. My blogroll disappears.

Uh-oh, it happened again. All my feeds have disappeared from Bloglines. Good job I export the feeds to OPML from time to time.

Tags: .

Sunday 23 October 2005

How do spammers spam?


A typical spam run sends millions of messages -- all identical (or very similar). How do they do it?

Of course, our spammers aren't sitting in a basement room, feeding Outlook Express with millions of names. So how do they manage to send so many messages so quickly?

As far as sending goes, there are two types of spammer... [read more].

Tags: .

Saturday 22 October 2005

No, still I'm not sending you spam

Hi. If you've come to to find out who's sending you spam ... it's not me. This happened last month as well. For more info, click here.

Tags: , .

Fast, Full-Text Search can be Serendipitous, but Should be Global

The benefit usually touted for indexed (or fast, full-text search) is that it's lightening fast. That's true: compare a vanilla Outlook search for text in the body of thousands of emails with the same search using Google Desktop or the new version of Eudora. The Outlook search will take a minute or two with a large mailbox, wheres the indexed search will take seconds.

However, speed isn't the only benefit. When it's easy and quick to search everything, one can often find lucky results that one wasn't expecting. For example, when I was searching for a message I received from a new client yesterday, I realized that I'd also corresponded with him in a previous job, several years ago. I'd forgotten about that and I suspect he had too. I was also presented with some web pages that mentioned his name, a saved PowerPoint deck authored by him, and his full contact details. All information that I had on my PC, but that I didn't know was there -- that's the point: global, indexed search allows one to find things one didn't know existed.

For the full benefit, the searching should be global -- i.e. the search tool should index everything on your PC, not just your email. Unfortunately, Eudora 7 only searches its own store, and therefore misses out on this serendipitous benefit.

Statistical Spam Filters are Too Hard to Use

Statistical spam filters use powerful mathematics to decide if a message is spam or not. They classify email as spam or ham, using Bayesian analysis and other statistical methods. Examples of such filters are SpamBayes, POPfile, DSPAM, and CRM114.

State of the art statistical filters can achieve levels of accuracy as good as or better than a user manually filtering spam with the Delete button. However, such filters require several months of training before they can achieve the accuracy required. Filters that rely on end-users to train them aren't suitable for the majority of users.

This training can be done by feeding the filter a "corpus" of spam and legitimate messages (i.e. an archive of several months of spam and ham). However, the initial and ongoing training requirements are onerous and error-prone. When users complain that a good statistical spam filter isn't accurate, it's usually because they haven't trained it properly; but that's hardly fair -- users just want their filter to work.

Tags: .

Friday 21 October 2005

Switching to an abbreviated feed

Up until now, I've always run a full feed on Richi'Blog. That is, the text you get in your reader or aggregator is the full monty, with no need to click through to the web page. It's with a heavy heart that I'm changing to an abbreviated feed. I'm doing this for a several reasons, chief among them that I'm fed up with my writing being ripped off by sploggers. Grrr.

Tuesday 18 October 2005

Ajax in the enterprise

Infoworld has a nice roundup of how Ajax is making inroads into business's IT departments. Includes coverage of Scalix, and NetSuite. Also mentions dev. tools: Backbase, JackBe, TibcoGI, and Ruby on Rails. And the obligatory quote from Jesse James Garrett ;-)

read more | digg story

Saturday 8 October 2005

Speaking of shills...

...which I was, here's a word to the wise...

Don't even think about posting comments to my blog from an IP address in the same /25 used by the company that you're promoting.

...unless you want to be accused of being a comment spammer.

Tags: , , .

Friday 7 October 2005

Is this is a 419 scam?

Yes, it is!

The weirdest message to hit my spamtraps in a long while. I suppose there must be an advance fee element involved somewhere, but it's certainly not the ordinary "this guy died and I need your help to liberate his money" scam.





I know you must be very surprised to be reading from me,considering the fact that you dont know the identity of the person writing you,but be rest assured that i am writing you with good intentions.

I am mrs nancy doyles,the president and owner of DOYLES FOOTBALL CLUB,a clubside based in dublin,republic of ireland.I got your e-mail contact adress from an internet directory here in bangkok thailand,while browsing the web,searching for talented,fit and skilled young football stars to build my new team with.

DOYLES FOOTBALL CLUB is a clubside i inherited from my late husband,mr patrick doyles.The clubside was disbanded after my husband's untimely death 2 years ago,but was re-opened late this year to carry on with my late husbands dreams of taking the team to greater heights.

For best results for the team,i hired the services of an experienced english coach that has been a former assistant coach in a premiership side in london,i also hired the services of many experienced english technical advisers and assistant coaches.I trust that with their services and that of talented young boys,the sky will be the limit for DOYLES F.C and our aim is to be at the top of the irish league,and onward gradual progress to the premier league.This feats can't be achieved without the help and assistance of young football stars.

DOYLES FOOTBALL CLUBis a football club we want to build around young football stars because we want to start from the grass roots level.The team is currently in bangkok thailand on a 2 months training and trial tour of Thailand.

REASON FOR CONTACTING YOU:If you can play active football well,or if you can not play,but knows or have a brother or a friend that can play well,please contact me on mail by replying to the e-mail adress below.

BASIC REQUIREMENTS:each player must be 17 - 29 years old,each player must report at the teams hotel camp here in bangkok thailand for the screening and trial of the player.Interested players should reply to the adress below for more informations on the travel procedures.

So far we have gotten 8 players from africa,5 from south america,4 asians and 3 europeans,1 american has confirmed his flight ticket and will join the rest of the team next week.

The selected players will return to the teams base in dublin,and must sign a compulsory 1 year contract with the team for the kickoff of the irish league,starting early next year.The dropped players will be compensated and they will return to their home country with their return tickets.

best regards and hoping to hear from you soonest.

nancy doyles.

for best correspondence,reply this mail only to:,

Tags: , .

Thursday 6 October 2005

CNN has no clue. I'm shocked.

Slow news day? CNN asks, Is it time for England to change its national flag?:

LONDON, England (CNN) -- British prison officers who wore a St. George's Cross tie-pin have been ticked off by the jails watchdog over concerns about the symbol's racist connotations.
The pins showing the English flag -- which has often raised hackles due to its connection with the Crusades of the 11th, 12th and 13th centuries -- could be "misconstrued," Chief Inspector of Prisons Anne Owers said in a section on race in a report on a jail in the northern English city of Wakefield.
Sigh. Where to start?

  1. First and most obviously, the Cross Of St. George isn't the flag of Britain. It's the English flag (inasmuch as England could even be said to have a flag -- see next point.)
  2. England isn't a nation, so CNN's silly Quickvote just shows someone's ignorance. It would be like asking if California should change its national flag. Huh? The "nation" is the UK (or properly The United Kingdom of Great Britain and Northern Ireland). Our flag is the Union Flag (shown to the right), not the CoSG.
  3. There are some in England who misuse the CoSG as a racist symbol. These people have no concept of the Crusades and if you asked them what happened almost 1000 years ago would probably give you a funny look, smash the top of their pint glass, and mash the jagged edges into your face "for being clever."
Hat tip: Mailgeek. Thanks to

Rant of the day: Microsoft "invented" AJAX?

I am so sick of Microsoft shills popping up everywhere to mention, "we invented Ajax."

Shoulders of giants, people! Just about every innovation has been built on top of other people's work. Look at the IE7 beta for example: a blatant Firefox ripoff.

Outlook Web Access, even in its 2003 incarnation, is a very poor example of an Ajax app. The term means more than simply sending information in the background. OWA's interface design is still fundamentally page based, so the real-world performance is much slower than something like Scalix.

(See also: more about AJAX and how it differs from conventional web methods.)

Tags: , .

Wednesday 5 October 2005

Use outbound spam filtering, as well as inbound

Inbound spam filtering is now commonplace. Outbound spam filtering isn't; but it should be.

If a PC in your organization gets infected with malware -- such as a remote-access Trojan -- it may become a spam zombie. This means that it will send spam under the remote control of a spammer. This can happen despite your best efforts with firewalls and anti-virus technology.

If the recipients of the spam track down the source of these spam messages, they won't reach the spammer -- they'll reach you. Your organization is therefore at risk of abuse complaints to your ISP or spam blacklisting, either of which can have serious consequences for your business operations. Outbound spam filtering can help prevent this from happening.

Tags: , , .

Tuesday 4 October 2005

INBOX'06 anyone?

I'm pleased to announce that The Golden Group, organizers of the INBOX conferences, have asked me to program their 2006 conference sessions. It's going to be a challenging task to slip into Martin Hall's shoes, but at least I have a similar funny accent.

Are you interested in being a part of the conference? Perhaps by speaking, moderating, or being on a panel? We're at the very early stages of planning, so there's no Call For Papers yet. If you're interested, drop me a line to the usual email address.

(INBOX'06 will be in early June, in San Jose, California. You can still find the previous year's conference web pages.)

Tags: .

Monday 3 October 2005

Why do spammers spam when they know we hate it so much?

Why indeed! For many users of email, spam is a constant source of frustration and annoyance ... Guess what? Spammers aren't sending all this email just because they enjoy annoying people. There's almost always a commercial motive. So, let's follow the money! ... What can we learn from this? Don't buy from businesses that spam — it only encourages them. [more]
Tags: .

Sunday 2 October 2005

Another anti-spam tool to avoid

Some company called hendrickson software components is touting a new spam filter called Em@ilCRX.

Guess what? It...

...uses an automated challenge response system, and reverse DNS validation to stop spam from making it into your email inbox.

Oh brother. All together now, say it with me:
  1. Challenge/response causes spam
  2. If you use it, you're a spammer
  3. If everyone used it, email wouldn't work!
This topic previously covered here and here.

Tags: .

Friday 30 September 2005

Ferris Research daily news feed

Ferris Research now has an RSS feed for its free daily news service. Check it out at

Tags: .

Wednesday 28 September 2005

Spam "from" the "Church" of Scientology

Hello, what do we have here?
The Church of Scientology Mission of Sherman Oaks is extending an invitation to you to receive an OCA Personality Test and evaluation free of any charge or obligation.

Your personality has everything to do with your income, your future, your personal relationships, and your life.

A test of this kind would normally cost you $500.00 and up. It is offered to you here free of charge as a public service.

[...blah blah blah...]

Contact Us:
Church Of Scientology Mission of Sherman Oaks
13517 Ventura blvd. , Sherman Oaks, CA, 91423

[...] This e-mail message is an advertisement and/or solicitation.
By the definition enacted in US Federal statue, this message is illegal spam, as it does not contain any mechanism for opting out.

It was also sent to an email address that should never receive unsolicited email, which indicates that the address might have been harvested -- also illegal. Has anyone else received this?

Most of the links point back to the domain, which seems to have stealthy whois information. The email originated from CrystalTech Web Hosting Inc. in Arizona.

Remember kids, Scientology isn't a religion. As Wikipedia says:
...the governments of Germany and Belgium officially regard the Church of Scientology as a totalitarian cult; in France, a parliamentary report classified Scientology as a dangerous cult; in the United Kingdom and Canada the Church of Scientology is not regarded as meeting the legal standards for being considered a bona fide religion.
L. Ron Hubbard would be proud...

Tags: , .

Tuesday 27 September 2005

Reading this in email? Please take action!

If you've been receiving Richi'Blog by email, I need you to take 30 seconds to ensure you continue to receive it...

You've probably been noticing some odd things happening with the email feed, like duplicated or missing posts. Sorry about that; it turns out that KBcafe's Rmail service isn't terribly reliable. So I've switched to using FeedBlitz.

Please take a moment to re-subscribe using the form below:

Once you're happy that it's working OK, you'll probably want to unsubscribe from the Rmail service, using the unsubscribe link at the foot of the emails that they send.


Friday 23 September 2005

What's Google's next move in wireless?

Google's in a great position to take advantage of the many things it has going for it. As well as search, these include:

  • Location dependence—Google is now very good at geo-location, and directs you to a search cluster near you for performance. It can also show you geo-targeted ads, which will probably become more fine-grained in future.
  • Wi-Fi—it seems that Google is about to launch a public access Wi-Fi product, which it will also secure with a VPN offering: Google Secure Access. This should mean that security will be better on a Google Wi-Fi access point than with, say, T-Mobile. Speculation is that access will be free, and Google will generate revenue by mining contextual information as they do today with Google ads.
  • Bandwidth—Google seems to have been busy buying up insane amounts of "dark fiber" capacity in the US, which it'll need if it's going to offer free Wi-Fi!
As an example, combine these three factors with Froogle (Google's comparison shopping service). If you're out shopping in bricks and mortar stores, what if you could connect using your PDA over Google Wi-Fi for free, it knew where you were, and could show you the closest place to purchase what you're looking for?

Tags: , , , .

Wednesday 21 September 2005

No, I'm not sending you spam

Hi. If you've come to to find out who's sending you spam ... it's not me.

Spammers usually send email with forged senders. In this case, using email addresses It appears that quite a number were sent out over the last few hours, with subjects like "FW: whats up?" and "lol, I found a gold mine"

In tech. circles, this is known as a Joe job:
an incident of spamming designed to tarnish the reputation of an innocent third party ... most email Joe jobs are acts of revenge.
Tags: , .

Thursday 15 September 2005

Here we go again

So... I was trying to make heads or tails of this press release this morning:

“The [spam] filtering approach was designed to handle junk mail for people receiving between zero and sufficient numbers to cause a nuisance. The real issue now is for people in the flood category, where filtering is not viable.”

Huh? What are they talking about? Let's read on...

Figures vary for the volume of unwanted mail. ... Using 80% means that four out of five mails for users [who get 30 or more spam messages per day] need to be marked, filtered, re-directed, quarantined and possibly archived.

Aside from the dodgy mathematics (there's no direct correlation between the number of legitimate and spam messages you receive), what is the point of all this?

High volumes are starting to strain the filtering approach because the filter has to take action on each mail it determines to be unwanted. This strains computing resources and also obliges recipients to take some action. Because the mail may have come from a source that has sent mail before, the receiver cannot ignore it.

Uh-oh, I have a bad feeling about this...

The alternative to filtering lies in the challenge-response method of dealing with spam, as used by the NMS’s Australian-developed TotalBlock solution.

Bingo! Yes, dear reader, it's our old "friend" challenge/response again. You may recall my previous post on this subject.

So, to summarize, if spam is a "nuisance" to you, why not turn it around and be a nuisance to legitimate senders who want to communicate with you, and be a nuisance to the poor people who are getting their email addresses used as forged spam senders? Yeah, and let's "oblige senders to take some action" instead. That's reasonable. Sheesh.

When will you people figure it out? In nice, simple language:
  1. Challenge/response causes spam
  2. If you use it, you're a spammer
  3. If everyone used it, email wouldn't work!

Tags: , .

Oh dear: false positives ahoy!

So the good people behind Mailinator (the disposable email address folks) decided to put up a Google Maps thingy that shows where spam has come from recently. Unfortunately, it only served to illustrate the false positive problem...

Tags: , , .

What is Ajax?

Ajax is an emerging way to design applications that run inside a web browser. Its key advantage over conventional web applications is that Ajax applications are much more responsive and interactive.

It's an acronym for Asynchronous JavaScript and XML. What that means is that the underlying data are exchanged between the browser and web server in XML and the display intelligence runs in a JavaScript program (AKA ECMAScript: i.e. the scripting language, not Java applets). The program is downloaded when the browser first connects to the web server.

A relevant example of a good Ajax application is Scalix Web Access (SWA): David Ferris of Ferris Research just called it "The Best Email Web Client." This alternative to Outlook is extremely fast, compared with the current Outlook Web Access and provides a comprehensive list of functionality, including email, calendar, scheduling, tasks, contacts, delegation, and public folders. Another example is Google's GMail, although the Gmail paradigm is a little too page-based for Ajax purists.

To contrast a conventional web application with an Ajax application:

  • Display intelligence runs in the web server, which generates HTML pages to be displayed in the browser
  • Display intelligence runs in the browser, which exchanges the underlying data in XML
  • Page-based user interface paradigm, unlike most desktop applications; each change requires a complete new page to be transferred and displayed
  • User interface design can be much more familiar -- similar to desktop applications; changes can simply modify an existing object on the page and usually do not require a server transaction
  • Mouse interactions are limited to clicks, which take considerable time to transfer to the server and be acted upon
  • Interactions can be far richer, including dragging
  • Users must wait for each interaction to complete
  • Data transfers can be scheduled in the background, meaning that users don't need to wait; data can be pre-fetched in anticipation
  • Must be online to use
  • Could work offline

There's much more that can be said about Ajax. If this short post has whetted your appetite, ask me more.

Tags: , .