Richi Jennings

Tuesday, 14 April 2009

Spam and its Carbon Footprint

April 17 1.05pm UTC: add ORF radio interview
April 17 8.20am UTC: add link to full report
April 16 11.55pm UTC: updated to stress that the methodology is available in the full report
April 16 5.15pm UTC: as someone on /. helpfully points out, 33 TWh/year is 3.8 GW
April 15 5.20pm UTC: welcome, Slashdotters
April 15 9.00am UTC: updated to add note on incremental energy

All uses of the Internet have an impact on climate change. Sadly, that includes the less-savory uses.

Spammers dumped 60 trillion messages onto the Internet in 2008. As the climate-change consensus becomes overwhelming, it's high time we looked at the environmental impact of spam.

Recently, McAfee commissioned climate-change consultants ICF calculate the carbon footprint of spam. McAfee also asked me to help. We calculated the energy use associated with each stage in the lifecycle of spam, including the energy used to transmit, process, and filter spam.

Globally, the annual spam energy use is 33 billion kilowatt-hours, or 33 TWh—that's as much electricity as 2.4 million U.S. homes use, with the same greenhouse gas emissions as 3.1 million passenger cars using 2 billion U.S. gallons of gasoline.^[1]

Two Surprising Conclusions

Far from being a net consumer of energy, spam filtering actually saves an incredible amount of energy. Imagine if all the spam filters in the world were switched off for a day. It would actually increase the carbon footprint of spam by at least five times.^[2] In other words, spam filtering saves 135 TWh of electricity per year—that's like taking 13 million cars off the road.

But we could do even better. Imagine if every inbox were protected by a state-of-the-art spam filter. We could save about 75% of the spam energy used today—25 TWh per year;^[3] that's like taking 2.3 million cars off the road.

Other Results

The average greenhouse gas emission associated with a single spam message is 0.3 grams of CO₂. That's like driving 3 feet (1 metre), but because of the annual volume of spam, it's like driving around the Earth 1.6 million times.^[4]

A year's email at a typical medium-sized business uses 50,000 KWh, more than one fifth of which is associated with spam.^[5]

Filtering spam is all well and good, but fighting spam at the source can have even better results. Taking McColo offline in late 2008 saved energy equivalent to taking 2.2 million cars off the road, before spammers rebuilt their sending capacity.

Energy use associated with spam is mainly consumed by end-users deleting spam and searching for legitimate email ("false positives"). Only 16% of energy use is from spam filtering itself.

Notes

My role in the McAfee project was to help ICF build a model that accurately reflected where energy was used in producing, transmitting, filtering, and dealing with spam. To this end I provided consultancy and data, plus some analysis of the results.

The data came from my 25 years of experience with email and spam, cross-correlated with data from other researchers (including McAfee and McAfee's competitors).

33 TWh of electricity use emits 17 million tonnes (19 million U.S. tons) of CO₂, equivalent to 3.1 million passenger cars, burning 7.6 billion litres of gasoline (2 billion U.S. gallons), or 2.4 million U.S. homes' electrical usage.
Switching off spam filtering for one day would multiply spam in the average inbox by 5x and multiply false positives by at least 10x. While no energy would be used by spam filters, this reduction is vastly outweighed by the energy used by end-users coping with spam.
Most inboxes are protected by spam filters, but many of them are less accurate than the best filters. Some inboxes are still completely unprotected. State-of-the-art filters can achieve better than 98% effectiveness/0.01% false positives and use less power: assumes 25% power saving over legacy spam filters.
Based on passenger car averaging 20 miles per U.S. gallon; mean equatorial circumference 40,041 km (24,870 miles).
Refers to an organization with 200 average business email users.

Incremental Energy

In any calculation such as this, there's always the concern that we're double-counting energy that would have been used whether or not there was spam. Let me assure you that this isn't the case. I only wanted to be involved in the project if we were measuring this meaningfully.

So the data and calculations were carefully designed so as to only measure energy that is used as a direct result of there being spam. In other words, it is "incremental" energy.

PCs and servers use less energy when idle than when doing "work"—in most cases it's this additional energy that we measured.

More About the Methodology

Some wags have complained that ICF doesn't publish its methodology. They have clearly not read the full report, including the appendix, which helpfully titled, err, Statement of Methodology. Perhaps they're confused by the 8 page summary?

Download Full Report

You can now get the full 28 page version at the usual place.

Radio Interview

ORF interviewed me on Friday. Download it here: richi-on-orf.mp3.

Monday, 6 April 2009

Hidden Risks of Intellectual Property in Email

I was talking recently to a client. We were discussing how organizations use email and the conversation turned to "inappropriate" use.

Ah, no. Not that sort of inappropriate use.

I'm talking about storing confidential information in an organization's email. Perhaps even sending that confidential information to people who have no right to receive it.

Just about every organization has this type of confidential information. It could be customer data or intellectual property, such as future product plans or patentable know-how. Whatever it is, organizations run a huge risk by allowing their users to email it.

And I'm not necessarily talking about attacks on the email system. Even the most sophisticated user can accidentally send the wrong thing to the wrong person.

Even if the information stays inside the organization, it's at risk from malicious insiders—if someone has access to an Exchange server, they can read the entire message database without needing any passwords.

And I'm not talking about a few scraps of information, either. When you look at a typical organization that's reasonably email-centric, a significant number of them might have the vast majority of their confidential information stored in their email—as much as 85% of it.

So what to do about it?

You could try educating your users... but that sounds like the epitome of cat herding.
You could shut down your email system... but isn't that rather self-defeating?
Various vendors will sell you tools for data leak prevention (DLP).

Combined with user education, DLP can do a reasonable job of helping email users "keep it in the family." But it isn't magic; it does need care and feeding.

It does need to be taught what your confidential information looks like, and who should be permitted to see it. And that teaching process isn't just a one-off chore—it needs to be ongoing.

Wednesday, 18 March 2009

Where's Richi?

Yes, I'm not blogging here much these days. So what am I up to?

In addition to my Computerworld daily ramblings and Ferris Researchings, you can mostly find me microblogging on Twitter.

You'll also find me at Facebook, LinkedIn, FriendFeed, and FriendFace.

Here's a full list of contact details.

Friday, 6 March 2009

IT Blogwatch roundup

Here's a catchup of the last two week's of IT Blogwatching. Sorry I missed last week's post: I had a rush job on.

March 6, 2009 - 10:24 A.M.

TomTom fights Microsoft to protect GPL?

In a special IT Blogwatch Extra, Richi Jennings watches the growing disquiet over Microsoft's TomTom patent lawsuit. Not to mention what Darth Vader finds disturbing...