UK Liberty

More data does not mean better data

Posted in database state, surveillance society by ukliberty on October 8, 2008

With the unending announcements of government databases and surveillance techniques storing more and more data, particularly the most recent confirmation that the Intercept Modernisation Programme (IMP) is proceeding, it’s useful to look at the results of schemes overseas such as Total Information Awareness (Wikipedia).

The New York Times (also The Register) reports of a federal panel of policy makers and scientific experts that,

… there is little evidence to confirm that [data mining] techniques work to actually find terrorists, despite the growing use in the last seven years, committee members said. Part of the problem, they said, is that the sample of known terrorists and actual attacks is so small that it is difficult to establish patterns of suspicious behavior.

The push to accumulate enormous amounts of information has also produced the risk of “a huge number of false leads” that could implicate people with no actual connections to terrorism, the committee said.

(executive summary and full report available online.)

Jim Harper, among others, wrote about this issue some time ago:

Data mining for terrorism prediction has two fundamental flaws:

— First, terrorist acts and their precursors are too rare in our society for there to be patterns to find. There simply is no nugget of information to mine.

— Second, the lack of suitable patterns means that any algorithm used to turn up supposedly suspicious behavior or suspicious people will yield so many false positives as to make it useless. A list of potential terror suspects generated from pattern analysis would not be sufficiently targeted to justify investigating people on the list.

You may be interested in a paper he co-authored with Jeff Jonas, where they describe the problem in more detail and also discuss how we are in fact better off not doing it.

Now, that is not to say they have no use at all, as Richard Clayton made clear to The Register::

“They do say that data mining can assist in investigations. If you’ve just seen a bomb go off, then checking who has bought lots of fertiliser in the last month, and collating that against purchases last year to filter out the farmers, might be useful in prioritising who you’d like to speak to.

“The objection to this part of IMP isn’t that it would not be useful to the police; it’s that the centralised database removes a lot of checks and balances in using the data.”

Professor Ross Anderson’s book Security Engineering may also be of interest  – one chapter in particular, on Terror, Justice and Freedom, is available online.

(hat-tip to Andrew Watson for Jim Harper links.)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: