UK Liberty

More data does not mean better data

Posted in database state, surveillance society by ukliberty on October 8, 2008

With the unending announcements of government databases and surveillance techniques storing more and more data, particularly the most recent confirmation that the Intercept Modernisation Programme (IMP) is proceeding, it’s useful to look at the results of schemes overseas such as Total Information Awareness (Wikipedia).

The New York Times (also The Register) reports of a federal panel of policy makers and scientific experts that,

… there is little evidence to confirm that [data mining] techniques work to actually find terrorists, despite the growing use in the last seven years, committee members said. Part of the problem, they said, is that the sample of known terrorists and actual attacks is so small that it is difficult to establish patterns of suspicious behavior.

The push to accumulate enormous amounts of information has also produced the risk of “a huge number of false leads” that could implicate people with no actual connections to terrorism, the committee said.

(executive summary and full report available online.)

Jim Harper, among others, wrote about this issue some time ago:

Data mining for terrorism prediction has two fundamental flaws:

— First, terrorist acts and their precursors are too rare in our society for there to be patterns to find. There simply is no nugget of information to mine.

— Second, the lack of suitable patterns means that any algorithm used to turn up supposedly suspicious behavior or suspicious people will yield so many false positives as to make it useless. A list of potential terror suspects generated from pattern analysis would not be sufficiently targeted to justify investigating people on the list.

You may be interested in a paper he co-authored with Jeff Jonas, where they describe the problem in more detail and also discuss how we are in fact better off not doing it.

Now, that is not to say they have no use at all, as Richard Clayton made clear to The Register::

“They do say that data mining can assist in investigations. If you’ve just seen a bomb go off, then checking who has bought lots of fertiliser in the last month, and collating that against purchases last year to filter out the farmers, might be useful in prioritising who you’d like to speak to.

“The objection to this part of IMP isn’t that it would not be useful to the police; it’s that the centralised database removes a lot of checks and balances in using the data.”

Professor Ross Anderson’s book Security Engineering may also be of interest  – one chapter in particular, on Terror, Justice and Freedom, is available online.

(hat-tip to Andrew Watson for Jim Harper links.)

All seeing state to end privacy and more besides

Posted in database state, state-citizen relationship, surveillance society by ukliberty on October 8, 2008

Great article by Jenni Russell in the Guardian:

You might suppose that the economic tornado hitting Britain would cause the government to focus its energy and resources very tightly on the political projects that are of undoubted value. This is not, after all, the moment to be wasting either political or financial capital. But you would be wrong. Faced with a crisis that it patently can’t control, the government is instead seeking to exert power where it still can: over us.

The state’s latest plan to watch us makes every other imminent intrusion seem limited. Next month’s Queen’s speech will contain a brief reference to an innocuous-sounding communications data bill. But what this means is the development of a centralised database that will track, in real time, every call we make, every website we visit, and every text and email we send. That information will then be stored and analysed – perhaps for decades. It will mean the end of privacy as we know it.  …