With the unending announcements of government databases and surveillance techniques storing more and more data, particularly the most recent confirmation that the Intercept Modernisation Programme (IMP) is proceeding, it’s useful to look at the results of schemes overseas such as Total Information Awareness (Wikipedia).
… there is little evidence to confirm that [data mining] techniques work to actually find terrorists, despite the growing use in the last seven years, committee members said. Part of the problem, they said, is that the sample of known terrorists and actual attacks is so small that it is difficult to establish patterns of suspicious behavior.
The push to accumulate enormous amounts of information has also produced the risk of “a huge number of false leads” that could implicate people with no actual connections to terrorism, the committee said.
Jim Harper, among others, wrote about this issue some time ago:
Data mining for terrorism prediction has two fundamental flaws:
— First, terrorist acts and their precursors are too rare in our society for there to be patterns to find. There simply is no nugget of information to mine.
— Second, the lack of suitable patterns means that any algorithm used to turn up supposedly suspicious behavior or suspicious people will yield so many false positives as to make it useless. A list of potential terror suspects generated from pattern analysis would not be sufficiently targeted to justify investigating people on the list.
You may be interested in a paper he co-authored with Jeff Jonas, where they describe the problem in more detail and also discuss how we are in fact better off not doing it.
“They do say that data mining can assist in investigations. If you’ve just seen a bomb go off, then checking who has bought lots of fertiliser in the last month, and collating that against purchases last year to filter out the farmers, might be useful in prioritising who you’d like to speak to.
“The objection to this part of IMP isn’t that it would not be useful to the police; it’s that the centralised database removes a lot of checks and balances in using the data.”
(hat-tip to Andrew Watson for Jim Harper links.)
You might suppose that the economic tornado hitting Britain would cause the government to focus its energy and resources very tightly on the political projects that are of undoubted value. This is not, after all, the moment to be wasting either political or financial capital. But you would be wrong. Faced with a crisis that it patently can’t control, the government is instead seeking to exert power where it still can: over us.
The state’s latest plan to watch us makes every other imminent intrusion seem limited. Next month’s Queen’s speech will contain a brief reference to an innocuous-sounding communications data bill. But what this means is the development of a centralised database that will track, in real time, every call we make, every website we visit, and every text and email we send. That information will then be stored and analysed – perhaps for decades. It will mean the end of privacy as we know it. …