Prism: the spying software reading your email and Facebook

The US government is accessing the world’s emails, Facebook messages and Skype conversations, through an intelligence tool called Prism, the Guardian revealed. Scooping up information from Skype, Apple and Google, Prism analsyses huge amounts of data to work out who is a terrorist threat and who is just using emphatic language in an email to their mum.

But in order to work out whether you, the email user, are a terrorist, agents will be using something not much more advanced than an online tool we use every day.

Professor Kevin Jones of City University told Channel 4 News: “It’s unlikely to be anything beyond what we’re all used to with search engines.

“It’s just likely to have been very fine-tuned to specific topics.”

How the spying software works

We do not know precisely what tool the US government uses to filter the masses of information it harvests. But crunching big amounts of data is a field that still relies on basic keyword searches.

However, it can be improved, said Professor Jones. Putting high-power computing behind the search engine means the NSA could crunch a lot of data very quickly. Scanning headers on data means search results can be filtered by location of the computer and the device.

And looking at how terrorists have talked to each other before could make it easier to spot new ones.

“If you have previous communications that you know are related to crime or terrorism, then you can use it as training,” says Dr Jun Wang of the Computer Science Department at University College London. “You can create a mathematical model to compare the features with new communications, as a tool for detection.”

So, says Dr Wang, “a computer will be able to tell you that a communication is say 70 per cent likely to lead to a terrorist attack.”

The software may also be specially tuned to interface with the systems of different internet companies such as Facebook if the Guardian’s account of access to the servers is correct.

One hiccup, though. Substituting a word like “balloon” for a more obviously terrorist-sounding one such as “bomb” would dodge the checks.

That is a limitation of even the smartest keyword searches, says Dr Wang. “If you encrypt it in some way using a code, it couldn’t spot it. If you don’t know the code used, you’d be unable to decode that message. “

So who are the companies making the software?

Many companies make software for intelligence services.

One such company is Palantir Technologies, named for the seeing stone in JRR Tolkien’s Lord of the Rings, which has worked closely with the CIA since it received money from the CIA’s investment arm In-Q-Tel in 2005. A 2009 Wall Street Journal article explained how the intelligence services used Palantir’s software to spot terrorism in the same way that PayPal uses software to spot fraud.

“Palantir Technologies has designed what many intelligence analysts say is the most effective tool to date to investigate terrorist networks,” the article said.

“The software’s main advance is a user-friendly search tool that can scan multiple data sources at once, something previous search tools couldn’t do.

“That means an analyst who is following a tip about a planned terror attack, for example, can more quickly and easily unearth connections among suspects, money transfers, phone calls and previous attacks around the globe.”

A computer will be able to tell you that a communication is 70 per cent likely to lead to a terrorist attack. – Dr. Jun Wang, UCL

Palantir has a product called Prism, says Gawker, but it’s incidental that it shares a name with the government service, says Palantir who are not commenting on their contracts.

In-Q-Tel invests in companies that could make products useful to CIA and other intelligence services. Recently that has included a company that produces “automatic journalism” – short prose summaries of large amounts of data.

Though it has been used to produce automatic sports writing, start-up Narrative Science could be useful to spies too. It signed a deal to develop a version of their automatic writing tool for government intelligence just last week.

UK-based Autonomy, part of Hewlett-Packard, also makes software that can analyse large amounts of social media data to uncover crime or public disorder before it happens. It ran a trial with the Metropolitan police during the Olympics do just that. The US Department of Defense is another Autonomy customer, but a spokesman for Hewlett-Packard has not commented on questions about their government contract.

Other companies that make intelligence software include BAE Systems’ Detica, whose software is used for government security. A spokeswoman for Detica said she cannot comment on questions that concerned national security.