Internet and phone firms are preparing to install "black boxes" to monitor UK internet and phone traffic, and decode encrypted messages - including Facebook and GMail messages.

'Black boxes' to monitor all internet and phone data (R)

As part of the Home Office's communications data bill, internet service providers (ISPs) and mobile phone companies will be obliged to collect communications records and keep them for a year, writes Channel 4 News Technology Producer Geoff White.

The government has insisted that the actual content of messages won't be stored, but until now it has not been clear how communications companies will be able to separate content from "header data", such as the sender and recipient of a message, and the date it was sent.

It has now emerged that the Home Office has held meetings with the UK's largest ISPs and mobile network operators, and has given them information about the hardware which companies will have to use to monitor traffic flowing through their systems.

When an individual uses a webmail service such as Gmail, for example, the entire webpage is encrypted before it is sent. This makes it impossible for ISPs to distinguish the content of the message. Under the Home Office proposals, once the Gmail is sent, the ISPs would have to route the data via a government-approved "black box" which will decrypt the message, separate the content from the "header data", and pass the latter back to the ISP for storage.

Dominic Raab, a Conservative MP who has criticised the bill, said: "The use of data mining and black boxes to monitor everyone's phone, email and web-based communications is a sobering thought that would give Britain the most intrusive surveillance regime in the west. But, many technical experts are raising equally serious doubts about its feasibility and vulnerability to hacking and other abuse."

A representative of the ISPs Association said: "We understand that government wants to move with the times, and we want to work with them on that. But this is a massive project. We'd rather they told us what they want to achieve, then sit down with us to work out how."

"Our other main concern with this is speed. If you're having to route all traffic through one box, it's going to cut down on connection speeds. The hardware can only look at a certain amount of traffic per second - if lots of streams from the BBC iPlayer are going through it, for example, how is it going to handle the traffic?"

A Home Office spokesman said -

"We have not issued any hardware or software specifications.

"The communications data bill is designed to allow the police to maintain their capability to catch criminals and protect the public as technology changes and people use more modern communications. Under this programme the emphasis is to work with industry to determine the best way to achieve this.

"The legislation is currently being scrutinised by parliament. Once it has been passed will we work with companies on how to best collect and store communications data, but not the content."

Follow @geoffwhite247 on Twitter