RBS meltdown blamed on human error and cuts

Technology forums are abuzz with discussion of the problems affecting NatWest and Ulster Bank customers, who been unable to access money paid into their accounts or have found themselves unable to make electronic payments.

The problems, which began almost a week ago, was caused by a software upgrade at an IT centre in Edinburgh. It has now been resolved, but RBS, which owns NatWest and Ulster Bank, is now dealing with a “significant” backlog of transactions.

The blame is being be placed on a procedure at RBS to update the CA-7 job processing software, but it has also been suggested that the company no longer has the IT expertise that would have quickly solved or avoided this problem following a cutback on back-office staff.

Human error

The chatter on IT and technology forums suggests that the problem has resulted from a human error during this procedure. A programmer who has worked on the RBS/NatWest system told the Guardian that whoever did the update managed to delete the files that hold the schedule for overnight jobs so that they did not run, or ran incorrectly.

Companies should not be complacent about mundane tasks; Anna Leach, The Register

Bryan Glick, the editor-in-chief of Computer Weekly, said CA Technologies products and similar software was widely used for batch processing jobs by large businesses, including banks.

He added: “RBS is using a highly customised version of CA-7. Software is developed and improved by people and people make mistakes. It seems that in this case it is more cock-up than conspiracy.”

Anna Leach, a reporter at the IT website The Register, said the problem is “unique to RBS” and not a flaw in CA-7’s system.

Inexperienced staff member

RBS has said little about what went wrong, or indeed where the error occurred. The trade union Unite has claimed that the transfer of IT work to India may have led to the problems, and The Register has reported that RBS was advertising for experts in the CA-7 tool as recently as February.

The website’s latest report has pinned the error down to an inexperienced member of staff, though it is unclear where this individual is based.

RBS has denied that the problem originated in India and has insisted that the offending update and the overseeing of its roll out was carried out in Edinburgh.

RBS Group’s director of customer services Susan Allen said: “We have great people in our teams in India… we are a global bank, but in this case the actual activity was undertaken in Edinburgh.”

She also said that the problem was spotted on the Tuesday but added that the backlog of unprocessed jobs led to problems on subsequent nights.

“I am pleased to say that now, as of today, we have cleared the huge backlog that led to this problem. What we are now dealing with has some low, knock-on impacts.”

Redundancies

It has not escaped the notice of IT experts on tech forums and trade union activists that RBS has made significant back-office redundancies over the past three years, including laying-off IT staff.

Ms Leach, who has reported on RBS’s problems , said: “RBS uses a complex system that has grown up over a number of years. The company’s older IT staff knows what goes on and how to fix problems when they crop up.

“It is hard to transfer that expertise to newer staff who don’t know the peculiarities in the mainframe system at RBS.”

“This shows the importance of these systems that a small mistake on a routine task can have an impact on millions of people. It shows that companies should not be complacent about mundane tasks like batch processing.”

Mr Glick said that cutting IT staff suggests a particular attitude in RBS about the importance of IT. “RBS has given the impression that it does not see having a highly-skilled IT department as core to its business. This is a wake up call to RBS and other banks about having well-skilled and resourced IT departments.”

Read more - Faisal Islam on why RBS's computer glitch just got more serious