The System Testing Conundrum

RBS and NatWest in the United Kingdom and Ulster Bank in the Republic of Ireland have all suffered several high profile technology failures in the past year, with the most recent glitch coming in March. The system problems have seen customers unable to withdraw cash, pay bills or even make critical transactions like mortgage repayments. These incidents only serve to reinforce the lesson that poorly implemented systems can have dire consequences for a financial institution's operations and reputation.

It's not clear exactly what went wrong at the banks mentioned above, so it's hard to know whether a more stringent system testing program would have made a material difference in these situations. It certainly demonstrates, however, that making sure your systems operate properly is paramount -- before going live. Having an efficient and accurate system testing regime therefore is critical to this process.

The survey (PDF) conducted by SunGard Global Services reflects the growing realization of system testing's importance. For instance, the survey shows that both the amount of time allocated to system testing programs and the amount of budget seems to be increasing. Another -- more important sign -- was the impact that system testing, once underway, has on a firm's day-to-day operations. This importance was placed at "medium" to "high" by the vast majority of respondents.

[For more on how financial firms are approaching systems testing, read: Why Public Opinion Matters, Even For System Testing.]

Given the issues poorly running technology can create, this is a troubling recipe. Increasing costs, longer timeframes, and high levels of disruption do not add up to efficient systems. It's not surprising, therefore, that most respondents said there was an appetite to improve the way system testing is carried out in their organizations.

The survey also looked at the types of people involved in system testing. I've discussed this in greater detail in a previous article, but the points I raised then are relevant here also. System testing, it appears, is being carried out to a large extent by staff unskilled in the area -- staff who, at a time of job cuts in banking, are an increasingly precious resource whose expertise should be better allocated.

Problems will always occur from time to time but if technical problems are to be minimized in any way, then organizations need to examine their system testing practices carefully. How testers can reduce the impact system testing has whilst improving its effectiveness and efficiency should be a key concern and point of interest for all involved.

Stephen O'Reilly is Quality Assurance & Testing Practice Lead at Sungard Global Services.