On what was perhaps the busiest online shopping day of the year, both RBS and NatWest banks suffered glitches to their banking systems which left thousands of customers unable to access cash, online banking or smartphone apps right across the UK.
Falling on Cyber Monday, many customers turned to Twitter to vent anger over the glitches, which forced shoppers to stop buying for three hours from 6.30pm.
RBS, which owns NatWest, issued a statement on Tuesday morning via the social network saying: "The systems issues that affected our customers last night have now been resolved and all of our services are now back working normally."
The failure is the third problem to hit customers in 18 months, and RBS said on Monday night that it is "very sorry for the system issues that affected our customers this evening."
Both banks came under criticism in March following a hardware fault that left many customers in the same position as Monday, and a 'major computer issue' in June 2012 meant customers were unable to use their online accounts or withdraw cash for several hours, costing the group £175m in compensation.
RBS installed a new computer system in September as a response to these earlier glitches, but experts have warned that it is not enough.
Software analysis company CAST says it has consistently warned of the dangers of the software quality of banks, and after NatWest's glitch, it offered several pieces of advice.
"Western banking systems are particularly exposed because they were the first to install computer systems and much of the sector was badly wounded by the credit crunch's knock-on effects," said CAST.
"Business software is becoming increasingly complex, composed of sub-systems written in different programming languages, on different machines by disparate teams. This means no single person, or even group of people, can ever fully understand the structure under the key business transactions in an enterprise. Testing alone is no longer a viable option to ensure dependable systems.
"The frequency of technical issues affecting banking is a wake-up call to business leaders. They need to carefully scrutinize the structural integrity of their software systems."
Iain Chidgey, VP EMEA for Delphix, explained why the RBS failures happened and what banks should be doing to prevent this in the future.
He said: "Software glitches are becoming more and more frequent in the banking industry. We have already seen disruption to RBS and NatWest services and last night the two banking giants again suffered problems when customers reported problems with online banking, inability to pay for meals and being turned away from shops where transactions were declined despite adequate funds available.
"But why is this continuing to happen? Often the cause is insufficient testing. The databases in financial institutions are large and often more complex than in other companies. IT departments provide copies of databases for testing, but by the time a copy is available, the data itself is often old. In our data intensive world, data can be obsolete after only a couple of hours, but when refreshing just a single testing data set can takes days, the data will never be up to date enough for risk free testing.
"Companies need to make testing a priority and equip their IT teams with technology and resources that will enable them to test often and on recent data. One way, which is currently used by many financial institutions, is database virtualisation. Companies use it to create virtual copies of databases for the purposes of better application testing and recovery. Rather than creating a physical copy which costs a lot of money and time, virtual databases are created, allowing for testing on more current data, dramatically reducing the risk of failure such as the one some customers experienced on Friday."
Neil Kinson, VP EMEA at Redwood Software, addressed the impact that IT failures can have across different areas of the business.
He said: "Gone are the days where business IT issues went largely unnoticed by the outside world. Business processes are more connected now than ever before, meaning a backend failure can have a knock-on effect on frontline services. It's like a domino effect; a persisting failure in the billing department can lead to greater pressure on the provider's website and call centres, as customers look to question the mistake.
"If large brands like RBS wish to protect [its] customers from the impact of IT failure, greater effort must be made to ensure business critical processes work efficiently 100 percent of the time, both at the front end and the back end, to deliver accurate and reliable results. By automating key processes such as billing, stock control and credit checks, organisations dramatically streamline operations, reducing the chance of errors and freeing up valuable IT team resourcing to focus their efforts on other business critical tasks, such as innovation."
Visa Europe estimated that £450m was spent online in 7.7m transactions by the end Monday night, which would make this year's Cyber Monday the busiest online shopping day recorded in the UK.
Qualys is the leading provider of on demand IT security risk and compliance solutions - delivered as a service. Qualys solutions enable...