Capita engineer travels 100+ miles to help fix outage
Updated 12:20 with comment from London Fire Brigade, Capita. CORRECTS outage from Sunday morning to Saturday morning.
London Fire Brigade’s 999 control room suffered a major IT systems outage early on Saturday morning, taking down primary mobilisation systems.
Control room staff were forced to resort to taking pen and paper notes, and patching calls through to local fire stations manually.
Sources said a Capita engineer was sent 100+ miles from Portsmouth to Stratford to help fix the issue. They claimed it took until 13:00 for all systems to be running smoothly again; in total a nearly eight-hour incident.
London Fire Brigade confirmed the 05:00 outage but said the mobilising system was “available [again] from 09:00”.
IT failures can significantly slow how fast fire crews get to an incident, with similar incidents in the North West delaying response times by up to eight minutes, the Fire Brigade Union (FBU) noted on Friday; as it listed a series of failures around the country of Capita-supplied systems.
999 Fire Control Room: Subsidiary Systems also Crashed
London’s 999 fire control room mobilisation systems crashed at approximately 05:00 on Saturday morning, two sources told Computer Business Review.
So did subsidiary software systems BOSS and IMapping, which have a range of functions including incident recording and remote online login.
Mobilisation systems display type of incident, location, and resources needed to help tackle it. They help control deploy the closest crew with the best appliances for the job, optimising visibility and efficiency.
Gareth Beeton, London regional chair, Fire Brigades Union, told Computer Business Review: “These issues can significantly increase the time [it takes for] calls to be sent to crews to attend incident, costing valuable seconds in a fire or any incident, and increasing risk to Londoners and London fire fighters.
He added: “The London FBU will be asking the London Fire Brigade for reassurances that this does not happen again.”
London Fire Brigade is one of the largest firefighting and rescue organisations in the world. It protects lives and property across the 1,587 square kilometres of Greater London.
A spokesman told Computer Business Review: “At around 5am on Saturday, 15 February, the Brigade’s control centre experienced issues with the mobilising system and immediately began using its standard secondary mobilising procedure.
“The London Fire Brigade has well-practised contingency plans in place to ensure that the ability to respond to incidents is not impacted should there be any issues with the system.
“The mobilising system was available from 9am.”
A Capita spokesperson said the issue was “being fixed remotely; an engineer was sent to support this work.”
Litany of Failures
The incident came just two days after the FBU warned that botched IT systems “upgrades” were putting lives at risk across the country.
On Friday the FBU said it had issued five Safety Critical Notices to fire control in the North West and East Coast over similar IT issues that were “seriously impacting the ability of understaffed control rooms to mobilise firefighters, fire engines, and other emergency resources to incidents.”
Another source told us that mobilisation system IT failures on the south coast meant brigades on the mainland were being mobilised for incidents on the Isle of Wight “as the systems don’t recognise the channel.”
The issues the FBU raised on Friday included:
- Failures in the mobilising system to document and record addresses, requiring staff to log off and on again before mobilising resources
- The mobilising system slows down to “unusable” levels during incidents on motorways
- Attempts to issue resource proposals are met with frozen white-out screens
- Risk-critical information is not properly communicated to control staff, such as whether the occupier is a hoarder, whether there has been a firearms risk, whether there is a risk of violence to crews
- Failure to send critical emails including arson threat referrals and accident/injury report forms, due to a migration to a new Microsoft Outlook server
Lynda Rowan-O’Neil, FBU Control Staff national committee, said: “We’ve repeatedly warned that these IT failings are dangerous, but have been ignored.
“Our control rooms are desperately understaffed and conditions have become completely untenable. The control room mergers involved massive cuts to staff numbers, which have seriously undermined our ability to handle the overwhelming volume of storm-related calls.”
She added: “We cannot keep allowing outdated, malfunctioning IT systems to delay emergency response. It’s no secret that, in an emergency, a matter of seconds could be the difference between life and death.
The affected control rooms cover some of the areas worst hit by recent storms, as well as Manchester International airport and Sellafield nuclear facility.
The incidents follow a recent report from the Home Office that found fire services now take 11 seconds longer to reach a “primary” (major) fire, compared with last year. Response times are at the slowest since current records began in 2009/10. Firefighters now take two minutes and 42 seconds longer to respond to a primary fire, compared with 1994/5.