Are your back up systems what they are - Back up??

The recent fire in the Worlds busiest airport at Atlanta in USA highlights the need for properly designed back up systems. Power supply in Atlanta airport was out for about 11 hours on Sunday due to a fire. The fire occurred in a tunnel that housed both the main power lines and a backup supply, news reports suggest.

The fire was traced to a failed switchgear, and was adjacent to the back up power supply cables whcih too got burnt!Have you made sure that your back up systems are just that - back up and always available when needed? It may be too late to find out during an actual emergency.

When the back up fails........

 When I was working in Saudi Arabia, the 3000MTD gas based methanol plant was operated with state of the art DCS systems. Just before a planned turnaround, instrument personnel were working with some control room underfloor cables when, by mistake, all power to the DCS displays were lost. It was restored after 30 minutes. The plant parameters were maintained throughout the upset without any safety issue as the controllers in auto were doing their job. However a case study presented in 2012 by Shri KC Tripathy and others of NTPC shows how safety hazards cropped up during "complete and simultaneous DCS failure in two 500MW units"
The case study mentions among other useful learnings, the following:
"All processors (active and redundant) abruptly rebooted at once. Thereby both redundant network & redundant processor concepts of DCS design were defeated".

 Read the case study in this link.



Contribute to the surviving victims of Bhopal by buying my book "Practical Process Safety Management"

Are your back up systems available?

On Thursday 14th January, it was reported that the entire radar systems at the Air Traffic Control (ATC) at the Indira Gandhi International airport at Delhi failed in the evening for about an hour. It was also reported that no takeoffs or landings could take place for more than two hours following the systems crash. The back up system also failed.The ATC system was finally restored at around 2030 hrs after reloading the system, with the entire procedure taking about three hours. The radar system collapse led to massive delays.
The above incident raises the importance of keeping back up systems always ready. A back up system is just that – it must back up when needed. Obviously something went wrong with the back up also. The only silver lining is that there must have been procedures for manual operation which was obviously put into place as there was no safety of flight issue. How good are your back ups for power supply for the DCS system? How often do you check them? Do not think that it will not happen. I had the personal experience of all 7 DCS screens of a methanol plant operating at full capacity, going blank all of a sudden! Develop procedures and train personnel for handling such situations.