More "Nines" Not Always Better

A mission critical facility (MCF) provides an environment where processes occur which are integral to an organization's viability. The MCF for a stock exchange includes its trading floor and related clearing houses. A bank's MCF may consist of a data center that processes billions of dollars of credit card transactions; the most critical portion of a hospital may be its operating and communicat...

06/01/2003


A mission critical facility (MCF) provides an environment where processes occur which are integral to an organization's viability. The MCF for a stock exchange includes its trading floor and related clearing houses. A bank's MCF may consist of a data center that processes billions of dollars of credit card transactions; the most critical portion of a hospital may be its operating and communication rooms.

Owners, engineers and contractors discuss the power availability of an MCF in terms of "nines," which defines the probability that a system will function at a future time. For example, a 4-nine facility has a 99.99% probability of power availability at a future instance, while a 5-nine building has a 99.999% probability of availability.

To achieve these levels, organizations spend considerable sums to enhance mechanical and electrical systems that sustain MCFs. In fact, some owners spend hundreds of dollars per sq. ft. to improve reliability because an infrastructure failure can lead to an unacceptable loss of revenue, or more importantly, a loss of life. Yet many owners miss the big picture in that many MCFs, even with a lot of "9s," will fall short of desired expectations because single points of failure are overlooked.

For example, a designer may have multiple feeds from the local utility in parallel with backup generators, but the feeders may terminate at a single switchboard. A less obvious example involves control components, such as programmable logic controllers and input/output modules, which are often specified without redundancy. Various components can fail within these devices and create an extended power outage.

At the same time, the various subsystems of an MCF should be designed to a commensurate level of reliability, because the overall availability of the facility is no greater than the weakest link.

The other major factor in MCF power failure has to do with operational deficiencies. The decisions and interactions of humans greatly affect the availability of systems. And while human error is the single biggest reason for MCF failure, this variable is often neglected by engineers when a system's theoretical availability is estimated, as it's difficult to quantify.

MCF owners are also guilty. While flipping for a costly infrastructure, they inadequately budget for the training of personnel and the development of coherent and consistent procedures and processes.

What should owners and engineers do? First, the acceptance testing and operator training process should commence at the early stages of the project. The design performance criteria, which include all failure scenarios, should be tested as a complete system prior to acceptance.

After the acceptance of the facility, strict procedures regarding change control, periodic predictive maintenance and facility work rules should be required. The lack of these measures can lead to outages, which are otherwise preventable. For example, it's not unreasonable that UPS systems may become overloaded over time; or that essential equipment, such as generators and backup pumps, may not start because of poor maintenance practices.

In sum, many 9s can still result in naught without reliability from more mundane system components and proper training. So when budgets factor, maybe one less 9 may ultimately mean greater reliability.



How to reduce the chances of MCF failure

Eliminate single points of failure

Dedicate the proper resources to commissioning, operation and maintenance

Maintain an adequate training budget



No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
Each year, a panel of Control Engineering editors and industry expert judges select the System Integrator of the Year Award winners.
Control Engineering Leaders Under 40 identifies and gives recognition to young engineers who...
Learn more about methods used to ensure that the integration between the safety system and the process control...
Adding industrial toughness and reliability to Ethernet eGuide
Technological advances like multiple-in-multiple-out (MIMO) transmitting and receiving
Big plans for small nuclear reactors: Simpler, safer control designs; Smarter manufacturing; Industrial cloud; Mobile HMI; Controls convergence
Virtualization advice: 4 ways splitting servers can help manufacturing; Efficient motion controls; Fill the brain drain; Learn from the HART Plant of the Year
Two sides to process safety: Combining human and technical factors in your program; Preparing HMI graphics for migrations; Mechatronics and safety; Engineers' Choice Awards
The Ask Control Engineering blog covers all aspects of automation, including motors, drives, sensors, motion control, machine control, and embedded systems.
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
News and comments from Control Engineering process industries editor, Peter Welander.
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
This is a blog from the trenches – written by engineers who are implementing and upgrading control systems every day across every industry.
Anthony Baker is a fictitious aggregation of experts from Callisto Integration, providing manufacturing consulting and systems integration.
Integrator Guide

Integrator Guide

Search the online Automation Integrator Guide
 

Create New Listing

Visit the System Integrators page to view past winners of Control Engineering's System Integrator of the Year Award and learn how to enter the competition. You will also find more information on system integrators and Control System Integrators Association.

Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Control Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.