More "Nines" Not Always Better

A mission critical facility (MCF) provides an environment where processes occur which are integral to an organization's viability. The MCF for a stock exchange includes its trading floor and related clearing houses. A bank's MCF may consist of a data center that processes billions of dollars of credit card transactions; the most critical portion of a hospital may be its operating and communicat...

06/01/2003


A mission critical facility (MCF) provides an environment where processes occur which are integral to an organization's viability. The MCF for a stock exchange includes its trading floor and related clearing houses. A bank's MCF may consist of a data center that processes billions of dollars of credit card transactions; the most critical portion of a hospital may be its operating and communication rooms.

Owners, engineers and contractors discuss the power availability of an MCF in terms of "nines," which defines the probability that a system will function at a future time. For example, a 4-nine facility has a 99.99% probability of power availability at a future instance, while a 5-nine building has a 99.999% probability of availability.

To achieve these levels, organizations spend considerable sums to enhance mechanical and electrical systems that sustain MCFs. In fact, some owners spend hundreds of dollars per sq. ft. to improve reliability because an infrastructure failure can lead to an unacceptable loss of revenue, or more importantly, a loss of life. Yet many owners miss the big picture in that many MCFs, even with a lot of "9s," will fall short of desired expectations because single points of failure are overlooked.

For example, a designer may have multiple feeds from the local utility in parallel with backup generators, but the feeders may terminate at a single switchboard. A less obvious example involves control components, such as programmable logic controllers and input/output modules, which are often specified without redundancy. Various components can fail within these devices and create an extended power outage.

At the same time, the various subsystems of an MCF should be designed to a commensurate level of reliability, because the overall availability of the facility is no greater than the weakest link.

The other major factor in MCF power failure has to do with operational deficiencies. The decisions and interactions of humans greatly affect the availability of systems. And while human error is the single biggest reason for MCF failure, this variable is often neglected by engineers when a system's theoretical availability is estimated, as it's difficult to quantify.

MCF owners are also guilty. While flipping for a costly infrastructure, they inadequately budget for the training of personnel and the development of coherent and consistent procedures and processes.

What should owners and engineers do? First, the acceptance testing and operator training process should commence at the early stages of the project. The design performance criteria, which include all failure scenarios, should be tested as a complete system prior to acceptance.

After the acceptance of the facility, strict procedures regarding change control, periodic predictive maintenance and facility work rules should be required. The lack of these measures can lead to outages, which are otherwise preventable. For example, it's not unreasonable that UPS systems may become overloaded over time; or that essential equipment, such as generators and backup pumps, may not start because of poor maintenance practices.

In sum, many 9s can still result in naught without reliability from more mundane system components and proper training. So when budgets factor, maybe one less 9 may ultimately mean greater reliability.



How to reduce the chances of MCF failure

Eliminate single points of failure

Dedicate the proper resources to commissioning, operation and maintenance

Maintain an adequate training budget



No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
Each year, a panel of Control Engineering editors and industry expert judges select the System Integrator of the Year Award winners.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Learn how to create value with re-use; gain productivity with lean automation and connectivity, and optimize panel design and construction.
Go deep: Automation tackles offshore oil challenges; Ethernet advice; Wireless robotics; Product exclusives; Digital edition exclusives
Lost in the gray scale? How to get effective HMIs; Best practices: Integrate old and new wireless systems; Smart software, networks; Service provider certifications
Fixing PID: Part 2: Tweaking controller strategy; Machine safety networks; Salary survey and career advice; Smart I/O architecture; Product exclusives
The Ask Control Engineering blog covers all aspects of automation, including motors, drives, sensors, motion control, machine control, and embedded systems.
Look at the basics of industrial wireless technologies, wireless concepts, wireless standards, and wireless best practices with Daniel E. Capano of Diversified Technical Services Inc.
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
This is a blog from the trenches – written by engineers who are implementing and upgrading control systems every day across every industry.
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Control Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.