What controller fits your application?
The choices for a controller are as diverse now as they have been at any time in the history of industrial control systems. Many types of controllers are available for discrete, process, and hybrid control applications, including programmable logic controllers (PLCs), programmable automation controllers (PACs), distributed control systems (DCS), industrial PCs (IPCs) or PC-based control, and embedded (board-level) controllers. And the lines that differentiate them continue to blur. What criteria should be used to choose among them and why?
As control system engineers who have designed many utility-based control systems, we see a large variety of controllers even within the same plant. This diversity seems to indicate a lack of standards, which leads to a lack of efficiency and reliability with regard to installation and ongoing maintenance of control systems. Because controllers sometimes last for more than 20 years, they often are forgotten behind gray panel doors where no one sees them and they tend to run until failure. Out of sight, out of mind, as the saying goes. Many times, there are plant expansions or retrofits where a vendor or contractor may install a later version of a controller-or even change to a different controller-guided more by personal preference than overall control system maintenance. Such was the status of operational technology (OT) for many years. Reliability was the key. The emergence of Windows-based human-machine interfaces (HMIs) required controllers to add Ethernet connectivity, but even then, control systems were largely isolated from the rest of the business. [subhead]
Controls for the 21st century
However, Internet connectivity now drives many technology decisions. More people are relying on smartphones and tablets to access data while on the go. Information technology (IT) changes have exploded and OT and IT are converging. The Industrial Internet of Things (IIoT) also has moved to center stage and businesses need to combine OT and IT to make it work. Control systems engineers are well-poised to help this merger because they understand plant floor operations as well as real-time data that operates in millisecond timeframes.
The convergence of OT and IT now drives the decision as to which controller best fits a particular application. Originally, control system engineers and technicians decided which controller best fit their application and others in the organization didn’t interfere. Hardware selection was based on factors, such as processing capability, controller memory, and modularity where input/output (I/O) could easily be added as needed. Consideration was given to availability of spare parts or support. It was up to the programmer or system integrator to use the device with which they were familiar. Sometimes as long as a device was "Ethernet capable" or "Modbus capable," that was good enough and that is why companies ended up with so many different controllers.
That decision model is no longer suitable for industrial control system (ICS) technology because it now evolves so quickly. And the data affects and requires input from more people. As a result, control system selection priorities have changed.
Historical controller selection criteria
In the past, criteria for control system selection decisions focused primarily on hardware and how it could be supported. They included:
- A hardware platform based on central processing unit (CPU) power and memory
- Scalability with modular I/O
- Availability of spare parts
- Software programming languages
- Programming and support.
Modern controller selection criteria
Today, control system selection decisions focus less on hardware-centric criteria and more on networking, security, and standardization. Modern ICS selection priorities include:
- Control systems with inherent networking and embedded cybersecurity
- Robust programming languages for rapid application development
- Scalability, standardization across the enterprise, and reduced part count
- Changing protocols
- Expertise on hand for support and quick modifications, and features to meet changing business needs (in-house versus an engineering/system integrator).
Control systems with inherent networking capabilities that include embedded cybersecurity features: It’s hard to imagine not having data at our fingertips. That would not be efficient. Companies need staff at all levels to have access to real-time data about operations-from the plant floor to the C-suite. Air gaps are no longer viable. In fact, as long as humans are involved, no true air gap exists. Controllers have been introduced within the past year that have deep encryption methods, yet can continue to provide local control functionality during denial of service attacks on their communications capability. They also may include component supply chain manufacturing security keys to ensure the highest levels of hardware and software authentication.
These features alone have changed the game in terms of control systems. A manager of a utility made this eye-opening statement, "Now, the control system decisions are no longer made just by operations and technicians staff, but it is up to the management to protect our utility’s infrastructure by selecting secure control systems platforms." He was wise to arrive at that conclusion, move to new control system standards, and be willing to present that challenge to the utility’s board so it could take proper action.
Robust programming languages for rapid application development: By having a complete tool set for controller programming, the integrator already has many advantages for efficiency and reliability. These are best expressed in IEC 61131 with function block (FB) diagram, structured text (ST), sequential function chart (SFC), and ladder diagram (LD). Having open (nonproprietary) programming standards that can now be incorporated with deeply-embedded security keys will aid control system engineers and system integrators to confidently deliver the solutions that businesses need. Be sure to check the licensing cost of your programming software. Some manufacturers are breaking free from the licensing models and offering integrated development environments that are free, allow immediate downloading, and can be used immediately.
Scalability, standardization across the enterprise, and reduced part count: If this appears to be competing interests, check again. At least one manufacturer whose systems we install has broken the mold when it comes to leveraging what was once competing standards. Scalable architecture with one backplane (in five-slot, 10-slot, and 20-slot options), one power supply, one blazing fast CPU module, and a few I/O modules (including a universal module that is software-selectable as input or output and discrete or analog). The platform is scalable to any size while reducing part counts to a dozen or so components, rather than having to select from thousands of part numbers like some vendors expect of you.
Standardizing on one platform throughout the enterprise can have a significant benefit. Moving data between controllers for proper automation functions can be a hassle. Having a single platform that can reach from field remote terminal unit (RTU) all the way up to plant controllers with DCS functionality of programming and HMI integration could yield benefits for years to come.
Even the protocols have changed: Even as fast as software applications change, historically, the base-level protocols were fairly constant-not so anymore. Publish/subscribe protocols have changed the way we implement networks. No more hammering the networks by polling for data continuously. With the pub/sub model, we have the ability to push the data-when changing data necessitates-to anywhere in the enterprise: to the ICS, HMIs, enterprise resource planning (ERP), inventory control, and so on. Protocols that do this are available today with message queuing telemetry transport (MQTT) and are in process with OPC UA (OPC-UA stands for OLE for process control, unified architecture).
The right expertise: Having the right expertise on hand to implement modern control technology is still key. Whether that expertise is in-house staffing, or outsourced engineering and integration, users will benefit from having a team that is responsive and agile at quickly meeting business needs. Training in industry standards such as IEC 61131-3 also will be a plus.
There are many disruptive technologies that affect the ICS. We are asking more of the industrial control system ICS than ever before. Staying with the status quo is no longer good enough. For digital natives who grew up in this space, the selection criteria will be second nature to them. This requires us to think differently about control system platforms, the vendors, and partners to determine which systems can now meet these challenges as we look ahead to increased growth, reliability, and cybersecurity of the ICS and the infrastructure it controls. It is time to stop thinking about hardware and start demanding the open and secure automation platform and tools required for the digital enterprise.
Read the case study to learn how a utility company upgraded their SCADA and RTU.
Case study: Utility upgrades SCADA and RTU
Clarksville, Ark. needed to implement water utility SCADA and HMIs as well as take advantage of the communications inherent in the protective relays that safeguard electrical circuits from abnormal conditions. Likewise, power metering equipment had to be integrated. Because the integration of electrical and power technologies together would be phased in over many years, the solution also had to be easily expandable.
Clarksville Light & Water Co. (CLW) is a municipally-owned utility that has been serving the Clarksville, Ark. community at not-for-profit rates since 1913. With annual revenues of approximately $25 million, it provides retail electric, water, and wastewater services to the community as well as wholesale water to eight other cities and water districts in the region.
A small town with a big vision, the city now is implementing a plan to realize as much as $2 million in savings through improvements in reliable, predictable, and consistent utility service levels, which is expected also to attract new investment from the business community and stimulate job creation. The plan included enhancing its municipal infrastructure with 16.7 miles of 288-strand fiber-optic cable in redundant loops throughout the city as part of a long-term vision for locally-owned and operated fiber optic networking capability.
CLW general manager John Lester also saw this as an opportunity to leverage overdue monitoring and control of the city’s four electrical substations, and water treatment and distribution operations, and eventually its wastewater utility systems, which include supervisory control and data acquisition (SCADA) and human-machine interface (HMI) implementations (see Figure 1).
Wireless becomes a challenge
In the short term, CLW used wireless radio communications. However, it soon became apparent that wireless could not deliver the reliability and security that a critical infrastructure demanded. The fiber network communication system, solved the communications problem with a reliable, cost effective, and secure network and the challenge was to find the most economical way to operate a SCADA system across it. It was also necessary to protect the electrical circuits from abnormal conditions. Protective relays were in place that communicated via a serial port, but this connection was not being used. Likewise, power metering equipment had to be integrated. And because the integration of electrical and power technologies would be phased in over many years, the solution also had to be easily expandable.
"We had an immediate need to implement water utility SCADA and HMIs and in the short term had decided to use wireless radios," said Lester. "However, it became apparent that wireless couldn’t deliver the reliability and security that our critical infrastructure demanded. A fiber network communication system was the most reliable, cost effective, and secure network we could put in place. It also offered the potential to layer in other revenue generation services, both internally and externally."
Solution: One platform for water and power
Clarksville, Ark. selected a control system from San Jose, Calif.-based Bedrock Automation. Bedrock-certified solution provider Brown Engineers LLC, Little Rock, Ark. completed the installation in the second quarter of 2016 and the system is now in full operation.
The new control system provides a single, cybersecure platform for SCADA remote terminal unit (RTU) monitoring and control of its electric, water, and wastewater utilities. The new control platform is paired with Inductive Automation’s Ignition software platform to deliver an integrated, secure open systems solution that enables CLW to proactively manage critical infrastructure assets both onsite and remotely (see Figure 2).
"The fiber optic network gave us a way to tap the new functionality for our remote operations and when we learned that choosing the new control system as our RTU would mean that military-grade cybersecurity was already built in," Lester said. "We also saw a very cost-effective way to reduce cyber risks while addressing the looming North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection (CIP) compliance requirements."
Many utility managers are trying to understand what they are supposed to do about cyber threats. This new control system, which embeds authentication into the hardware layer, offers the most hardened and easily-implemented solution we’ve seen.
The new control platform can be deployed as a PLC, DCS, SCADA, or RTU, which allowed CLW to simplify and standardize on a secure control environment across all its departments (see Figure 3).
Extending the new SCADA deployment to the grid offers a simple and effective way to add secure control and monitoring of remote assets. At each of CLW’s substations, a new controller directs and protects data transfers along the new city-wide fiber ring. Its design allows all five of the new controllers to connect directly to the fiber optic cable without the need for additional connectors. With this foundation in place, CLW now plans to use automated circuit switching in addition to monitoring and load balancing on the local grid. This also presents an opportunity to realize demand-side management for the electric utility. CLW expects to realize improved overall reliability, shortened response times, and reduced power supply costs to the utility and its customers.
We wanted each substation RTU to have enough horsepower to aggregate all power meter data and protective relay data for sequence of event recording (see Figure 4). We also considered future development needs of power management techniques that allow for demand management and load shedding controls. The new controllers provide those features in an easy-to-manage integrated development environment (IDE) as well as built in cybersecurity protections embedded at the hardware level.
Protective relays monitor electrical parameters to detect abnormal conditions that could result in damage to assets (see Figure 5). At CLW, the existing protective relays were capable of communicating via a serial port, but this connection was not being used. Brown Engineers developed an application that extended the communication libraries provided by the Bedrock IDE to allow communication with the existing protective relays. A model of the protective relays was created in the Bedrock IDE, and a corresponding model was created in Ignition, the software CLW used from Inductive Automation. Both protective relay models were connected by configuring a few communication parameters and then the data from the protective relays was made available to Ignition software where it is displayed, stored, and used to create alert notifications (see Figure 6).
The same methodology was used to integrate power metering equipment. Design elements common to both protective relays and power meters were uniformly incorporated into both models, and only non-common design elements were developed in each respective model. If the models for the protective relays and the power meters had been more similar, a separate model would have been created with the common design elements and this model would have been extended to provide the specific requirements for protective relays and power monitors.
Reaping the benefits
The flexibility and power of this solution will pay dividends into the future. In the near term, CLW intends to introduce localized backup power generation during peak times by extending the new control platform into key nodes, such as backup generators installed at hospitals or other major facilities. It’s all part of Clarksville’s intelligent grid vision for better managing power demand costs and reducing energy loss.
With the new system in place, CLW now controls functionality via remote access for RTU sites as well as 24/7 automated and on-demand remote monitoring of key assets. That kind of connectivity enables the utility to optimize asset maintenance actions through custom email and text alerts based on real-time data. The system already is generating results. "This system has reduced overtime-a direct savings," said Lester. "Problems can be identified, sometimes even solved, without having to physically be at the plant or in the field. That translates into cost savings, improved reliability, and higher customer satisfaction."
In addition to cost savings, it’s improving personnel efficiency and increasing public safety. "I was away from the office when two low-level tank alerts popped up on my tablet. That saved at least an hour responding to a water line break, helping us prevent low-pressure areas or a dry-tank event, which would have triggered a boil order for the public."
See an additional case study on how a water utility upgraded their control system.
Case study: Water utility upgrades control system
Russellville City Corp. has provided Russellville, Ark. with quality water and wastewater systems for approximately 25 years. The utility oversees 25 square miles of the water and wastewater systems in Russellville, serving more than 20,000 customers and supplying drinking water to residents living within an additional 240 square miles outside the city.
Challenges of obsolescence
The controls for the city’s treatment facilities were becoming obsolete. One of the PLCs running automatic control of the digestion blowers, clarifiers, sludge pumps, and chlorination chemical feed pumps had failed and it was necessary to replace it. Instead of replacing it with more of the same, Steve Mallett Jr. PE, general manager of Russellville City Corp., wanted something that would meet future demands. Driven in part by concerns of a former U.S. military general who sat on the corporation’s board, he especially was looking to protect the city’s water supply from cyber threats.
"The PLCs running the automatic control of our digestion blowers, clarifiers, sludge pumps, and chlorination chemical feed pumps had become obsolete," said Mallett, who is responsible for the Arkansas pollution control facility. "When one of them failed, we wanted to replace it with something that would provide a path to the future."
The Russellville, Ark. water and sewer system, operated by Russellville City Corp., replaced its legacy PLC with a secure industrial control system from San Jose, Calif.-based Bedrock Automation. Bedrock-certified solution provider Brown Engineers LLC, Little Rock, Ark., completed the upgrade in November 2015.
Although it mostly performs the same functions as the legacy PLC, the new system is quite different from the legacy PLC. Instead of a traditional pin-based backplane, it uses an electromagnetic backplane, which eliminates pin corrosion and breakage, improving long-term reliability and enabling embedded security by preventing the possibility of using counterfeit input/output (I/O) modules. The electromagnetic backplane also creates a galvanic isolation barrier between field wiring and the controller, and provides a high-performance, deterministic I/O update rate to support current functionality and additional planned expansions. The new system also is different from the legacy PLCs in the following ways:
- The new system runs a military-grade safe and secure real-time operating system, further embedding security into the software and firmware used to control the facility.
- It can operate from 90 to 260 V ac or dc power without fans or dual inline package (DIP) switches for simplicity and robustness; and embeds standard, open system technologies, including OPC UA, a fully compliant IEC 61131-3 programming environment, and standard Ethernet support at the control and I/O networks.
- The new control system consists of only a dozen part numbers, reducing installation and maintenance costs.
- The system is scalable for more advanced control functions, such as serving as a supervisory control and data acquisition (SCADA) system, remote transfer unit (RTU), or distributed control system (DCS).
The new controller is part of an integrated plantwide SCADA system. Users connect to that system via Ignition software from Inductive Automation. The software is an industrial applications platform that coordinates control and data acquisition for all plant PLCs and RTUs.
There is an increased interest in cybersecurity among municipal utility clients, such as City Corp. Many executives at these organizations want to control security functions from their tablets and control centers because their networks are getting hammered every day by probes and attempted intrusions. Russellville’s new controller gives the company another layer of protection beyond firewalls and virtual private networks (VPNs). It is unique in that as it powers up, it checks to be sure that all hardware and software components are validated. Regular PLCs just can’t do that.
According to Mallett, since the upgrade was completed, the system has been performing to specifications and running without issue.
Dee Brown, PE is principal and co-founder of Brown Engineers LLC, Little Rock, Ark. He manages projects involving industrial, municipal, and commercial facilities for power distribution, software based controls, radio telemetry, video surveillance, access control systems, and networks. During his 25 years of electrical engineering project experience, he has specified and supervised numerous generator installations and generator system improvements, both as stand-alone projects and as components of much larger construction jobs. Brown has a BSEE and an MSEE from Louisiana Tech University.
– See other articles from the supplement below.