Water treatment plant upgrades automation
Almost all water and wastewater plants that are more than a few years old have been through a control system migration—some more than once. Our water district in California started modernizing by installing a minicomputer-based SCADA system in 1986. In 1995, we decided to replace that control system with programmable logic controllers (PLCs), and we have never looked back.
During the 17 years from 1995 to the present day, we have been through several generations of PLC hardware, software, networks, and I/O. We have suffered trials and tribulations, and dealt with the same problems that water plants all over the world encounter every day. Here’s the tale of our control system odyssey.
Goodbye SCADA system
In 1986, our water district purchased a complete turnkey SCADA (supervisory control and data acquisition) system:
- A central VAX minicomputer and software from Digital Equipment Corporation
- Communication links using a LAN with RS422 running at 9,600 baud
- A telephone-based modem WAN running at 1,200 baud, and
- Two types of proprietary remote terminal units (RTUs).
This system controlled and monitored processes at the water treatment plant and the water distribution system. The SCADA application software ran on the VMS operating system and used a pair of redundant Micro VAX II computers and a pair of redundant communication front-end data collection RTUs. The cost of the system was $1.2 million in 1986. The system worked well, but over time the vendor was unable to provide compatibility with newer hardware and software. In 1993, the last available RTU for this system was purchased at a cost of more than $20,000.
In 1995, the control system migration to PLCs commenced with an update to the water treatment plant’s filter control system. In a typical water treatment facility, dual media filters are used to purify the water. These filters have a layer of charcoal and sand which helps capture any remaining particles that don’t settle out from earlier processing stages.
When the filter is online and producing water, the water flow is modulated based on filter level. The filter level will vary based on how long the filter has been on online and on how much water the plant is treating. After about 40 to 120 hours of runtime, depending on water quality, the filter must be backwashed. During the filter backwash, hundreds of thousands of gallons are normally used in a complex series of controlled steps. At our facility, about 335,000 g of water are used to wash a filter. About 35,000 g wash the surface of the filter, while the remaining 300,000 g are used in a reverse flow direction to lift and clean the filter media.
Like most filter control applications, ours has two main control modes. The first is when the filter is online and producing drinking water and the second is the backwash mode. The backwash process was controlled by 30-year-old hardwired relay logic and was starting to become unreliable. The 1965 relay-based controls seemed to hang up more in the backwash mode than in the online filter mode, and troubleshooting the relay logic was time-consuming and challenging, so it was time for an automation upgrade.
Because of cost and compatibility issues with the SCADA RTUs, our utility started looking at other options, and we evaluated several possible choices for filter control using PLCs or RTU technology. All of the leading PLC and RTU products were evaluated, and the choice was made to use the AutomationDirect DL340 PLC along with its DirectSOFT PLC programming software. This was the beginning of the end for the legacy SCADA system.
According to our head control system programmer, the most welcome feature of the PLC was the intuitive way in which the programming software worked. In 1995, most PLC and RTU programming software used a proprietary programming unit, or at best a PC-based DOS application.
By contrast, AutomationDirect and its partner Host Engineering in Johnson City, Tenn., had developed the PC-based DirectSOFT PLC programming software product. This Windows-based software was a superior alternative, both in terms of ease-of-use and compatibility with modern PC-based applications.
We replaced the timing and control relays on the filter automation system with a PLC that controlled both the filter level and the automatic backwashing system. The PLCs were interfaced to the legacy SCADA system through an existing RTU using Modbus. Over time, the filter control program has been modified to improve filter performance and water quality. The original PLC programming, as well as all changes, were done in-house as the system was easy to modify to meet our changing needs. In the last few years, the filter control system was upgraded to a DL250 PLC and programmed in state logic, as opposed to the ladder logic used in the older PLC. We’ve since found that state logic makes the automation system even easier to troubleshoot.
Replacing the VAX
In 1996, the plant’s SCADA hardware and software was upgraded to a Windows PC-based SCADA system, and the plant began replacing all the legacy RTUs with PLCs. The PLCs controlled several plant processes, from data collection and plant alarm to a very advanced chemical pacing system.
The PLCs were on a Modbus network running at 38.4 K baud. While this seems slow today, it was faster than a typical Modicon PLC could communicate at the time. One problem we encountered was the use of a Modbus tag server. When the SCADA system PC is talking to a PLC, the data is usually in a raw or BCD format, and PLC registers don’t match up cleanly to a Modbus register because the PLC uses an octal numbering system.
To solve this problem, we abandoned the PC SCADA provider’s tag server software and started to use DSData software from Host Engineering. DSData was able to read data using a native addressing scheme, and our SCADA system could use an item modifier so that data could be collected in a real number format. What this meant was that we no longer had to drop a “poke point” in BCD into the PLC and convert it to a real number in ladder, but could drive the memory location directly in a real number format. Another nice feature of DSData was that it could display the value that was being collected by the main SCADA software.
The PLCs and DSData software performed so well at the treatment plant that we decided to use PLCs in the distribution system and in an upcoming plant expansion in 1997. By this time, the water distribution system had more than 250,000 customers and covered a 50-square-mile area. We installed 50 PLCs to control tanks and pump stations and to act as the master PLC or data concentrator at the treatment plant.
The photo shows a typical distribution site control cabinet. The RTU was replaced with a PLC, the modem and interface box were replaced by a license-free spread-spectrum radio modem, and the battery pack and battery charger were replaced with an uninterruptible power supply. The use of a PLC in our pump stations has been a good decision in several ways. First, the PLCs have been easy to maintain. The PLCs use DIN rail mounting, removable terminal connectors on the cards, and a modular expansion design, making changes and upgrades easy to implement. In addition the hardware has been reliable and low cost, and has had excellent technical support.
In contrast, the RTUs were expensive and hard to maintain in a number of ways. One example of how difficult it was to work on the RTUs is that when a board failed, the RTUs didn’t have removable terminal blocks. The technician had dozens of wires to unscrew, tape off, and label before a failed board could be replaced.
The photo shows a PLC typical of those used in a four-pump pump station. The discrete input cards are rated at 120 Vac. There’s one four-channel analog input card bringing in suction pressure, discharge pressure, and flow rate, with a 4-20 mA output card to drive local indicators. The PLC drives relay outputs that interface to the motor control centers. The bottom communication port is configured as a Modbus master and is used to read tank level. The top port is used to communicate to a text panel.
From 1998 to 2002, the main treatment plant was expanded again, this time to 106 million gpd, and 11 more PLCs were installed as part of the plant expansion. Since 1995, the district has standardized on AutomationDirect for PLCs and related control system hardware. Currently, we have 31 PLCs at the treatment plant and 129 in the distribution system. The distribution system units are primarily used in the pumping stations. The end result is that a 106 million gpd water utility is running its water treatment plant and distribution system fully on PLCs.
As the plant expansion project was ending, the plant staff was able to convert from the old serial network to an Ethernet-based network. The Ethernet network was easy to cable up, uses standard hardware, and runs at high speed. Modern Ethernet switches run at speeds of 1 GB/sec, are inexpensive, and resolve CDMA collisions and other issues. This year we converted the PLC network to gigabit Ethernet technology, and have been wholly satisfied with the results.
By 2011, we’d replaced DSData with Kepware communication servers. We’re now using DirectSOFT 5 software, and the treatment plant’s network uses gigabit Ethernet technology. Most of our PLCs are now around a decade old and are working fine, indicating that the hardware has been very robust even in our demanding environments. To date, our district has purchased more than $250,000 worth of control equipment from AutomationDirect, so in 17 years of almost continual updates, expansions, and plant modernizations, we’ve spent less than the cost of the original control system purchased in 1986.
Henry Palechek is the information systems and process control supervisor at a water utility in California. He’s been at the utility more than 16 years and holds a California T3 Operator license.