Water utility systematizes maintenance, operational procedures

Waterford Township standardizes workflow procedures with links to data from SCADA, CMMS, DMS, and other operational systems.
By Ian Blyth, Gray Matter Systems March 14, 2012

While the DPW covers more than 35 square miles in Oakland County, Mich., the control point is the main facility where all the field operational data is collected and analyzed. Courtesy: Gray Matter Systems and the Waterford Township DPWTo minimize the effects of escalating workforce retirements, the Waterford Township Dept. of Public Works (DPW) implemented a new work process management platform to link its SCADA, computer maintenance management (CMMS), and document management (DMS) systems, and to standardize workflow procedures.

Located in the center of Oakland County, Mich., Waterford Township is known as a lake-land paradise for its 34 lakes that surround the area, which cover 35.3 square miles. Over 71,700 Waterford residents rely on DPW for service on a daily basis. Its assets include 360 miles of water main and appurtenances, and an overall water and sewer infrastructure that exceeds 80,000 features that require maintenance and work orders.

Like many of its counterparts across the country, Waterford Township has been faced with losing a significant number of DPW staff to retirement, some leaving with more than three decades of water and wastewater experience. To mitigate the loss of all that institutional knowledge, DPW implemented Proficy Workflow work process management software from GE Intelligent Platforms.

“We were looking for a solution that would link real-time operational data in our SCADA system to our CMMS and DMS to create standard operating procedures and work orders automatically when conditions were met in defined workflow procedures,” said Terry Biederman, Waterford DPW director. “With SCADA continually sending real-time operational data through defined workflow procedures, the system automatically identifies abnormalities in operations and creates an electronic standard operating procedure (eSOP) and work order to ensure proper and timely correction.”

Biederman added that eSOPs also provide added benefits of “capturing the best institutional and professional methods for resolving the problem, which aids significantly as our workforce retires and becomes less experienced. One key aspect of the project was to get operating procedures standardized and in a format where DPW staff in the field, who might not be familiar with the system, could go in and follow the steps necessary to correct the issue.”

Now, any time an event occurs, DPW staff members assigned to follow up are presented with a step-by-step list of instructions on how to resolve the issue. Because the process is electronic, it also captures and logs the process for future reference and evaluation. 

To Biederman, Workflow provides a “middleware” solution that works in conjunction with Proficy iFix HMI and Historian, which support Waterford’s SCADA system, and Azteca Cityworks, its CMMS. Integrating these systems was an important operational improvement, providing a crucial bridge between the SCADA and CMMS platforms, Biederman said.

Linking these platforms also provided the benefit of automatic work order generation based on SCADA data that flowed automatically into Workflow. Prior to installing Workflow, DPW staff had to look through operational data manually for conditions that may have presented a problem, which alarming didn’t address. Once identified, work orders would be generated with very little, if any, eSOP to correct the problem. With Workflow, the SCADA data is monitored continuously and when deviations occur, appropriate workflows and work orders are automatically created, saving valuable staff hours.     

By utilizing real-time SCADA data and Workflow, Biederman said DPW staff members are directed in a focused manner to resolve obscure operational problems along with the root cause instead of never identifying the problem and/or root cause because that is the way it has always operated.

The work order screen explains the situation in plain English, including step-by-step instructions on how to diagnose the situation and make corrections. Courtesy: Gray Matter Systems

“Defined events are flagged automatically and consistently, forcing DPW staff to resolve them in a timely manner consistently,” Biederman said. “If you have a sewer pumping station with multiple pumps and one is starting a lot more than another, you have to first be aware that it is happening and then ask yourself, ‘Why?’ Workflow does this and creates an eSOP and work order automatically to correct the problem. The same is true for pump runtimes and required maintenance on pumps when they reach runtime maintenance thresholds. This system ensures that we are proactive instead of reactive in nature.”

Mark Meisel, director of water/wastewater solutions at Gray Matter Systems, agrees. Along with being a GE Intelligent Platforms authorized representative, Gray Matter Systems is an established provider of automation solutions to Waterford Township. “We’ve been working with Waterford Township for a long time, so to us, the Workflow installation just made sense,” Meisel said. “At the end of the day, it’s all about providing the best service to the public, and this project demonstrates DPW’s dedication to doing just that.” 

Implementation in phases

DPW implemented Proficy Workflow to provide the department with the ability to apply logic to SCADA values for work order generation, employ eSOPs, and create inspection forms for data collection.

The goal of the project was three-fold:

  1. Utilize Cityworks Work Order API (application programming interface) to integrate SCADA workflows with Cityworks
  2. Employ eSOPs to provide a method for documenting proper process and transference of institutional knowledge, and
  3. Provide auto task generation by creating a workflow component for SCADA.

The project was split into two phases, the first centered on automating workflows triggered by two types of incidents: those based on data coming in from the SCADA system, such as pump starts and runtimes, and other regularly occurring scheduled events with specific activities, such as sewer station inspections. The first phase also included integrating the work order API to automate work order generation fully.

Phase two of the project included integration of the department’s DMS, which provides staff with seamless links to documents, drawings, agreements, manuals, etc., from the archive that are used to develop additional workflows. The Workflow integration included creating four main components: the equipment model, events, Workflow templates, and schedules. Creating and using these components builds a process-driven workflow for managing by exception.

The department first modeled its system in Workflow by linking relevant database tags from the SCADA system. This allowed data to flow in real time from SCADA into defined Workflows. A trigger event was then created to initiate a defined Workflow based on conditional expressions or time-based factors.

Condition-based events use real-time iFix tag values transferred in such as pump starts and stops, pump runtimes, water levels increasing or decreasing out of range, or changes in flow. These events use values stored in the equipment configuration to evaluate expressions to determine automatically whether or not action needs to be taken.

With condition-based logic, events can be triggered based on multiple sets of criteria varying from a single subset to entire categories that must be met for a workflow to be triggered. Logical expressions can also be designed to evaluate criteria defined in the workflow process, such as whether one is already running for the same event.

Another type of event trigger is time. Time-based events use a data/time expression to determine when an event should be triggered, such as inspecting sewer stations at two-week intervals.

The next step was to create Workflow templates that contain procedures and steps for DPW staff to follow when completing the specified task. Procedures and steps can be executed automatically, such as having a pump turn on or off, or manually through interaction by the user. These procedures and steps can be modified by the workflow authors, and services can be added by the administrator to refine the process.

Finally, schedules are created by defining time-based activities within the Workflow template, if necessary.   

When a workflow is triggered, an e-mail is sent to appropriate DPW staff members to alert them of the assignment and provide them with the work order, if it is defined to be automatically generated. Those individuals can then begin to process the workflow and view all of the details involved. Real-time SCADA data related to the specific task can also be displayed to aid in resolving the event, as well as specific eSOPs and documents such as operational manuals.

The eSOPs outline steps to resolve the problem in a numbered format along with room for the operator to enter information and comments. The steps guide users through resolution of the issue and have expiration times in place. If a step is not completed in a certain amount of time, escalation processes such as supervisory notifications can occur.

At any time, DPW staffers can get a list of completed workflows and check a workflow history to view its details. Managers can develop workflows based on functions of their groups. They can also delegate tasks and see the status of operations by viewing pending workflows.

The staff is looking ahead to develop more Workflows to assist in automating preventive maintenance on equipment such as pumps. Examples include runtime-driven maintenance strategies, such as bearing lubrication and seal inspection. Instead of hoping the right individual will remember to perform this critical maintenance at the manufacturer’s recommended runtimes, a workflow can automatically be created and work order generated to ensure it will be done.

Biederman sums up the situation by observing, “Workflow is more about its potential than anything. If you have an operational process, it can most certainly benefit from a workflow designed to monitor abstract and varied functionality to ensure proper operation and efficiency. No doubt it takes work at the beginning to develop them, but when they are in place, hundreds of DPW staff hours are saved and effective and consistent operations are assured.”

Blyth is director of marketing for Gray Matter Systems.

By the numbers: Waterford Township DPW’s assets

Lest you think that the infrastructure of a township or municipality doesn’t match a more serious process manufacturing plant, consider the scale of this equipment spread over 35 square miles:

  • 24,000 customer water leads
  • 25,000 customer sewer leads
  • 360 miles of water main and appurtenances
  • 19 production wells
  • Two elevated and one ground storage tanks
  • 11 iron filtration plants
  • One high-service pumping station
  • 355 miles of sanitary sewer
  • 63 sewer lift stations
  • 15 township buildings
  • 230 vehicle feet
  • 3,500 gate valves
  • 3,400 fire hydrants
  • 7,000 water main segments
  • 11 water treatment plants
  • 8,300 sewer manholes
  • 8,500 sewer main segments, and
  • 64 sewer pumping stations.