DCS, SCADA, Controllers

 

Process manufacturing insights

• A pharmaceutical company wanted to integrate a control system into an existing process cell, but an overhaul of the process was needed to make this work.
• A wrapper logic was created in phase manager phases to allow the software suite to control the existing equipment models and sequences.

---

A pharmaceutical company was in the process of commissioning a control system for a process cell and asked ECS Solutions, a system integrator, to provide site support to complete the commissioning and qualification.

The control system supplier had taken a simplistic approach to keep costs down. One controller with multiple human-machine interfaces (HMIs) provided the heart of the control system that also interfaced with some original equipment manufacturer (OEM) skids. The control hardware consisted of a controller with multiple HMIs. The controller code was all custom with little use of off-the-shelf products and was an assortment of custom blocks of code tied together, providing minimal functionality.

Project preparation: Operators, sequence, procedures, documentation

Little thought had been put into operators and how they would run the system to create quality product. Standard operating procedures had not yet been developed. Producing a quality product was dependent upon the operator selecting the correct sequence of functions from operator stations at multiple vessels. This approach exposed the company to risk in producing a quality product and controlling the rate of production.

Since procedures had not been considered to this point, the system was not well documented and inevitably in store for substantial changes through the commissioning and qualification phases of the project.

As the owners started to think practically about how operators would make product, change requests started pouring in. Changes to the custom logic were challenging. It was difficult to determine what effect small changes would have in other areas of the program.

The integrator evaluated the situation and recommended a migration to the control system, providing batch software for the batch automation with process objects in the controller replacing much of the custom code.

Procedural control, parameter transfers, data gathering

The probability of errors in production led the integrator to recommend many activities carried out by the operators be significantly reduced. This was achieved by automating the operator activities related to procedural control, parameter transfers, and data gathering of the system. The integrator proposed the software be integrated into the system to provide a layer of automation that would reliably set the parameters for the equipment module and sequences at the right time, every time. The company agreed to add this layer of automation but requested the implementation of the software not interfere with the ongoing activities and all existing code and functionality be preserved without any changes being made.

With the proposed integration, the operator would be required to simply select a recipe (stored by the sequencing engine) and select which equipment was needed to make the batch. The sequencing engine coordinated all activities, including the transfer of parameter values, and the capturing of reports information. Furthermore, the system prompted the operator when a task required operator interaction. This work was carried out in such a way as to have no impact on the functionality of the existing equipment modules or sequences.

Equipment model, phase classes, phases

To do this, the integrator created an area model in the software suite to represent the existing equipment in the process area, with phase classes and phases representing the individual PLC equipment modules and PLC sequences.

A wrapper logic was created as needed in phase manager phases to allow the software suite to properly control the existing equipment models and sequences. This resulted in a system capable of executing recipes without parameters being entered manually or equipment being started by an operator while capturing all pertinent report values. This improved the reliability and repeatability of the existing system while reducing human error.

At this point to make a batch recipe, the integrator created procedures to encompass everything that needed to be done with every piece of equipment. Phases were created that could talk directly to the coordination sequences or coordination equipment modules. A wrapper was created for all their equipment modules, which lacked some of the flexibility required by the company. It should be noted the control system was integrated with the batch software, including key procedures and direct signatures. Existing custom code was replaced with a process objects standard to provide a more reliable and sustainable solution.

Global standard library, standardized objects

The absence of a standard process object was the major driver for the migration. The custom code in the existing system had some limitations and issues in terms of coding, and later the control module was migrated.

However, the equipment module layer remained standard since it was clearly commissioned. Some problems with the control module layer were identified – inconsistent logic, absence of signal filtering, and software anomalies – so the entire standard was replaced by a global standard library custom code, with standardized objects in place of the custom code.

A significant amount of input/output (I/O) was migrated, and at the same time, the integrator was commissioning other systems. It became critical to maintain the timeline as the code was replaced and ensure it was thoroughly tested prior to validation. This required a risk response plan, recognizing errors due to migration would occur. Each controller was tested in a similar environment to prevent issues in terms of coding and ensuring sufficient memory in the processor to perform the migration.

Simulated automation system, testing before live

In terms of architecture, the integrator initially created a simulation system at their facilities, which allowed for testing and building recipes, and a test could be performed before implementing them in the live system. The simulation system was identical to the production system, giving latitude to things that could be done, and implementation was less invasive in the current architecture.

Three different workstations, two of which were automation workstations, provided access to batch software and how the system was configured. At the automation work stations, recipes could be edited, changes made to the HMI, or the controllers could be configured. Thin manager clients are available, and it is only necessary to provide the IP address and credentials of the thin manager server to automatically withdraw information so deploying a new client or replacing an existing client is quite straightforward. Every PC is virtualized with VMware and there is an ESXi post maintaining all machines, including the HMIs.

In addition to the batch reports, the integrator also collaborated with the pharmaceutical company to produce custom reports. The reports show triggered operations and whether the operation passed or failed. The reason for a failure is also recorded. The major components of the reports are a recipe information section and a unit procedure summary. These sections capture details of interest regarding specific operations. The reports are created in a format used by the company.

Changing role for control system integrator

The integrator’s role in the project changed as time progressed. Initially, the aim was to put a wrapper on the existing code and not modify it, that is, the original agreement with the company. Subsequently, it was decided to make changes to improve the system, due to the lack of required functionality, and today the company can make extended batches, considered to be active production batches while the product awaits FDA approval.

The batch software also has enabled a wealth of historical data to be logged in the form of batch records and reports, including electronic signatures. The modification has allowed the company to maximize the utilization of its equipment. Using a feature in the batch software, electronic prompts allow operators to review and respond, including provision of detailed photographs of equipment setup for operators’ absolute activity clarity.

Using the software suite recipes to drive clean-in-place (CIP) processes simplified the use of the CIP system and established repeatable consistent cleaning cycles that provide as much insight as product batches. This accelerated the commissioning and validation of those systems.

The migration provided the consistency and reliability needed to provide the plant with the necessary functionality and achieved qualification and validation. This has allowed the pharmaceutical company to author version-controlled, electronic recipes that execute consistently from batch to batch.

Christopher Rudd, CEO, Rayn Business Development; Randy Otto, CEO, ECS Solutions.

MORE ANSWERS

Keywords: system integration, batch manufacturing

ONLINE

See additional system integration stories at https://www.controleng.com/system-integration/

CONSIDER THIS

Have you done an overhaul of your control system and what were the results?