Project: Biopharmaceutical filtration automation-December 19, 2006
December 19, 2005
Project Status Summary:
1. Control software development code complete 100%
2. HMI software code complete 100%
3. HMI external application configuration 100%
4. MF beta testing 90%
5. UF beta testing 90%
Customer Acceptance Testing
We completed the majority of the customer acceptance testing this week, which is a significant milestone for the project. The customer testing is always one of the most exciting, yet stressful, events for any automation project. For weeks now, we have been plugging away at the implementation based on evolving requirements and our running dialogue with the customer. Acceptance testing is the point where the end users get to see the system come alive for the first time– and see what all the automation specifications, theories, and hard work ultimately yielded.
Two different client groups are performing the testing. Last week, the process engineers performed functional testing on the system, along with verifying the look and feel of the graphics and recipe editor. This week, the customer’s automation engineer will perform more “under the hood” testing and will also work with us on the plan for integrating the data to their central servers. The process team spent three days at our office to complete their tests and it is expected the automation engineer will spend another two at our office next week. The five-day total is rather typical for a system of this size and scope.
The first part of any FAT (factory acceptance test) is the initial training of the engineers so they can familiarize themselves with the actual implementation. Although the faceplates for the equipment modules and control modules are rather intuitive, it takes a bit of hands on use to understand all of the capabilities and features consolidated in each. After gaining familiarity with the system, we tested each equipment module for proper functionality. The equipment modules were run both individually and concurrently, mimicking process conditions as much as possible. The testing is enhanced with the use of our simulator, which simulates the analog values in a response representative to the process conditions. The simulation is key for proper testing, for it allows the process to play out as it might normally do in the field, and allow the testers to see actual conditions unfold. For example, there may be significant or subtle differences in processing when a tank drains at its normal rate, versus simply driving the analog level value from 100% to 0%. The simulator is yet another tool employed to help ensure success at the field start-up and prevent code changes in the future. The extensive equipment module testing was performed over a two-day period and uncovered some minor changes to the sequence of operations, which is expected for any acceptance testing. For the scope of the project (78 equipment modules), the amount of sequence changes was remarkably low– less than five!
The graphics and interfaces were also examined during testing and were accepted with minimal changes. This is an area where customer preferences come into play much more than any specification or document. Most end users are very particular about the look and feel of the piping diagrams, the color schemes, the fixed text and dynamic text on their system. Our graphics, dynamos and general layout have been evolving over the years to where now we have a rather fine tuned package. We have developed graphical interfaces that maximize the information with minimal screen footprint. It is also critical not to clutter process screens. The use of features such as tags on/off and displaying of device mode states by exception (only display if not in auto) help keep the screens clean and easy to read.
Another key aspect to acceptance testing is the documentation review. During our in-house development testing, each tested module is signed and dated per cGMP (current good manufacturing practices) documentation requirements and compiled into a completed test document. During the visit, it is essential that the customer reviews and accepts the testing documentation that has been generated; it is their record that each line of code has been tested and acceptably documented. Some customers prefer to perform some of the testing themselves and sign off on the documentation, or in this case, the functional testing acted as a spot check on our work.
The customer’s automation engineer will visit our San Francisco office to finish acceptance testing; meanwhile some system components will be shipped to the equipment fabricator for incorporation into the filter skid electrical enclosures. We now begin system clean-up per the acceptance testing punch list before we hit the road for the factory test at the equipment fabricator’s shop.
This blog will not be updated again until the week of January 9, when work resumes on the project following the holiday season.