Achieving on-stream control room relocations
A natural gas processing and compression facility control rooms had to relocate, which caused many challenges in an application with many potential hazards during the process. Project included control panel upgrades, communication digitalization and moving process control and process safety systems.
- A natural gas facility’s control rooms had to be relocated to blast-resistant modules (BRMs).
- Among the many challenges were migrating the hardwired emergency stop (E-stop) circuits to the new control room and control panels with minimal transition time and without a widespread outage.
- Careful planning and factory acceptance test (FAT) helped the team understand how everything worked and minimized confusion.
Control room relocation insights
- Relocating a control room in energy production facilities while staying online is necessary to upgrade capabilities and avoid service disruption, but can include potential hazards.
- Advantages included standardization and modernization including digitalization of automation and process control communications with new programmable logic controllers (PLCs); migrating hardwired emergency stop (E-stop) circuits to the new control room was a particular challenge.
Due to potential blast hazards to plant personnel the control rooms of a natural gas (NG) processing and compression facility had to be relocated to blast-resistant modules (BRMs). The project required control panel upgrades, communication digitalization and moving process control and process safety systems. This complex cutover for personnel protection needed to be completed without shutting down either of two interdependent NG production facilities. A shutdown would result in a process upset at 12 oil production facilities, which relies on the natural gas feed for their operation.
Control room, process control system migration project challenges
The facility feeds large volumes of compressed natural gas to 12 oil production facilities through sections of piping up to 5 feet in diameter with compression being achieved by enormous gas turbines. The number of drawings, valves, interconnecting wires and control schemes that needed to be evaluated was extensive and complex.
Normal gas operation controls such as turbine controls, panel boards, and valve interfaces such as the main gas header valves feeding the plant had to be migrated which included basic process control systems (BPCS), safety systems, fire & gas (F&G) systems, halon fire suppressant circuits and emergency shutdown (ESD) circuits.
A misstep on these critical process control and plant safety systems could have brought the facility down, which would have cascaded to downstream facilities that rely on the natural gas for operations. Furthermore, the NG facility equipment spanned 50 years of modifications, and many upgrades which were needed to universalize and modernize the many generations of equipment, including digitalizing hardwired signals into a new programmable logic controller- (PLC) based BPCS.
E-stop migration, process isolation safety complications
The greatest challenge was migrating the hardwired emergency stop (E-stop) circuits to the new control room with minimal transition time and without a widespread outage. In fact, because of the hazards and complexities involved in a complete shut down and restart, the NG facility had only been shut down twice since being placed into service many decades earlier, and it had been over a decade since the last complete outage. E-stops are used by operators as a last resort in response to a critical process excursion. The E-stop circuit, when tripped by the operators, opens valves to the flare relief system, closes isolation valves, and shuts down running equipment. Complicating matters is the fact E-stop circuits are fail-safe designed, which means if any circuit is opened, the E-stop actions are triggered.
Project lessons learned: System integration project management
It became apparent having a single point of contact for project management was key to planning and developing a cutover strategy starting from the early appraisal stages of the project to the final execution stage. Detailed advanced planning was needed with careful consideration to the scope of this complicated project and the numerous stakeholders involved. Any shortcut or attempt to skip a project step could have had potential consequences to an effective cutover, so understanding the big picture and high-level activity as well as delegating specific tasks to specific people from a central focal point became essential for success. Channeling communication through single points of contact ensured all input, concerns and feedback were received and addressed.
Understanding the many stakeholders involved was important. The project team was an extensive network of people with different backgrounds and responsibilities, and the systems integrator helped coordinate and integrate the needs and influences from affected stakeholders. All groups needed to collaborate to identify long-term operability constraints and/or mitigate potential future operability issues with the reconfigured system. Integrating moving parts to strategize the project execution was key to avoiding siloed groups of disciplines.
Six steps for project success included control system, safety system cutover, factory acceptance test, control panel upgrade
The system integrator designed the control room and took responsibility for all stages of the project, managing tasks from conceptual and preliminary engineering to detailed design and managing on-site activity and personnel during the complex cutover. Beginning with a comprehensive survey of the existing facilities, the systems integrator cataloged equipment and systems that needed to be relocated, identified stakeholders and resources involved, and then created a detailed plan for cutover without requiring plant downtime. The six steps for the project were:
Performed a detailed option analysis to evaluate potential best solutions.
Analyzed existing terminations and every wire to understand what it does and temporary bypasses to be put into place; maintained a well-documented log to track what had been completed.
Developed step-by-step cutover procedures and checklists and an integrated schedule that identified specific people and detailed daily tasks over the year leading up to the cutover of both NG facilities. Impacted stakeholders included facility engineers, operators, support staff, design team, contractors, vendors, construction crew, and IT networking crew. A comprehensive responsibility matrix was maintained to coordinate all tasks required.
Developed a contingency plan to guard against emergency events and mitigate risk of valves changing position during cutover process for safety purposes; had personnel stationed at all the critical valves during the cutover process to monitor and respond to potential process hazards; processing natural gas is an exercise in precision, and even the smallest changes in pipeline pressure, flow rates, temperature, and gas composition can have huge impacts.
Set up a local integration center, i.e., factory acceptance test (FAT) lab, to convene equipment and everyone involved and determined ways to install additional panels. They also designed, fabricated, tested and provided new control panels needed for the PLC and control systems as well as safety systems such as E-stops and F&G system. They also developed a scheme of transitioning existing terminations to new termination locations without causing a shutdown and prepared all personnel involved within the FAT lab to ensure a smooth cutover process during implementation in the field.
Managed on-site activities helping direct the cutover process while implementing the cutover plan, including directing facility operators, engineers, support staff, construction and information technology (IT) staff.
Project results: Project management, standardization, control panels
The systems integrator’s role evolved from system integrator to becoming the focal point of coordinating schedules and staffing and choreographing the entire cutover project. aeSolutions started by surveying the existing equipment and systems. By having a single point of contact for project management, the client benefited from standardization of approaches and technology, a higher degree of coordinated activity, and a shortened timeframe. The integrator developed an execution strategy and found ways to integrate multiple teams into the plan while handling the conceptual engineering, preliminary engineering, detailed design, and manufacturing and implementing the required components.
The local FAT lab was instrumental in enabling the team to confirm how equipment worked, verifying detailed FAT plans and test procedures, and developing methods to (dis)connect test panels with everyone in the same room. Equipment was evaluated during and after the assembly process to verify it was built and operated in accordance with design specifications. The FAT lab provided an off-site opportunity to reconcile and unify all the moving parts involved with the unit before the equipment was shipped to the site. This integration strategy saved time and travel costs to cutover the controls on a NG operating plant without impact to production. In the end the project achieved its two objectives: 1) the NG facility employees were relocated to the new BRMs and 2) the project was completed without a shutdown or process upset of the operating facilities.
Lessons for future projects: Front-end loading, codes and standards
As a result of the success of this project, the integrator established a project development team dedicated to scope development and planning for complex projects. This team is involved in the early stages of complex projects to guide clients to a project vision, identify risks, advise on codes and standards, develop and clarify scope, and consult and check in on project progress.
This upfront involvement with focus on front-end loading (FEL) what’s practical, paves the road for project managers and engineers to deliver on the project vision, manage the risks, scope, schedule, budget, and expectations. It also coordinates across disciplines.
In turn, the work can be executed by the technical leads and discipline specialists with minimal obstacles and yield a maximum quality product that is safe, intuitive, reliable, maintainable, fit for purpose and adheres to standards and best practices.
Ted Hoffman, PE, senior project development engineer; Charles Mills, automation control specialist, both with aeSolutions. Edited by Chris Vavra, web content manager, CFE Media and Technology, firstname.lastname@example.org.
Keywords: control room migration, system integration
See additional system integration stories at https://www.controleng.com/system-integration/
What major system integration projects have you implemented and what were the lessons?