Doing a Double-Take: SCADA replication software refines Carlsberg plant operations

At the Leeds, U.K. brewing & bottling plant of Danish brewing group Carlsberg, an asynchronous fail-over and replication solution from Double-Take Software is keeping production going when server failures take critical systems off-line.

05/09/2008


At the Leeds, U.K. brewing & bottling plant of Danish brewing group Carlsberg , an asynchronous fail-over and replication solution from Double-Take Software is keeping production going when server failures take critical systems off-line.
Brewing three million hectoliters of beer each year—including the company’s popular British brand, Tetley's—and operating 24 hours a day, server failure used to hit production hard.
Over the last five years, explains Senior Systems Engineer Graeme Walker, server failure has knocked out the brewery’s SCADA system twice. On each occasion, he relates, it has taken IT engineers between two and four hours to replace the hardware and reinstate the system. And with its SCADA systems inactive, brewing operations would grind to a halt—with a four-hour interruption to brewing the equivalent of a loss in production of approximately 190,000 liters of beer.
In mid-2007, explains Walker, the decision was made to upgrade the plant’s Intellution SCADA to a more current version—now sold and supported under the GE Proficy banner. But while this contained an improved fail-over capability, the extent of the data replication wasn’t total. Recognizing that the upgrade would require new servers, Carlsberg began to explore other options.
"Given that the fail-over mechanism didn’t meet our requirements, we decided to look at an external solution,” says Walker. “We initially considered a Microsoft cluster, but it seemed very complex and the cost was prohibitive. Instead, Polestar , our IT partner, suggested we take a look at the data replication and fail-over capabilities of Double-Take Software."
A test system was set up to see if the Double-Take solution would deliver the required fail-over capability. A model of the proposed infrastructure was built by establishing an ethernet connection between a production unit on the plant floor and a "target" server located elsewhere in the facility, to which data would be replicated. Using a variety of methods to recreate realistic failure scenarios, Carlsberg engineers then caused the production server to fail—monitoring whether the fail-over process worked, and if all the required data was present.
“In every test, the Double-Take solution failed over and failed back seamlessly. After the trials were completed, we had no hesitation in giving Polestar the green light to implement the full solution,” recalls Walker.
It was an evaluation process that was fairly typical, adds Double-Take’s Worcester, U.K.-based Sales & Marketing Director, Ian Masters. Some of the company’s largest customers are in the manufacturing industry, he notes, with a common requirement being a low server load.
“With asynchronous replication, there’s no real performance impact: The CPU load is just 3 or 4 percent,” says Masters. “Asynchronous replication doesn’t guarantee total replication. Over IP networks, packet loss can happen, but the data on the target system will be usable,‘crash-consistent’ data. We’re hardware- and application-agnostic, and all the customer needs is a Windows operating system.”


Carlsberg Senior Systems Engineer Graeme Walker says asynchronous fail-over and replication software from Double-Take is keeping production going when server failures take critical systems off-line.


Carlsberg’s server upgrade took place in January, relates Walker, with the SCADA upgrade following immediately. From a previous installation that he characterizes as “15 servers and no redundancy”, the plant now has five primary SCADA servers; two secondary servers used for server databases, pictures, and recipes; and two servers dedicated to the role of replication target servers.
And in addition to their back-up role in the event of server failure, the replication servers also offer operational benefits, he adds. Server maintenance, for example, has been greatly simplified.
“If we need to investigate problems, install patches, or implement server or software upgrades, we manually fail over to the target units, leaving us free to work on the production servers without compromising performance,” says Walker. “Another benefit of having two identical datasets is that we are able to run our tape backup from the target servers. This means that the back-up process doesn't interfere with production performance, and we run the backups during standard working hours.”
Carlsberg recently had genuine need to call on its new fail-over capability—just weeks after going live. “Some plant engineers accidentally cut the power supply to a production server,” says Walker. “On the shop floor, the operators didn’t notice a thing.”





No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
Each year, a panel of Control Engineering editors and industry expert judges select the System Integrator of the Year Award winners.
Control Engineering Leaders Under 40 identifies and gives recognition to young engineers who...
Learn more about methods used to ensure that the integration between the safety system and the process control...
Adding industrial toughness and reliability to Ethernet eGuide
Technological advances like multiple-in-multiple-out (MIMO) transmitting and receiving
Virtualization advice: 4 ways splitting servers can help manufacturing; Efficient motion controls; Fill the brain drain; Learn from the HART Plant of the Year
Two sides to process safety: Combining human and technical factors in your program; Preparing HMI graphics for migrations; Mechatronics and safety; Engineers' Choice Awards
Detecting security breaches: Forensic invenstigations depend on knowing your networks inside and out; Wireless workers; Opening robotic control; Product exclusive: Robust encoders
The Ask Control Engineering blog covers all aspects of automation, including motors, drives, sensors, motion control, machine control, and embedded systems.
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
News and comments from Control Engineering process industries editor, Peter Welander.
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
This is a blog from the trenches – written by engineers who are implementing and upgrading control systems every day across every industry.
Anthony Baker is a fictitious aggregation of experts from Callisto Integration, providing manufacturing consulting and systems integration.
Integrator Guide

Integrator Guide

Search the online Automation Integrator Guide
 

Create New Listing

Visit the System Integrators page to view past winners of Control Engineering's System Integrator of the Year Award and learn how to enter the competition. You will also find more information on system integrators and Control System Integrators Association.

Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Control Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.